Skip to content

Commit e16cb1e

Browse files
adithyasamavedhi-ttdadithyasamavedhi-ttd
committed
relove-PR-comments-on-example-notebook
1 parent 46ec0d9 commit e16cb1e

1 file changed

Lines changed: 44 additions & 10 deletions

File tree

‎example_notebook/TTD Connector Data SDK Example Notebook.ipynb‎

Lines changed: 44 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,10 @@
2222
"metadata": {},
2323
"outputs": [],
2424
"source": [
25-
"%pip install ttd-databricks"
25+
"%pip install ttd-databricks\n",
26+
"\n",
27+
"# Recommended to restart kerner to use updated packages\n",
28+
"dbutils.library.restartPython()"
2629
]
2730
},
2831
{
@@ -104,7 +107,22 @@
104107
{
105108
"cell_type": "markdown",
106109
"metadata": {},
107-
"source": "## Step 4: Inspect the required input schema\n\nUse `get_ttd_input_schema()` to see which columns your DataFrame must contain.\n\nFor more information on the meaning of particular fields and supported data types supported per endpoint refer to the following table:\n\n| Endpoint | Context | Data API | Documentation |\n|---|---|---|---|\n| Advertiser | `AdvertiserContext` | `POST /data/advertiser` | [openTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-advertiser-external) |\n| Third Party | `ThirdPartyContext` | `POST /data/thirdparty` | [openTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-thirdparty) |\n| Offline Conversion | `OfflineConversionContext` | `POST /providerapi/offlineconversion` | [openTTD](https://open.thetradedesk.com/advertiser/docsApp/GuidesAdvertiser/data/doc/post-providerapi-offlineconversion) |\n| Deletion / Opt-Out — Advertiser | `DeletionOptOutAdvertiserContext` | `POST /data/deletion-optout/advertiser` | [openTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-deletion-optout-advertiser-external) |\n| Deletion / Opt-Out — Third Party | `DeletionOptOutThirdPartyContext` | `POST /data/deletion-optout/thirdparty` | [openTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-deletion-optout-thirdparty) |\n| Deletion / Opt-Out — Merchant | `DeletionOptOutMerchantContext` | `POST /data/deletion-optout/merchant` | [openTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/retail/doc/post-data-deletion-optout-merchant) |"
110+
"source": [
111+
"## Step 4: Inspect the required input schema\n",
112+
"\n",
113+
"Use `get_ttd_input_schema()` to see which columns your DataFrame must contain.\n",
114+
"\n",
115+
"For more information on the meaning of particular fields and supported data types supported per endpoint refer to the following table:\n",
116+
"\n",
117+
"| Endpoint | Context | Data API | Documentation |\n",
118+
"|---|---|---|---|\n",
119+
"| Advertiser | `AdvertiserContext` | `POST /data/advertiser` | [OpenTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-advertiser-external) |\n",
120+
"| Third Party | `ThirdPartyContext` | `POST /data/thirdparty` | [OpenTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-thirdparty) |\n",
121+
"| Offline Conversion | `OfflineConversionContext` | `POST /providerapi/offlineconversion` | [OpenTTD](https://open.thetradedesk.com/advertiser/docsApp/GuidesAdvertiser/data/doc/post-providerapi-offlineconversion) |\n",
122+
"| Deletion / Opt-Out — Advertiser | `DeletionOptOutAdvertiserContext` | `POST /data/deletion-optout/advertiser` | [OpenTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-deletion-optout-advertiser-external) |\n",
123+
"| Deletion / Opt-Out — Third Party | `DeletionOptOutThirdPartyContext` | `POST /data/deletion-optout/thirdparty` | [OpenTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-deletion-optout-thirdparty) |\n",
124+
"| Deletion / Opt-Out — Merchant | `DeletionOptOutMerchantContext` | `POST /data/deletion-optout/merchant` | [OpenTTD](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/retail/doc/post-data-deletion-optout-merchant) |"
125+
]
108126
},
109127
{
110128
"cell_type": "code",
@@ -114,7 +132,9 @@
114132
"source": [
115133
"input_schema = get_ttd_input_schema(TTDEndpoint.ADVERTISER)\n",
116134
"print(\"Required input schema:\")\n",
117-
"print(input_schema.treeString())"
135+
"\n",
136+
"for field in input_schema.fields: \n",
137+
" print(f\" {field.name}: {field.dataType.simpleString()} (nullable={field.nullable})\")"
118138
]
119139
},
120140
{
@@ -136,13 +156,13 @@
136156
"# Example: create a small sample DataFrame\n",
137157
"# In practice, read from a Delta table or other data source\n",
138158
"sample_data = [ \n",
139-
" {\"id_type\": \"tdid\", \"id_value\": \"a3f1c2d4-8e7b-4f6a-9c0d-1b2e3f4a5b6c\", \"segment_name\": \"segment_1\"}, \n",
140-
" {\"id_type\": \"daid\", \"id_value\": \"7d9e0f1a-2b3c-4d5e-6f7a-8b9c0d1e2f3a\", \"segment_name\": \"segment_2\"},\n",
159+
" {\"id_type\": \"tdid\", \"id_value\": \"a3f1c2d4-8e7b-4f6a-9c0d-1b2e3f4a5b6c\", \"segment_name\": \"segment_1\"}, \n",
160+
" {\"id_type\": \"daid\", \"id_value\": \"7d9e0f1a-2b3c-4d5e-6f7a-8b9c0d1e2f3a\", \"segment_name\": \"segment_2\"},\n",
141161
" # intentionally incorrect format for ramp_id to showcase error enrties in output \n",
142162
" {\"id_type\": \"ramp_id\", \"id_value\": \"c4d5e6f7-a8b9-4c0d-1e2f-3a4b5c6d7e8f\", \"segment_name\": \"segment_3\"}, \n",
143-
" {\"id_type\": \"tdid\", \"id_value\": \"1f2a3b4c-5d6e-4f7a-8b9c-0d1e2f3a4b5c\", \"segment_name\": \"segment_4\"}, \n",
144-
" {\"id_type\": \"daid\", \"id_value\": \"9b0c1d2e-3f4a-4b5c-6d7e-8f9a0b1c2d3e\", \"segment_name\": \"segment_5\"}, \n",
145-
" ] \n",
163+
" {\"id_type\": \"tdid\", \"id_value\": \"1f2a3b4c-5d6e-4f7a-8b9c-0d1e2f3a4b5c\", \"segment_name\": \"segment_4\"}, \n",
164+
" {\"id_type\": \"daid\", \"id_value\": \"9b0c1d2e-3f4a-4b5c-6d7e-8f9a0b1c2d3e\", \"segment_name\": \"segment_5\"}, \n",
165+
"] \n",
146166
" \n",
147167
"input_df = spark.createDataFrame(sample_data) \n",
148168
"display(input_df)"
@@ -268,7 +288,21 @@
268288
"execution_count": null,
269289
"metadata": {},
270290
"outputs": [],
271-
"source": "# Run batch processing (reads from input_table, writes to output_table)\n\n# process_new_records_only=True filters to rows where updated_at > last run date\n# On the first run, metadata_table is empty so all rows are processed\nclient.batch_process(\n context=context,\n input_table=input_table,\n output_table=output_table,\n metadata_table=metadata_table,\n process_new_records_only=True, # Processes rows updated after last run; processes all rows on first run\n batch_size=1600, # Number of rows grouped together in a single request to The Trade Desk\n parallelism=16, # Number of paralellel workers processing the entries from the input table\n)"
291+
"source": [
292+
"# Run batch processing (reads from input_table, writes to output_table)\n",
293+
"\n",
294+
"# process_new_records_only=True filters to rows where updated_at > last run date\n",
295+
"# On the first run, metadata_table is empty so all rows are processed\n",
296+
"client.batch_process(\n",
297+
" context=context,\n",
298+
" input_table=input_table,\n",
299+
" output_table=output_table,\n",
300+
" metadata_table=metadata_table,\n",
301+
" process_new_records_only=True, # Processes rows updated after last run; processes all rows on first run\n",
302+
" batch_size=1600, # Number of rows grouped together in a single request to The Trade Desk\n",
303+
" parallelism=16, # Number of paralellel workers processing the entries from the input table\n",
304+
")"
305+
]
272306
},
273307
{
274308
"cell_type": "code",
@@ -297,4 +331,4 @@
297331
},
298332
"nbformat": 4,
299333
"nbformat_minor": 4
300-
}
334+
}

0 commit comments

Comments
 (0)