chore: add support for multi-source platforms in snowflake connectors [CM-1028]

[mbani01]

This pull request introduces a significant refactor to the Snowflake export pipeline to support multiple data sources per platform, enabling more granular and flexible data exports. The changes include new abstractions for data sources, updates to workflows and activities to handle source names, and improvements to S3 path structure and error handling.

Multi-source platform support and abstractions:

• Introduced the DataSource abstraction and DataSourceName enum in integrations/types.ts, allowing each platform to define multiple exportable sources, each with its own query and transformer. Platform definitions now include a sources array instead of a single transformer/query. [1] [2]
• Updated the Cvent integration to use the new multi-source structure, with event-registrations as the first defined data source. [1] [2]

Workflow and activity changes for source names:

• Refactored the export workflow to iterate over all source names for each enabled platform, launching a separate child workflow for each source/platform combination. The workflow and activity signatures now include sourceName as a parameter. [1] [2] [3] [4] [5]
• Added the getDataSourceNamesForPlatform activity and supporting integration methods to enumerate available sources for a platform. [1] [2] [3]

S3 path and transformer improvements:

• Updated S3 path construction to include the source name, ensuring exported files are organized by both platform and source. Added a utility to parse the source name from S3 paths for downstream processing. [1] [2]
• Modified the transformer consumer to resolve the correct data source and transformer based on the parsed source name from the S3 path, improving robustness and flexibility. [1] [2]

Other improvements:

• Improved error handling and logging throughout the export and transformer processes, including more detailed context in log messages and stricter error checks. [1] [2]
• Minor fix to only call the batch completion callback if rows were exported in SnowflakeExporter.

---

Note

Medium Risk
Refactors the Snowflake export/transform pipeline and S3 path conventions to be source-aware; incorrect source naming or path parsing could break exports/consumers or create duplicate/missing jobs.

Overview
Adds multi-source support per platform by introducing DataSource/DataSourceName and changing platform definitions to expose a sources list, plus new lookup helpers (getDataSource, getDataSourceNames).

Updates the Temporal workflow and export activity to run exports per (platform, source), include sourceName in workflow IDs/args, and write S3 files under {bucketPath}/{platform}/{sourceName}/... with improved logging context.

Adjusts the transformer consumer to select the correct transformer by parsing sourceName from the S3 path, and tweaks SnowflakeExporter to only invoke onBatchComplete when a batch actually exported rows.

^{Written by Cursor Bugbot for commit 66e0f4c. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

[cursor]

Migration fails if non-cvent rows exist in table

Low Severity

The UPDATE on line 3 only sets sourceName for rows WHERE platform = 'cvent', but line 5 then applies a NOT NULL constraint to all rows. If any rows exist in snowflakeExportJobs with a platform value other than 'cvent', those rows will still have NULL for sourceName, and the ALTER COLUMN SET NOT NULL will fail, blocking the migration. Updating all rows unconditionally (or using a DEFAULT on the column) would be more resilient.