Chord Data Source Ingestion Guidelines

chord supports data source connections outside of snowflake note that chord's schema expectations and iam configurations can vary by cloud provider find instructions on how to connect your bigquery data source to chord below we'll be adding more documentation about additional data sources in the coming weeks have immediate follow up questions or need additional guidance? reach out to help\@chord co ! export data to a gcs bucket via bigquery brands can export data to a gcs bucket with appropriate iam permissions note that chord grabs the data from the gcs bucket, not directly from bigquery this recommendation only works for moving data from bigquery to a gcs bucket, but azure and aws also have cloud storage buckets here is more information about the required iam configurations, which vary by cloud provider aws gcp azure specifically for gcp, you need to grant chord’s service account access to gcp buckets containing your oms data follow the directions in step 3 grant the service account permissions to access bucket objects chord’s gcp service account is klk920000\@awsuseast2 a5c7 iam gserviceaccount com once you have configured this, please provide chord with the in scope bucket names to which the policies are applied recommended data structure within the gcs bucket additionally, here’s some requirements on how the data should be structured in the gcs bucket brands should provide a backfill of historical data and then, moving forward, also provide incremental files that represent new and changed records when processing a high volume of files from blob storage, it’s helpful to name files according to a hierarchy that can be easily queried when landing data in gcs, please use the following key format \<source>/\<collection>/\<year>/\<month>/\<day>/\<hh\ mm\ ss>/\<partition> \<id> parquet here’s a specific example of what this might look like oms/orders/2025/08/11/20 29 25/2025 08 10 14 00 e791c45e parquet this means the file is the orders collection from the oms data source it was uploaded to gcs on 2025 08 11 at 20 29 25 utc and the data is for the hour of 2025 08 10 14 00 it’s also helpful to include a unique identifier for the file, in this case shortened guid e791c45e corresponds to the compute process that produced this file note for brands using bigquery if brands are using bigquery as their data source, brands can use an export data statement to move data from bigquery into a gcs bucket