Overview
ELT & CDC uses Snowflake compute resource in auto suspend and auto-resume mode to run queries to load data from an external stage. You have an option to either run a ELT & CDC dedicated compute warehouse and provide a database for ELT & CDC to load data or choose an existing warehouse and database to load.
Using an existing warehouse is recommended to minimise compute resources usage and cost for loading data from any source to Snowflake
Requirements
- Any one of the ACCOUNTADMIN, SYSADMIN or SECURITYADMIN access for running the prerequisite/setup steps.
Setup script
Copy the user script and run on your snowflake worksheet.
Copy the role script and run on your snowflake worksheet.
Copy the integration script and run on your snowflake worksheet.
Register your Snowflake Account Identifier, Warehouse, Username, Password, Role, Database and Schema on the dashboard
Note that the all details here (username, password, etc) should be the same with the one you executed on your snowflake worksheet. Input your snowflake identifier per dashboard instruction.
How to get your Snowflake Account Identifier:
- Via classic console: from the url eg: https://xxxxxxx.us-east-1.snowflakecomputing.com)
xxxxxxx.us-east-1 --> this is the value you need to input in the dashboard.
- Via Snowsight web interface: go to your Account > Copy account URL
Snowflake Network Policy
If you have an existing Network Policy, please modify your network policy add ELT&CDC IP Addresses
Note
- The maximum allowed VARCHAR/TEXT length of single-byte characters is 16,777,216 and 4,194,304 for 4 byte characters (per snowflake's documentation).
- Snowflake advice not to use ACCOUNTADMIN when creating objects. If database exists, make sure it is under the SYSADMIN role.