Skip to main content

​
Overview

Integrate.io ELT & CDC replicates your data to Databricks as Delta tables in Unity Catalog. Data is loaded in periodic batches and merged into the target tables based on the primary key, so each table always reflects the latest state of your source. ELT & CDC connects to Databricks through a SQL warehouse and loads data into the catalog and schema you choose. You provide the warehouse connection details, a catalog and schema, and a personal access token.

​
Requirements

  • A running Databricks SQL warehouse. You need its Server Hostname and HTTP Path.
  • A Unity Catalog catalog (or hive_metastore) and a target schema.
  • A personal access token belonging to a user or service principal that holds the Unity Catalog privileges listed below.
For production, use a service principal token rather than a personal user token so the connection keeps working if the user leaves.

​
Setup

​
Connection details

In your Databricks workspace, open SQL Warehouses and select the warehouse you want to use. On the Connection details tab, copy the Server Hostname and HTTP Path into the Integrate.io destination form.

​
Personal access token

Integrate.io authenticates with a personal access token (PAT).
  1. In Databricks, click your username (top right) → Settings → Developer. Next to Access tokens, click Manage → Generate new token.
  2. Enter a Name and a Lifetime (days). The token stops working when it expires, so pick a long lifetime and rotate before then, or use a service principal token.
  3. If a Scope selector appears, choose BI Tools. If you pick Other APIs, select the sql scope. No other API scopes are needed.
  4. Click Generate and copy the token immediately. Databricks shows it only once.
See Databricks personal access token authentication for details.

​
Grant Unity Catalog privileges

The token’s user or service principal needs privileges on the target catalog and schema to create and load tables. If it is already a workspace or metastore admin, or owns the catalog, you can skip this step. No storage credential or external location is required. Run the following queries as a metastore admin or the catalog owner. Replace <principal> with the token’s identity (a user email, service principal application ID, or account group name), and <catalog> and <schema> with your target catalog and schema. 
-- Catalog access (always required):
GRANT USE CATALOG ON CATALOG <catalog> TO `<principal>`;
GRANT CREATE SCHEMA ON CATALOG <catalog> TO `<principal>`;

-- If the schema already exists, also grant on it
-- (skip if Integrate.io creates it; the creator owns the schema):
GRANT USE SCHEMA, CREATE TABLE, MODIFY, SELECT ON SCHEMA <catalog>.<schema> TO `<principal>`;

​
Destination credentials

Enter the Server Hostname, HTTP Path, personal access token, catalog, and schema in the Integrate.io destination form.

​
Databricks IP access list

If your workspace enforces IP access lists, add the ELT & CDC IP addresses to the workspace allowlist.

​
Required privileges

PrivilegeScope
USE CATALOGCatalog
CREATE SCHEMACatalog
USE SCHEMASchema
CREATE TABLESchema
MODIFYSchema
SELECTSchema

​
VARIANT columns

By default, JSON, object, and array columns are stored as STRING for broad runtime compatibility. If your SQL warehouse runs Databricks Runtime 15.3 or later, you can opt into the VARIANT type for these columns. Contact support to enable this option on your destination.

​
Schema evolution

New columns added to source tables are added to the destination Delta table with ALTER TABLE ADD COLUMNS. Column type changes are not applied automatically.

IP Allowlist

Initial Sync Process

Last modified on July 7, 2026