| Description | Load replicated data into MySQL or MariaDB |
| Type | RDBMS Destination |
| Supported Replication | Initial Sync, Continuous Sync |
| Authentication Type | Password Authentication |
Overview
Integrate.io ELT & CDC syncs data to MySQL using a two-stage bulk-load pattern. Batches are streamed into a temporary staging table withLOAD DATA LOCAL INFILE, then merged into the destination tables using inserts, updates, and deletes. No files are written on either side: the data streams directly over the connection.
Because staging uses session TEMPORARY tables, there are no orphan staging tables to clean up. The destination database doubles as the schema, so there is no separate schema field to configure.
Requirements
- MySQL 5.5 or later, or MariaDB 10.x.
local_infileenabled on the server (covered in the steps below).- An admin user for the one-time setup (your RDS master user works).
- Network access from Integrate.io’s egress IPs, or an SSH tunnel.
Create sync user
Create the sync user. Run the following as an admin user (your RDS master user works):Grant database privileges
Grant the sync user privileges on the destination database. The sink creates and alters destination tables, stages batches inTEMPORARY tables, and merges with inserts, updates, and deletes. Specify the database you want the data synced to and run:
Required privileges
| Privilege | Why it’s needed |
|---|---|
SELECT, INSERT, UPDATE, DELETE | Read and merge replicated rows into destination tables. |
CREATE, DROP, ALTER | Create and evolve destination tables as the source schema changes. |
CREATE TEMPORARY TABLES | Stage each batch in a session temporary table before merging. |
Enable local_infile
The sink loads data withLOAD DATA LOCAL INFILE, which requires local_infile to be enabled on the server.
-
Self-managed MySQL: run the statement below.
-
Managed MySQL (Amazon RDS, Aurora, Google Cloud SQL): set
local_infile = 1in the server’s parameter group instead.
If
local_infile is disabled, the connection test fails with a remediation message. Enable it before saving the connection.Features
| Feature | Supported | Notes |
|---|---|---|
| Full (Historical) sync | Yes | |
| Incremental sync | Yes | |
| UPSERT | Yes | Last write wins, keyed by primary key. |
| Replicate DELETE | Yes | |
| Append only mode | Yes | Same-batch replays are de-duplicated. |
| SSH tunnel | Yes | SSH Tunnel Guide |
Limitations
- String primary keys are stored as
VARCHAR(191)to stay within the InnoDB 767-byte key limit on the MySQL 5.5 and 5.6 floor. Primary key values longer than 191 characters are truncated at load, and prefix-colliding keys may collapse into a single row. VARCHARprimary keys longer than 191 characters require a MySQL 5.7 or later destination.
Frequently Asked Questions (FAQs)
Why does the connection test fail with a local_infile error?
Why does the connection test fail with a local_infile error?
The sink loads data with
LOAD DATA LOCAL INFILE, which requires local_infile to be enabled on the server. Run SET GLOBAL local_infile = 1; on self-managed MySQL, or set local_infile = 1 in the parameter group on managed MySQL (RDS, Aurora, Cloud SQL), then test again.Do I need to create the destination database first?
Do I need to create the destination database first?
No. If you grant the sync user
CREATE privileges, the CREATE DATABASE IF NOT EXISTS step creates the database for you. The destination tables are created and altered automatically as data syncs.Is there a separate schema to configure?
Is there a separate schema to configure?
No. In MySQL the schema is the database, so there is no separate schema field. Data syncs into the database you specify in the grant step.
Does the sink leave temporary staging tables behind?
Does the sink leave temporary staging tables behind?
No. Staging uses session
TEMPORARY tables on a pinned connection per batch, so there are no orphan tables to clean up.What MySQL versions are supported?
What MySQL versions are supported?
MySQL 5.5 or later, and MariaDB 10.x. Some features, such as
VARCHAR primary keys longer than 191 characters, require MySQL 5.7 or later.