Amazon RDS MySQL source for ELT & CDC

Create Read-Replica (optional)

We recommend syncing data using a read-replica of your database to ensure sync doesn’t affect your primary database instance. If you wish to sync against Primary, feel free to skip this step.Refer this documentation to create a read-replica.

Create user

Create a sync user for Integrate.io by executing,

CREATE USER 'flydata'@'%' IDENTIFIED BY '<your_password>';

Grant necessary privileges,

GRANT SELECT, RELOAD, REPLICATION SLAVE, REPLICATION CLIENT, LOCK TABLES ON *.* TO 'flydata'@'%';
FLUSH PRIVILEGES;

Enable binary logging

Create a DB Parameter Group and set the following parameters:

Parameter	Value
binlog_format	ROW
binlog_row_image	FULL
binlog_expire_logs_seconds	604800 (Minimum 1 day or 86400)

These are recommended values for replication but not required. Set the recommended values if you encounter timeouts.

Parameter	Value
net_read_timeout	3600
net_write_timeout	3600
wait_timeout	86400

Assign the newly created parameter group to your database and reboot your database instance. Confirm your changes by checking the binlog status,

SHOW GLOBAL VARIABLES LIKE 'log_bin';

Increase binlog retention

To increase the binlog retention of the DB cluster, use the mysql_rds_set_configuration procedure. You can run the following command and sample parameters on the writer instance to retain the binlog files for 7 days (Minimum 1 day)

CALL mysql.rds_set_configuration('binlog retention hours', 168);

Getting Started

Pipelines

Sources

Destinations

Monitoring

Security

Transforms

Data Observability

Amazon RDS MySQL source for ELT & CDC

Create Read-Replica (optional)

Create user

Enable binary logging

Increase binlog retention

​Create Read-Replica (optional)

​Create user

​Enable binary logging

​Increase binlog retention

Create Read-Replica (optional)

Create user

Enable binary logging

Increase binlog retention