Skip to main content
Use the PostgreSQL (Snapshot CDC) source component to read only the records that have changed in your PostgreSQL tables since the previous pipeline run, instead of reprocessing the entire table every time. The component produces two outputs: Upserted records (new and updated rows) and Deleted records (rows removed from the source).

​
Connection Setup

Before reading changes, set up your PostgreSQL connection. See Connecting to PostgreSQL for the credentials and network access required. Once the connection exists, add the PostgreSQL (Snapshot CDC) source component to your pipeline and select it.

​
Snapshot Storage

PostgreSQL Snapshot CDC uses File Based snapshot storage. The snapshot of your previous run is stored as a Parquet file on Integrate.io managed cloud storage, which means:
  • Only read access to your PostgreSQL database is required.
  • No snapshot tables are created in your source database.
  • On the first run, every current record is treated as upserted because there is no previous snapshot to compare against.
Database (table-based) snapshot storage is only available for SQL Server connections. For PostgreSQL, File Based storage is the only option, and you do not need to configure an S3 connection of your own.

​
Change Detection

Choose how the component decides whether a row has changed:
  • Primary Key: matches rows by a unique identifier column (for example id or order_id) and detects updates by comparing the remaining column values. Best when the table has a reliable key.
  • Composite Hash: builds a hash from all or selected columns and compares hashes between runs. Best when the table has no reliable primary key.
With File Based storage you can also switch the Source Mode from Table to Query to track changes across a custom SQL query, including JOINs across multiple tables.

​
Example

Sync inventory changes to a downstream system:
  • Source table: inventory
  • Method: Composite Hash
  • Custom columns: sku, quantity, warehouse_id
  • Upserted records: update the inventory management system
  • Deleted records: remove discontinued items

​
Full Configuration Reference

Snapshot CDC Source reference

Configuration steps, change detection methods, query mode, best practices, troubleshooting, and limitations that apply to every Snapshot CDC source.
Last modified on June 29, 2026