Configure the REST API destination component in Integrate.io ETL to send pipeline data to any HTTP endpoint with custom headers, auth, and batching.
The REST API destination sends data from your pipeline to any HTTP endpoint. You can configure the request method, headers, body format, authentication, and batching to match your target API’s requirements.
Choose one of the following authentication options:
None - No authentication. Use this when the target API requires no credentials or when you pass authentication through custom headers.
Basic - HTTP Basic authentication. Provide a Username and Password that are sent as a Base64-encoded Authorization header with each request.
Connection - Use a saved connection for SSH tunnel access. When an SSH tunnel is active, HTTPS URLs are automatically rewritten to route through the tunnel’s local port.
Enter the target API endpoint URL. You can include dynamic values from your input data using the #{fieldName} syntax. For example:
https://api.example.com/contacts/#{contact_id}
The #{contact_id} placeholder is replaced with the value of the contact_id field from each input record. All field references in the URL are validated against the input schema.
Add custom headers as name/value pairs. Header values also support the #{fieldName} syntax for dynamic values. Common headers like Content-Type are set automatically based on your chosen request body format, but you can override them here.
Enable Custom Payload to define your own request body template instead of using the automatic field-to-body mapping. The template is a raw string where you can reference input fields. When the body format is JSON, the custom payload is validated for correct JSON syntax.
Enable Send records as nested attribute to wrap your record data inside a named object. For example, if you set the nested attribute name to records, the request body becomes:
Controls how input records are grouped into HTTP requests:
Single per record - Sends one request per input record. Use this when the target API expects individual records (most common for create/update endpoints).
Batch - Groups records into batches of a specified size and sends one request per batch. Set the Batch size to control how many records are included in each request. The batch size must be greater than 0.
One request - Sends all records in a single request.
Set a delay (in milliseconds) between requests when using Single per record mode. Valid range: 0 to 60,000 ms. This is useful for respecting API rate limits.
When enabled, the job fails if any API request returns an error response (HTTP 4xx or 5xx status). When disabled, the job continues processing remaining records even if individual requests fail.
Enable this to write API response data to file storage. Configure the Response file path (an HDFS-compatible path) and Response file format to control where and how responses are saved. This is useful for logging, auditing, or downstream processing of API responses.
