The Log Drain tab is being rolled out gradually. If you do not see it in your account settings, contact Integrate.io support to enable it for your account.
What Gets Delivered
A log drain can subscribe to three events.
A succeeded job carries no log body because Integrate.io ETL only writes a job log when something goes wrong. That is expected, not a delivery fault.
Workflows
When a workflow finishes, the drain delivers a record for each job the workflow ran, not for the workflow itself, since a workflow orchestrates other jobs and has no log of its own. Succeeded jobs deliver a summary, failed jobs deliver their full log. Only the workflow’s direct children are covered. If a workflow runs another workflow, the inner workflow’s jobs are not included.Record attributes
Integrate.io ETL sends OTLP/HTTP protobuf, gzip compressed, to your endpoint’s/v1/logs path. Each delivery carries attributes you can filter and group on in your observability platform:
The drain token identifies the drain in delivered records. It is not a secret and is safe to share with support.
Sensitive Data
Every log is scrubbed before it leaves Integrate.io ETL. Credentials are replaced with<REDACTED>, and the surrounding context is kept so the log stays readable:
key=value secrets, AWS access key IDs, JSON web tokens, and the secret variables defined on your package. It matches known credential formats, so treat it as a safety net rather than a guarantee, and avoid printing secrets from custom code.
Large Logs
A single delivery is capped at 4 MB. If a job log exceeds that, Integrate.io ETL keeps the newest lines, because that is where the stack trace usually is, and inserts a marker where content was removed:Creating a Log Drain
Log drains are defined at the account level by account administrators. You can create up to 3 log drains per account.1
Click your avatar, then click Account settings.
2
Click Log Drain from the left pane menu.
3
Click New Log Drain.
4
Enter a Name for the log drain.
5
Enter the Endpoint URL, the base URL of your OpenTelemetry (OTLP) endpoint. Use HTTPS. Integrate.io ETL appends
/v1/logs automatically unless your URL already ends with it.6
Enter the auth header if your endpoint requires one. The Auth header key defaults to
Authorization, and Auth header value takes the full value, for example Bearer <your token>. Leave both blank if the endpoint needs no authentication.The value is stored encrypted and is never shown again after saving. When editing a drain, leave the value blank to keep the stored secret.7
Select which events to drain: Job failed, Job succeeded, or Cluster error. At least one event is required.
8
Choose the scope: the packages, or whole workspaces, whose jobs this drain covers. At least one package or workspace is required, so a drain is always scoped.
9
Click Save. The drain starts active. Set it to Inactive to stop deliveries without deleting it.
Configuration
Cluster Error Scope
The Cluster error event ignores the package and workspace scope: a drain scoped to one package still receives every cluster error in the account. Cluster errors are not tied to a package, so there is nothing to scope them by.Delivery Health
Each log drain shows a health status in the list and in the edit form.
Failed deliveries are retried with a growing delay, and Integrate.io ETL honors a
Retry-After header if your platform sends one. A drain pauses itself when the problem is clearly not going to resolve on its own:
- After 10 consecutive configuration failures, such as a rejected token or an unreachable host
- After 50 consecutive temporary failures that have continued for at least an hour
Resuming a Paused Drain
1
Fix the cause, for example replace an expired token or correct the endpoint URL.
2
Click Resume Log Drain in the drain’s list actions.
Delivery Statistics
The edit form shows a delivery panel with the drain token, the last delivery, the last error, and per-day delivery totals for the most recent 7 days. The API retains 30 days of those counters, covering records sent, failed, and dropped, plus the lines and bytes delivered.Permissions
Log drains have their own permissions, separate from service hooks, because a drain sends full job logs to a third party. Account owners and administrators have them by default. For a custom role, grant them under Global Permissions:
Holding the service hook permissions does not grant any of these.
Best Practices
- Scope each drain to the packages that matter to the team watching that endpoint, rather than sending everything to one place.
- Subscribe to Job failed if you only want problems. Add Job succeeded when you need a record that a job ran at all.
- Use a short-lived token where your platform supports it, and update the drain before it expires, since logs produced while a drain is paused are not recoverable.
- Click Send test after any change to the endpoint or token.
- Filter on
xplenty.package.namein your platform to build per-pipeline views, and alert onxplenty.event = job.failed.
FAQ
Q: Why is the log empty for a job that succeeded? Integrate.io ETL only writes a job log when a job fails, so a succeeded job delivers a summary record (Job #12345 succeeded) and no body. This is normal.
Q: I added a workflow to a drain but received nothing for the workflow itself.
That is by design. A workflow has no log of its own, so each job it ran delivers its own record instead. Look for records naming the child job IDs.
Q: My endpoint returned an error and the drain paused. Where are the missing logs?
They are not retained. Fix the endpoint, then click Resume Log Drain. Delivery restarts from the next job, and the gap is not backfilled.
Q: Can I send logs to a plain HTTP endpoint, or to an address on my private network?
No. Endpoints must be HTTPS and must be reachable on the public internet. Private, loopback, and link-local addresses are rejected.
Q: Does the payload use OTLP JSON?
No, it is OTLP protobuf, gzip compressed, sent as application/x-protobuf. Most collectors reject OTLP JSON.
Q: My log arrived truncated. Why?
The job log exceeded the 4 MB per-delivery limit. The newest lines are kept, since that is where the failure usually appears, and the record carries xplenty.log.truncated.
Q: Can a drain cover every job in the account?
Not implicitly. Every drain must name at least one package or workspace. To cover everything, select all the workspaces you want included.