Overview
The Sync Lag metric measures the difference between the source data and the pipeline in megabytes (MB), providing a real-time indicator of pipeline performance. Customers can set up alerts to monitor this metric and receive notifications if the sync lag reaches a high value, which could signal abnormal pipeline behavior.
This metric is currently available for MySQL pipelines but can be made available for other pipelines upon request.
Key Features
-
Sync Lag Monitoring:
Track the difference between the source and the pipeline in MB, ensuring transparency in pipeline performance.
-
Alert Setup:
Easily configure alerts for Sync Lag metrics. Receive timely notifications if the lag exceeds a defined threshold, indicating potential pipeline issues.
-
Automatic Fetching:
The system automatically retrieves the most recent Sync Lag data for pipelines that are actively running, calculating averages over the last 5 minutes.
Note: Regardless of the frequency set for the alert (e.g., every 10 minutes or every hour), the Sync Lag data fetched is only from the last 5 minutes preceding the alert check.
Considerations
- Pipeline State:
- Alerts are triggered only for pipelines in the Running state.
- If a pipeline is running but no sync lag data is available, this is treated as abnormal behavior, and an alert will be issued.
- Expected Sync Lag Range:
- The acceptable sync lag between the source and the pipeline may vary depending on the pipeline load. However, as a general guideline, the sync lag should stay below 100MB. A lag exceeding this threshold could indicate abnormal pipeline behavior.
- Metric Scope:
- This metric does not measure the difference in lag between the source and the destination. Instead, it measures the lag between the source and the pipeline. The data transfer to the destination is influenced by the sync frequency configured at the pipeline level.