Service Hooks

Service hooks are notifications that you can subscribe to which allow you to receive API updates when events happen in ETL such as cluster created, or job failed. Service hooks allow you to avoid polling the API to receive up-to-date statuses and have the notifications delivered to other services that are not calling ETL's APIs (such as monitoring services).

To use service hooks, you can create a webhook via ETL's UI. Each webhook has a URL and a set of events you want to be notified about. After defining the webhook, ETL will POST to the URL notifications, every time the events are taking place. 

The notification's web request body is the same as the response body for an API request for the same resource. Read more here.

Creating Service Hooks

Service hooks are defined at the account level by account administrators. To create a service hook:

  1. Click your avatar, then click Account settings
  2. Click Service Hooks from the left pane menu.
  3. Click + new hook.
  4. In the list, choose Webhook.
  5. Enter webhook URL (HTTP or HTTPS).
  6. Select the events you want to be notified about using this webhook:
    All Job Notifications:
      All events below.
      • Job Submitted: Job was submitted by user or schedule for execution.
      • Job Started: Job execution started.
      • Job Stopped: Job stopped by user.
      • Job Completed: Job completed successfully.
      • Job Failed: Job completed with failure.
    • All Cluster Notifications: All events below.
      • Cluster Requested: Cluster creation requested by user or schedule.
      • Cluster Available: Cluster is available and jobs can be run on it.
      • Cluster Terminated: Cluster is terminated.
      • Cluster Idled: No activity on the cluster for a specified interval.
      • Cluster Error: Cluster error occurred and jobs can't be executed on it.
  7. By default, the webhook is created active but you can disable it if you wish.
  8. Click create webhook
  9. You can trigger a ping event to test the webhook.

Securing webhooks

We support both HTTP and HTTPS subscriptions and we recommend you provide an HTTPS URL to protect the notifications. Once a webhook is defined, a secret is created and can be viewed in the webhook's page. We use this secret to sign the notification as an HMAC hex digest of the request body using the secret as the key. The signature is then put in the X-Hook-Signature request header.