Get real-time updates on your data pipelines with Fivetran webhooks

With Fivetran webhooks, developers can use real-time messages to power user experiences, transform data, drive error alerting and more.
May 12, 2022

Fivetran has dramatically simplified data ingestion, and as a result, data pipelines have become increasingly sophisticated. Our fast, simple and reliable data integration solution has paved the way for better transformations, more orchestration and new customer experiences.

With data pipelines growing in complexity amid an almost limitless array of potential use cases, Fivetran recognized the widespread customer need for one key feature: webhooks. We got to work, and now they’re here.

What are webhooks?

You can think of a webhook like a reverse API.  With the standard Fivetran API, you request data that we then serve. But once you’ve set up a webhook, Fivetran serves you the data automatically, without you having to request it.  

Imagine how notifications work on your phone. We’ve all been there before: You’ve got somewhere to be, so you’ve called an Uber. The driver’s stuck in traffic, so you keep opening up the Uber app to see how much longer you’ll have to wait. Frustrated, you make peace with the fact that checking the app won’t speed up your driver. A few moments later, your phone vibrates. Uber just sent you the notification you've been waiting for: “Your driver is arriving soon.”

Manually checking for Uber updates is a pull action, whereas the notification Uber sends you is a push action. That’s the magic of a webhook. Instead of endless checks to find out if the information you want is ready, you can make a webhook do the work for you. Once your targeted conditions have been met, the webhook immediately notifies you, eliminating guesswork and freeing up your time.

How do Fivetran webhooks work?

Until now, if you wanted information on the current state of your connectors, you’d have to repeatedly poll our API or configure a complex workflow using entries from our platform connector. In both cases, you’d still be proactively pulling the data. You’d get there in the end, eventually finding the connector update you want. But now there’s a better way.

That’s because our new webhook feature takes the uncertainty (and the extra effort) out of the equation. We watch for the events you tell us to track, and once they happen, we push the update to the endpoint that you set up. Previously, you had to repeatedly ask Fivetran for this information, inefficiently querying for the same update.  Now, you can sit back and wait until Fivetran tells you.

Streamline your downstream data operations with Fivetran webhooks

Using webhooks as part of your Fivetran workflow opens up the door to a wide range of possibilities.

If you’re developing with Powered by Fivetran, for example, you could create a webhook to send emails, Slack alerts or in-product notifications when an end user’s data has finished syncing and is ready for consumption. You could also use a webhook to trigger orchestration you might have with Fivetran or elsewhere. If your data pipeline breaks down, a Fivetran webhook can trigger a notification to PagerDuty and other DevOps tools your organization might be using, enabling them to apply a faster fix.

What events are currently available?

During beta, the following events are available:

   sync_start
   sync_end
   status
   transformation_run_start
   transformation_run_succeeded
   transformation_run_failed
   dbt_run_start
   dbt_run_succeeded
   dbt_run_failed

Try out our webhook sample code

If you’re interested in using webhooks in Fivetran, check out our webhook example project, which we’ve built with Express and Ngrok. The repo ReadMe contains a guide that will get you up and running, so you can start ingesting Fivetran webhooks.

For a comprehensive summary of Fivetran webhooks, see our documentation.

Harnessing the power of webhooks opens the door to countless ways for you to improve both existing and future workflows. We’re thrilled to put this feature in your hands and can’t wait to see how webhooks will help you take your Fivetran implementation to the next level.

Do you have webhook event requests? We’d love to hear them. Head on over to our webhook event support form and let us know what you think.

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Product
Product

Get real-time updates on your data pipelines with Fivetran webhooks

Get real-time updates on your data pipelines with Fivetran webhooks

May 12, 2022
May 12, 2022
Get real-time updates on your data pipelines with Fivetran webhooks
With Fivetran webhooks, developers can use real-time messages to power user experiences, transform data, drive error alerting and more.

Fivetran has dramatically simplified data ingestion, and as a result, data pipelines have become increasingly sophisticated. Our fast, simple and reliable data integration solution has paved the way for better transformations, more orchestration and new customer experiences.

With data pipelines growing in complexity amid an almost limitless array of potential use cases, Fivetran recognized the widespread customer need for one key feature: webhooks. We got to work, and now they’re here.

What are webhooks?

You can think of a webhook like a reverse API.  With the standard Fivetran API, you request data that we then serve. But once you’ve set up a webhook, Fivetran serves you the data automatically, without you having to request it.  

Imagine how notifications work on your phone. We’ve all been there before: You’ve got somewhere to be, so you’ve called an Uber. The driver’s stuck in traffic, so you keep opening up the Uber app to see how much longer you’ll have to wait. Frustrated, you make peace with the fact that checking the app won’t speed up your driver. A few moments later, your phone vibrates. Uber just sent you the notification you've been waiting for: “Your driver is arriving soon.”

Manually checking for Uber updates is a pull action, whereas the notification Uber sends you is a push action. That’s the magic of a webhook. Instead of endless checks to find out if the information you want is ready, you can make a webhook do the work for you. Once your targeted conditions have been met, the webhook immediately notifies you, eliminating guesswork and freeing up your time.

How do Fivetran webhooks work?

Until now, if you wanted information on the current state of your connectors, you’d have to repeatedly poll our API or configure a complex workflow using entries from our platform connector. In both cases, you’d still be proactively pulling the data. You’d get there in the end, eventually finding the connector update you want. But now there’s a better way.

That’s because our new webhook feature takes the uncertainty (and the extra effort) out of the equation. We watch for the events you tell us to track, and once they happen, we push the update to the endpoint that you set up. Previously, you had to repeatedly ask Fivetran for this information, inefficiently querying for the same update.  Now, you can sit back and wait until Fivetran tells you.

Streamline your downstream data operations with Fivetran webhooks

Using webhooks as part of your Fivetran workflow opens up the door to a wide range of possibilities.

If you’re developing with Powered by Fivetran, for example, you could create a webhook to send emails, Slack alerts or in-product notifications when an end user’s data has finished syncing and is ready for consumption. You could also use a webhook to trigger orchestration you might have with Fivetran or elsewhere. If your data pipeline breaks down, a Fivetran webhook can trigger a notification to PagerDuty and other DevOps tools your organization might be using, enabling them to apply a faster fix.

What events are currently available?

During beta, the following events are available:

   sync_start
   sync_end
   status
   transformation_run_start
   transformation_run_succeeded
   transformation_run_failed
   dbt_run_start
   dbt_run_succeeded
   dbt_run_failed

Try out our webhook sample code

If you’re interested in using webhooks in Fivetran, check out our webhook example project, which we’ve built with Express and Ngrok. The repo ReadMe contains a guide that will get you up and running, so you can start ingesting Fivetran webhooks.

For a comprehensive summary of Fivetran webhooks, see our documentation.

Harnessing the power of webhooks opens the door to countless ways for you to improve both existing and future workflows. We’re thrilled to put this feature in your hands and can’t wait to see how webhooks will help you take your Fivetran implementation to the next level.

Do you have webhook event requests? We’d love to hear them. Head on over to our webhook event support form and let us know what you think.

Verwandte Beiträge

No items found.
No items found.
Fivetran Product Update: October 2024
Blog

Fivetran Product Update: October 2024

Beitrag lesen
Data governance is a top priority for AI success
Blog

Data governance is a top priority for AI success

Beitrag lesen
Build vs. buy for AI: Choosing the right data foundation
Blog

Build vs. buy for AI: Choosing the right data foundation

Beitrag lesen

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.