With automated data replication you no longer have to worry about day-to-day management of your data pipelines, including schema and API management. However, we have received requests from the curious regarding how Fivetran works behind the scenes. With that in mind, we’ve released our Log Connector. The Log Connector records events pertaining to Fivetran activities including sync statistics, user activities on each connector and detailed sync information.
The Log Connector offers a few benefits, including:
- Added transparency to your sync. You’ll now know exactly what occurs in each sync, down to when each table is read into your database or exactly when we’re processing the API responses from each source system.
- Ability to trigger post-load workflows. You’ll be able to trigger downstream systems based on the progress in the sync, including when syncs finish, or simply surface the ending timestamp in your dashboards so users are assured that data is fresh.
- Monitoring. You can create a dashboard to monitor any alerts or anomalies with the Fivetran sync to surface data replication health to your stakeholders.
- Tracking Monthly Active Rows. Accompanying our new Consumption Based Pricing, this table helps to track unique (active) rows to each source.
The Log Connector is a fully-featured system that is free for all Fivetran Customers. If you’d like to further explore what Fivetran is doing today, take a preview of the Log Connector Schema and go ahead and add the connector to your list!