Sorenson plugs and plays data integration at scale

Learn how the world's leading language services provider leverages Fivetran to offer its customers real-time, data-dependent services
January 10, 2024

Programmatic control over data operations is an essential capability for data teams. As an organization’s data needs grow in scale and complexity over time and data teams pursue a wider variety of analytics and data-driven products, manual construction, maintenance and administration become increasingly untenable. Teams must also be able to integrate their data operations with additional tools and technologies. 

Sorenson Communications is a leading provider of captioning and interpretation services for the hard-of-hearing and deaf, with the mission to make communication accessible and clear regardless of signed or spoken language. Automated, real-time translation of all kinds depends heavily on natural language processing and the data used to train it.

Travis Smith, Senior Manager of Data Engineering at Sorenson, led the company’s efforts to better offer its customers real-time, data-dependent services. Sorenson needed to improve the speed and responsiveness of its data integration. Fivetran was integral to this effort, which included migrating to the cloud and modernizing its infrastructure. According to Smith:

“Fivetran came and saved the day for us. We had some changes in our data structure and a really fast go-to-market planned with a new CRM, and without Fivetran there was no real way to grab the data without a lot of custom development.” 

The data team faced challenges such as the time required to build and maintain data connectors as well as difficult, obscure bugs resulting from corner cases. The team also had to cope with the sheer diversity of its data sources. Ultimately, the data team connected over 50 database sources through Fivetran.

The Fivetran REST API, with its support for automated scripting using Python and the multitude of robust off-the-shelf scripts from the open-source framework, offered many important capabilities to Sorenson. Some of the most vexing problems are the simplest; their SQL Server certification rotation previously required periodic manual revalidation, typically prompted by a sudden loss of access. The data team replaced this process with a script that monitors and, if necessary, rotates certificates every ten minutes.

Smith used the Fivetran REST API jumpstarted efforts to systematize his team’s entire data operation from front to back, making procedures visible and repeatable. This was further accelerated with the help of off-the-shelf software packages such as the Python framework for the REST API as well as off-the-shelf, analytics-ready data models. The team was able to develop their infrastructure using CI/CD and programmatically create and edit connectors to a wide range of sources, including test and development environments. Moreover, using the Fivetran Platform Connector, the team was able to extract log data. By combining log data with the average time in minutes it would have taken to manually manage each event, the team was able to quantify the savings in engineering hours with a simple formula: SUM(events * avg_time/60).

By leveraging automation along with better observability and monitoring, Sorenson’s data team was able to considerably improve its SLA. As Smith remarked:

“Good data engineering is when I push a button to make something happen. Great data engineering is when you schedule automations that run whenever they need to and I don’t even need to click the button. That’s Fivetran.” 

All the Sorenson data team needed to contribute was a configuration file. The most time- and labor-intensive part of the process involved going into the portals for data sources to retrieve the necessary credentials.

Freed from day-to-day pipeline management by the API framework, sample scripts and automation, Smith’s team was able to set and forget their data integration and direct resources to new AI initiatives.

Fivetran REST API brings programmatic control to data integration

Programmatic control is central to performing administrative tasks at scale, integrating other tools and technologies into a data operations ecosystem and building a sustainable foundation for the future expansion of analytics and data products.

The Fivetran REST API and its accompanying Python framework enable programmatic control over important aspects of the Fivetran application, including:

  • Certificate management – such as validating transport layer security (TLS) certificates
  • Connector management – viewing, creating, editing, removing and running connectors and their schema configuration files
  • Transformation orchestration – viewing, creating, editing, removing and running sequences of transformations 
  • Destination management –  creating, editing and removing destinations
  • Team management – viewing, editing and deleting teams 
  • User management – viewing, inviting, editing and deleting users
  • Webhook management – viewing, creating, editing, removing and testing webhooks

In addition, the REST API supports monitoring of lists and groups. The following diagram illustrates how all of these entities relate to each other: 

To experience the power of the Fivetran REST API and see how it can accelerate your data operations for yourself, consider a free 14-day trial or demo.

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Data insights
Data insights

Sorenson plugs and plays data integration at scale

Sorenson plugs and plays data integration at scale

January 10, 2024
January 10, 2024
Sorenson plugs and plays data integration at scale
Learn how the world's leading language services provider leverages Fivetran to offer its customers real-time, data-dependent services

Programmatic control over data operations is an essential capability for data teams. As an organization’s data needs grow in scale and complexity over time and data teams pursue a wider variety of analytics and data-driven products, manual construction, maintenance and administration become increasingly untenable. Teams must also be able to integrate their data operations with additional tools and technologies. 

Sorenson Communications is a leading provider of captioning and interpretation services for the hard-of-hearing and deaf, with the mission to make communication accessible and clear regardless of signed or spoken language. Automated, real-time translation of all kinds depends heavily on natural language processing and the data used to train it.

Travis Smith, Senior Manager of Data Engineering at Sorenson, led the company’s efforts to better offer its customers real-time, data-dependent services. Sorenson needed to improve the speed and responsiveness of its data integration. Fivetran was integral to this effort, which included migrating to the cloud and modernizing its infrastructure. According to Smith:

“Fivetran came and saved the day for us. We had some changes in our data structure and a really fast go-to-market planned with a new CRM, and without Fivetran there was no real way to grab the data without a lot of custom development.” 

The data team faced challenges such as the time required to build and maintain data connectors as well as difficult, obscure bugs resulting from corner cases. The team also had to cope with the sheer diversity of its data sources. Ultimately, the data team connected over 50 database sources through Fivetran.

The Fivetran REST API, with its support for automated scripting using Python and the multitude of robust off-the-shelf scripts from the open-source framework, offered many important capabilities to Sorenson. Some of the most vexing problems are the simplest; their SQL Server certification rotation previously required periodic manual revalidation, typically prompted by a sudden loss of access. The data team replaced this process with a script that monitors and, if necessary, rotates certificates every ten minutes.

Smith used the Fivetran REST API jumpstarted efforts to systematize his team’s entire data operation from front to back, making procedures visible and repeatable. This was further accelerated with the help of off-the-shelf software packages such as the Python framework for the REST API as well as off-the-shelf, analytics-ready data models. The team was able to develop their infrastructure using CI/CD and programmatically create and edit connectors to a wide range of sources, including test and development environments. Moreover, using the Fivetran Platform Connector, the team was able to extract log data. By combining log data with the average time in minutes it would have taken to manually manage each event, the team was able to quantify the savings in engineering hours with a simple formula: SUM(events * avg_time/60).

By leveraging automation along with better observability and monitoring, Sorenson’s data team was able to considerably improve its SLA. As Smith remarked:

“Good data engineering is when I push a button to make something happen. Great data engineering is when you schedule automations that run whenever they need to and I don’t even need to click the button. That’s Fivetran.” 

All the Sorenson data team needed to contribute was a configuration file. The most time- and labor-intensive part of the process involved going into the portals for data sources to retrieve the necessary credentials.

Freed from day-to-day pipeline management by the API framework, sample scripts and automation, Smith’s team was able to set and forget their data integration and direct resources to new AI initiatives.

Fivetran REST API brings programmatic control to data integration

Programmatic control is central to performing administrative tasks at scale, integrating other tools and technologies into a data operations ecosystem and building a sustainable foundation for the future expansion of analytics and data products.

The Fivetran REST API and its accompanying Python framework enable programmatic control over important aspects of the Fivetran application, including:

  • Certificate management – such as validating transport layer security (TLS) certificates
  • Connector management – viewing, creating, editing, removing and running connectors and their schema configuration files
  • Transformation orchestration – viewing, creating, editing, removing and running sequences of transformations 
  • Destination management –  creating, editing and removing destinations
  • Team management – viewing, editing and deleting teams 
  • User management – viewing, inviting, editing and deleting users
  • Webhook management – viewing, creating, editing, removing and testing webhooks

In addition, the REST API supports monitoring of lists and groups. The following diagram illustrates how all of these entities relate to each other: 

To experience the power of the Fivetran REST API and see how it can accelerate your data operations for yourself, consider a free 14-day trial or demo.

Verwandte Beiträge

Manage data pipelines at scale with the Fivetran REST API
Product

Manage data pipelines at scale with the Fivetran REST API

Beitrag lesen
Build Connector Status Dashboards With the Fivetran REST API
Product

Build Connector Status Dashboards With the Fivetran REST API

Beitrag lesen
Behind the data model: How we transform your data
Data insights

Behind the data model: How we transform your data

Beitrag lesen
No items found.
Announcing Fivetran Managed Data Lake Service
Blog

Announcing Fivetran Managed Data Lake Service

Beitrag lesen
Fivetran Product Update: June 2024
Blog

Fivetran Product Update: June 2024

Beitrag lesen
Implementing a data fabric: From silos to insights
Blog

Implementing a data fabric: From silos to insights

Beitrag lesen

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.