Announcing Fivetran Managed Data Lake Service

Fivetran's new offering simplifies data lake table maintenance, delivering optimized, query-ready data that is always in sync.

At Fivetran, we understand the complexities and challenges of managing data lakes. That’s why we’re excited to introduce our latest innovation: Fivetran Managed Data Lake Service. This new offering is designed to automate and streamline your data lake management, allowing you to focus on what truly matters: making use of your data and driving innovation. Fivetran Managed Data Lake Service is currently available on Amazon S3, Azure Data Lake Storage (ADLS), and Microsoft OneLake.

Fivetran Managed Data Lake Service helps transform traditionally ungoverned data lakes into organized, governed, continuously optimized data stores. With native integrations with data catalogs, including AWS Glue, Databricks Unity Catalog, and Polaris Catalog, users can quickly discover, access, and govern key datasets from the lake. From there, users can query and modify the data with Python, SQL, or other supported languages by leveraging compatible compute engines like Databricks, Snowflake, Starburst, or Redshift. Or, they can transform the data with tools like dbt, visualize it with Power BI, or build and deploy AI/ML models with tools like AWS Sagemaker, Azure Machine Learning, or Databricks Mosaic AI.

The power of a managed data lake

Data lakes are critical for organizations looking to leverage big data for analytics, machine learning, and AI. However, the upkeep of a data lake — handling data ingestion, ensuring data quality, managing schema changes, and optimizing performance — can be resource-intensive and complicated. Recognizing these challenges, Fivetran has developed a service that not only simplifies these tasks but also transforms data lakes from cumbersome data stores into dynamic, efficient, and governed data environments. 

Fivetran Managed Data Lake Service automatically integrates data from over 700 pre-built or custom sources, then normalizes, compacts, and deduplicates it before landing it in your data lake in Delta Lake or Apache Iceberg open table formats. By automating this conversion, we provide features typical of data warehouses, such as ACID transactions and scalable metadata handling, directly on the data lake. From there, we continuously monitor and maintain your data lake, handling updates, merges, and deletes, ensuring it’s always optimized, up-to-date, and query-ready.

This level of automation and maintenance is crucial for many organizations. As Nick Chmura, Head of Data at Luma Financial Technologies, explains, “Automated table maintenance is the killer feature for us with Fivetran because we have so many different source connectors. To try to build change data capture and manage that for everything…would be prohibitively costly in terms of time.”

Key features and benefits

  • Automated data integration: Fivetran supports ingestion from over 700 applications, databases, files, and event data sources, enabling seamless integration into any major data lake destination. This ensures that all your data is consolidated, organized, and easily accessible. Plus, Fivetran covers the costs of ingestion into your data lake, greatly reducing your TCO.
  • Data standardization on open table formats: By normalizing and standardizing your data into query-ready open table formats (Apache Iceberg or Delta Lake), we make it easier for you to perform analytics and gain insights without the hassle and compute cost of manually converting data to a standard format.
  • Continuous maintenance: Fivetran handles all aspects of ongoing data lake maintenance, from schema evolution to performance optimization. This ensures your data lake is always up-to-date and functioning at its best.
  • Robust governance tools: With built-in data governance features and native integrations with popular data catalogs, your data is not only well-managed but also compliant with industry standards and regulations like GDPR.

“We are very excited about Fivetran supporting Delta Lake as a direct destination,” said Himanshu Raja, Director of Product, Databricks. “With this new capability, customers can now use Fivetran to build an open lakehouse with Delta Lake powered by the Databricks Data Intelligence Platform. We are also very excited about the upcoming Fivetran integration with Unity Catalog to provide out-of-the-box governance and security for all Fivetran-generated tables.” 

We're eager for you to try the new Managed Data Lake Service, but it's not a perfect fit for everyone. If your organization relies primarily on real-time streaming data with sub-second latencies, or if you prefer not to use an open table format like Delta Lake or Iceberg, this service may not be the ideal choice. However, we encourage you to get in touch with us — we have other data lake options that may better align with your requirements.

Ready to experience the future of data lake management? 

With Fivetran Managed Data Lake Service, we're making data as accessible and reliable as electricity, empowering businesses to unlock new opportunities and drive innovation.

As data continues to be a pivotal asset for businesses, managing it efficiently and effectively becomes crucial. We fully automate and manage data standardization as we move it to data lake destinations, making it available to businesses to find new ways to innovate with data.

Summer at the lakehouse

Now, Fivetran users can try our Managed Data Lake Service with free usage from June through August. Connectors set up to new data lake destinations will be eligible for this summer promotion*.

To take advantage of this promotion, you need to:

  • Have a Fivetran account in good standing, and
  • Create a new connector with S3, ADLS, or OneLake as the destination during the Promotion Period (between June 1, 2024 at 00:01am UTC and August 31, 2024 at 11:59pm UTC).

To get started, head straight to your Fivetran dashboard, sign up for a 14 day free trial of Fivetran or reach out to sales@fivetran.com with any questions. 

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Product
Product

Announcing Fivetran Managed Data Lake Service

Announcing Fivetran Managed Data Lake Service

June 4, 2024
June 4, 2024
Announcing Fivetran Managed Data Lake Service
Fivetran's new offering simplifies data lake table maintenance, delivering optimized, query-ready data that is always in sync.

At Fivetran, we understand the complexities and challenges of managing data lakes. That’s why we’re excited to introduce our latest innovation: Fivetran Managed Data Lake Service. This new offering is designed to automate and streamline your data lake management, allowing you to focus on what truly matters: making use of your data and driving innovation. Fivetran Managed Data Lake Service is currently available on Amazon S3, Azure Data Lake Storage (ADLS), and Microsoft OneLake.

Fivetran Managed Data Lake Service helps transform traditionally ungoverned data lakes into organized, governed, continuously optimized data stores. With native integrations with data catalogs, including AWS Glue, Databricks Unity Catalog, and Polaris Catalog, users can quickly discover, access, and govern key datasets from the lake. From there, users can query and modify the data with Python, SQL, or other supported languages by leveraging compatible compute engines like Databricks, Snowflake, Starburst, or Redshift. Or, they can transform the data with tools like dbt, visualize it with Power BI, or build and deploy AI/ML models with tools like AWS Sagemaker, Azure Machine Learning, or Databricks Mosaic AI.

The power of a managed data lake

Data lakes are critical for organizations looking to leverage big data for analytics, machine learning, and AI. However, the upkeep of a data lake — handling data ingestion, ensuring data quality, managing schema changes, and optimizing performance — can be resource-intensive and complicated. Recognizing these challenges, Fivetran has developed a service that not only simplifies these tasks but also transforms data lakes from cumbersome data stores into dynamic, efficient, and governed data environments. 

Fivetran Managed Data Lake Service automatically integrates data from over 700 pre-built or custom sources, then normalizes, compacts, and deduplicates it before landing it in your data lake in Delta Lake or Apache Iceberg open table formats. By automating this conversion, we provide features typical of data warehouses, such as ACID transactions and scalable metadata handling, directly on the data lake. From there, we continuously monitor and maintain your data lake, handling updates, merges, and deletes, ensuring it’s always optimized, up-to-date, and query-ready.

This level of automation and maintenance is crucial for many organizations. As Nick Chmura, Head of Data at Luma Financial Technologies, explains, “Automated table maintenance is the killer feature for us with Fivetran because we have so many different source connectors. To try to build change data capture and manage that for everything…would be prohibitively costly in terms of time.”

Key features and benefits

  • Automated data integration: Fivetran supports ingestion from over 700 applications, databases, files, and event data sources, enabling seamless integration into any major data lake destination. This ensures that all your data is consolidated, organized, and easily accessible. Plus, Fivetran covers the costs of ingestion into your data lake, greatly reducing your TCO.
  • Data standardization on open table formats: By normalizing and standardizing your data into query-ready open table formats (Apache Iceberg or Delta Lake), we make it easier for you to perform analytics and gain insights without the hassle and compute cost of manually converting data to a standard format.
  • Continuous maintenance: Fivetran handles all aspects of ongoing data lake maintenance, from schema evolution to performance optimization. This ensures your data lake is always up-to-date and functioning at its best.
  • Robust governance tools: With built-in data governance features and native integrations with popular data catalogs, your data is not only well-managed but also compliant with industry standards and regulations like GDPR.

“We are very excited about Fivetran supporting Delta Lake as a direct destination,” said Himanshu Raja, Director of Product, Databricks. “With this new capability, customers can now use Fivetran to build an open lakehouse with Delta Lake powered by the Databricks Data Intelligence Platform. We are also very excited about the upcoming Fivetran integration with Unity Catalog to provide out-of-the-box governance and security for all Fivetran-generated tables.” 

We're eager for you to try the new Managed Data Lake Service, but it's not a perfect fit for everyone. If your organization relies primarily on real-time streaming data with sub-second latencies, or if you prefer not to use an open table format like Delta Lake or Iceberg, this service may not be the ideal choice. However, we encourage you to get in touch with us — we have other data lake options that may better align with your requirements.

Ready to experience the future of data lake management? 

With Fivetran Managed Data Lake Service, we're making data as accessible and reliable as electricity, empowering businesses to unlock new opportunities and drive innovation.

As data continues to be a pivotal asset for businesses, managing it efficiently and effectively becomes crucial. We fully automate and manage data standardization as we move it to data lake destinations, making it available to businesses to find new ways to innovate with data.

Summer at the lakehouse

Now, Fivetran users can try our Managed Data Lake Service with free usage from June through August. Connectors set up to new data lake destinations will be eligible for this summer promotion*.

To take advantage of this promotion, you need to:

  • Have a Fivetran account in good standing, and
  • Create a new connector with S3, ADLS, or OneLake as the destination during the Promotion Period (between June 1, 2024 at 00:01am UTC and August 31, 2024 at 11:59pm UTC).

To get started, head straight to your Fivetran dashboard, sign up for a 14 day free trial of Fivetran or reach out to sales@fivetran.com with any questions. 

*How to qualify for this promotional offer:

Eligibility: 

  • Customer must have a Fivetran account in good standing. Customers with valid trial accounts also qualify for this promotion. 
  • This offer is for connectors set up to load data to new qualifying data lake destinations only, which includes: Amazon S3, Azure Data Lake Storage (ADLS), and Microsoft OneLake.
  • Qualifying connectors with data lake destinations must be created between June 1, 2024 at 00:01 UTC and August 31, 2024 at 11:59pm UTC (the “Promotion Period”). 
  • Usage from connectors to qualifying data lake destinations will appear in the product dashboard the same as any other connector, however usage will be marked as free each month during the Promotion Period.

Promotion Terms and limitations:

  • Fivetran is offering free usage of its managed data lake service up to $10,000 per customer during the Promotion Period. Fivetran may, in its sole discretion, either (i) charge any usage that exceeds this limit according to contracted rates and/or (ii) restrict creation of connectors to new qualifying data lake destinations in the event Fivetran determines customer use has become excessive.  
  • This offer and promotion ends on August 31, 2024 at 11:59pm UTC. Any usage from qualifying data lake connectors occurring after this promotion ends will be charged at the contracted rates. 
Topics
Share

Related blog posts

A deep dive into data lakes
Data insights

A deep dive into data lakes

Read post
Why Fivetran supports data lakes
Product

Why Fivetran supports data lakes

Read post
How to govern your S3 data lake
Data insights

How to govern your S3 data lake

Read post
No items found.
7 data and AI predictions for 2025
Blog

7 data and AI predictions for 2025

Read post
Generative AI: A 2-year retrospective and what's next
Blog

Generative AI: A 2-year retrospective and what's next

Read post
Why you need both technical and business data catalogs
Blog

Why you need both technical and business data catalogs

Read post

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.