We're excited to announce that a new data lake destination is now publicly available in Beta at Fivetran: Delta Lake on Azure Data Lake Storage (ADLS).

Delta Lake is the preferred format for data on both Microsoft and Databricks, and Fivetran automates the process of extracting your data, then cleansing, conforming and converting it to Delta Lake format, before loading it into Azure Data Lake Storage. So whether you're already using Azure, Databricks or both, skip the pain of the move and start doing advanced analytics and AI on your data faster.

Why is Fivetran's new Delta Lake on Azure destination a big deal?

Normally, it's difficult to land data from multiple sources into ADLS because a typical organization might have dozens of data sources that they need to build pipelines for, each with its own API and specific quirks. Even if your data engineers succeed in setting one up initially, keeping data in sync is harder than most people imagine. Change data capture can become a major headache, as evolving schemas and API updates can cause breaking changes that lead to sleepless nights.

Fivetran's Delta Lake on ADLS destination is the simplest way to unify all of your data in one place, the lakehouse. Why? We've spent years building and refining our over 300 connectors and have hundreds of engineers whose sole responsibility is maintaining them to ensure an SLA of 99.9 percent uptime. Plus, our platform is completely no-code, changes to your source data are always reliably synced with CDC and a single connector can be set up in less than 10 minutes.

Finally, Delta Lake on Azure is a big deal because it features native integration with Databricks Unity Catalog, and support for Microsoft Purview is also coming soon. You can govern and track lineage for your data, enabling you to have full visibility into how your teams are using it to build downstream tables and applications.

How does Fivetran's new Delta Lake on Azure destination work?

With Fivetran, you can seamlessly load data from over 300 sources, including on-premises and cloud data warehouses, databases and SaaS applications. Regardless of where your data resides, you can effortlessly copy it into your lakehouse for advanced analytics and AI, in no time flat.

But we don't stop at basic data replication. We also cleanse, conform, deduplicate and normalize your data, ensuring its quality and consistency. Say goodbye to messy, fragmented data, and hello to synchronized, reliable datasets that are ready for advanced analytics right out of the box.

Finally, no matter what format your initial data is in, our newest destination automates the process of converting it into Delta Lake format. Delta Lake provides enhanced reliability, scalability and performance for your lakehouse, enabling efficient query processing and data manipulation with Python, SQL or Scala.

What are the limitations of the new destination?

As of August 8, 2023, support for Databricks Unity Catalog is live, Microsoft Purview is coming soon and the Delta Lake on Azure destination is now available in all Fivetran-supported regions.

How does Delta Lake on ADLS differ from Fivetran's existing support for Databricks as a destination?

Our new lakehouse offering isn't specific to just Databricks. Many of our customers using the Azure cloud also use Delta Lake along with Azure Synapse Analytics and other native Azure services.

For customers who are planning to land data in Delta Lake on ADLS using Databricks, we made a number of enhancements to this offering that make it different, including:

• Integration with Databricks Unity Catalog (live) and Microsoft Purview (coming soon)
• Less expensive ingestion using Fivetran compute, instead of Databricks compute

What's coming next for data lake destinations at Fivetran?

After we finalize support for Microsoft Purview, we’ll also be adding support for Microsoft Fabric and OneLake. Ultimately, our goal is to support data ingestion from all of our data sources onto any cloud, in any analytics format.

To get started with Delta Lake on Azure as a destination, start a 14-day free trial.