When Thomas Edison switched on his first working light bulb, even he could not have predicted that this new technology would eventually revolutionize every aspect of modern life.
Today, the world of analytics holds just as much potential to reimagine the way businesses run — and a host of new tools within the modern data stack mean that tapping into the flow of data is as easy as flipping a switch.
“We’re working hard at Fivetran to build the infrastructure to make that possible,” said co-founder and CEO George Fraser at Modern Data Stack Conference 2020. “Our mission is to make access to data as simple and reliable as electricity. We view ourselves through that lens, as a kind of utility. Our job is to build and maintain perfect connectors to every data source that businesses use.”
In his keynote, Fraser shared his vision for how data infrastructures will continue to advance as innovators break new ground.
1. Modern Data Stack: Automate Your Data Integration
The keystone of the modern data stack (MDS) is a powerful cloud data platform, separating storage and compute with elastic scalability. Commonly, that’s a cloud warehouse and it can also include data lakes. Into that warehouse, data is loaded from source systems including databases, web applications and APIs. To do this, a reliable transformation layer is employed that converts raw data into reliable, query-ready datasets. Lastly, a collaborative business intelligence and visualization solution enables the business to interact with the data and draw actionable insights to guide business decisions.
The Modern Data Stack can blend cloud-based data sources with legacy, on-premises solutions that exist in businesses of all sizes. It offers data professionals an automated way of keeping up with the business and supporting decision makers with mission-critical and consistently up-to-date information. Check out our MDS bootcamp what it takes to set up a modern data stack.
2. On Premise vs Cloud Data Warehouses
Having shot up in popularity over the past few years, these new solutions are fundamentally different than simply on-premises warehouses recreated in the cloud. They offer a far better user experience, enhanced performance and a new world of potential compared with legacy solutions. “If you’re building a new system, cloud-based data warehouses that separate compute from storage are the ideal center of your modern data stack,” said Fraser.
3. Innovations in Data Transformation Tools Will Grow
With automated pipeline solutions like Fivetran taking care of the “E” and “L” in “ETL,” a new category of technologies is emerging: the in-data-warehouse transformation ecosystem. These tools are designed to work with a modern data stack and transform data from one schema to another inside your cloud-based data warehouse. Fraser mentioned the open-source transformation tool dbt in particular (in which Fivetran is integrated with), which has a devoted community of users who help each other learn and grow their skill sets.
4. Making Data Actionable
“Taking action on data is really the frontier of data warehousing,” Fraser said. “It means closing the loop. Instead of your data warehouse producing a bar chart, and someone looking at that bar chart and taking action, a machine goes directly from the insight that’s the result of the query to actually doing something out in the world.” Some of the most exciting use cases he’s seen among Fivetran customers include automating payroll and billing, monitoring intrusion detection, and detecting marketing regressions that could save a company millions.
5. The Modern Data Stack Will Shrink Latency Time
Achieving lower latency time will be key to supporting these new use cases. “There is no latency that is too low, and with the new features that are being developed in the data warehouses, latencies of seconds and tens of seconds are fundamentally possible,” Fraser said. “And we’re working hard at Fivetran to keep battling every element of the pipeline, keep battling down that latency number.”