Our first-ever Modern Data Stack Conference EMEA just concluded, and our EMEA team was thrilled with the turnout and quality of the speakers and discussions. Some of the leading thinkers in the data space offered insights into many of the technical and business challenges faced by today’s data analysts, data engineers and data scientists. Discussion topics included:
- Predicting the future of data and analytics
- Building a 360° view of the customer
- Transferring data into query-optimized structures
In the wake of the conference, we wanted to peek around the corner and imagine the future state of the modern data stack. I recently sat down with Fivetran CEO and co-founder George Fraser to ask how a few more years of rapid evolution might change the technologies that power modern analytics.
How do you see the modern data stack evolving over the next four or five years? What will it look like in 2025?
I think we’re going to see the continued evolution of the modern data stack towards being a hybrid analytical-operational system. One of the things we see Fivetran users do is use their data warehouse to build all kinds of operational applications. It can be as simple as an account manager dashboard that shows you what accounts you should call today. These things are really valuable, and we see them proliferating as they become more possible with modern data warehouses.
That’s one use case, but what do you see as other future use cases that might become possible as the modern data stack improves?
I think you’ll see more companies serving production use cases in their application, straight out of their data warehouse, as data warehouses become better at doing those kinds of hybrid workloads.
I also think you’ll see more interfaces that are centered around looking at small amounts of data and the relationships between them, rather than looking at aggregates. The classic use of data warehouses is looking at aggregated data, and that’s still very useful, but when you have all of your data in one place, you can explore individual entities and how they relate to one another, and try to make not just big discoveries but also small discoveries.
And I think we’re going to see more of those kinds of use cases and workflows evolve on top of the data warehouse. Data warehouses are very fast at reading data, and that’s where a lot of the aggregation use cases come from, but they’re also the place where all the data from your business lives, and so they tend to attract those kinds of workloads.
How will the role of professionals that work with the modern data stack evolve over time? Traditional ETL engineers have been able to evolve in the type of work they do, but which roles will become more prevalent or shift as the modern data stack continues to develop?
Analytics teams will have to act more and more like engineering teams — and this is happening already — because they’re operating production workloads now that can’t go down. So they have to adopt practices like testing and continuous integration, and that’s already underway, but that will continue to happen.
In terms of data literacy and business users being able to leverage self-service systems, if data teams are like engineering teams, they’re involved in big projects for long periods of time. If organizations want to be agile and make decisions on the fly, should analysts be involved, or should that agility be facilitated by the data stack?
Well, there are multiple kinds of business users. There are business users who have very high levels of data literacy, who can go in and self-serve, and that’s great, but that’s not the only way to be a great businessperson. Some users don’t have that level of literacy, they have other skills, so you need to be able to serve both. That’s not going to change.