“With Snowflake, Fivetran and Coalesce, my team’s productivity is up 10x. We are now able to go from idea to insight in just two days, compared to as long as three months with our old legacy ETL solution.”
- Gu Xie, Head of Data Engineering at Group 1001
Key Results
- Increased productivity of data engineering time by 1000%, thanks to faster access to complete and accurate insights
- Reduced time to insight to two days from three months (98% improvement)
Group 1001 data stack
- Pipeline: Fivetran
- Transformations: Coalesce
- Destination: Snowflake
- Data sources: Databases (Postgres), File and Object Storage, SaaS Apps (Zendesk, GA, Salesforce), APIs (REST, GraphQL), Events, Logs
- Other tools: Dagster (orchestrators), PowerBI (reports), Sigma (embedded analytics), Atlan (data discovery), Soda (data quality and monitoring)
Group 1001 is a disruptor in the insurance industry that manages more than $60 billion across multiple brands with a wide range of insurance products, including annuities.
When Gu Xie joined as Head of Data Engineering, he set out to enable a data-driven culture and improve business operations with accurate data. At the time, Group 1001 had no way of seeing all their assets and how they were distributed by companies, product lines or customers.
The legacy data stack Xie inherited made it impossible to provide the granular visibility leadership needed to make key decisions. The system needed to be equipped to handle policy management datasets and additional data sources related to customer support, sales and marketing. Data processing cycles took over five hours to complete. Jobs often had silent failures that took even longer to fix, leading the business to use incomplete and incorrect data in its reporting.
A new approach was needed, one that could automate the creation of pipelines from many data sources, transform data efficiently and deliver accurate results to improve visibility into the customer experience.
Building a modern data stack, accelerating analytics
Xie and his team redesigned Group 1001’s data stack with automation as a core value driver for his team and the business.
In less than a year, the small data engineering team onboarded Snowflake, automated all its deployments, migrated 160 PowerBI reports, moved 200 different pipelines to Dagster and replicated 66 databases. Using Fivetran, nearly 4,000 table feeds were loaded every day, across the database and SaaS sources, adding up to nearly four terabytes of uncompressed data. Thanks to the fully managed approach of the platform, Xie was able to do this significantly faster and with a leaner team compared to building these pipelines manually or via legacy software.
If they had taken a code-first approach or used legacy transformation solutions, his team “would have needed five times as many people and twice as much time.”
“We can only focus on strategic outcomes that enable the organization if we’re not spending time firefighting cycle issues. I came from an Informatica shop, and with Fivetran, Coalesce and Snowflake, we’re fundamentally shifting from manual processes and guesswork to fully automated insights.”
- Gu Xie, Head of Data Engineering at Group 1001
Automating value in annuity provisioning and process optimization
Reports and dashboards are now built on high-quality data, giving Group 1001 business users easy access to support information from servicing tickets, customer intent data from Zendesk, and marketing and traffic information from Google Analytics.
“By using data and insights instead of speculation throughout our organization, we know the real issues, where we’re failing and where we can optimize,” Xie says. An important aspect of the annuities business, for example, is the provisioning of policies to customers in a timely manner. It’s a process that can easily get bogged down by collecting documents, getting signatures, conducting Know Your Customer and Anti-Money Laundering analyses and other regulatory paperwork.
With so many moving parts, visibility into the sales funnel was a critical component that Group 1001 lacked with its legacy solution.
Accelerating time to insights from three months to two days
Because business users with great ideas want to seize each opportunity and take action quickly, optimizing productivity was another top priority for Xie and his team. The combination of cloud-based solutions — Fivetran, Coalesce and Snowflake — enabled key stakeholders to go from idea to insight in just two days. “This was unheard of in our organization,” Xie says. “Our productivity is up 10X.”
Getting from idea to insight used to take three full months, with 80% of the time spent building ETLs to bring the data in. Now that most of Group 1001’s data is seamlessly replicated over to Snowflake, Fivetran can integrate additional data sets and have them populating dashboards in a day.
“This level of visibility [with Fivetran, Coalesce and Snowflake] doesn’t exist ever in the industry and what you’re enabling will fundamentally change how data teams will operate.”
- Gu Xie, Head of Data Engineering at Group 1001
By eliminating outdated, legacy ETL systems, data leaders like Xie are giving business users the tools they need to solve problems quickly and affordably. Xie’s experience at Group 1001 shows that you don’t need a big data engineering team to get game-changing results that help business users better serve customers.
Download this IDC report to learn how Fivetran drives millions in financial impact and enables new business initiatives for enterprises.
.png)
.svg)
.svg)
.svg)