When you think about agriculture, the first thing that comes to mind is probably farming — not global supply chain data analysis or computer vision applications for drone video footage.
This, however, is the day-to-day reality for Riley Buss, Senior Manager of IT Engineering - Analytic & Automation at CHS Inc. His work involves managing multiple teams of data engineers, data scientists and robotic process automation (RPA) developers. He’s one of 10,000 CHS employees working around the world toward building efficient supply chains across global markets.
CHS is an essential piece of our food supply. The enterprise is a $45B+ diversified global agribusiness cooperative owned by farmers and local cooperatives across the United States. They drive sustainable growth for more than 75,000 food producers by providing access to petroleum, energy, transportation, logistics, grains and nutrients.
On the latest episode of the Fivetran Data Podcast, Buss dove into the diverse and complex data work that keeps food on the table and keeps farms running.
Terabytes of real-time ingestion and difficult bandwidth challenges
Buss manages terabytes of data from various sources, such as Internet of Things (IoT) devices and programmable logic controllers (PLCs) that feed into growth monitoring indices (GMIs). “We’re a very asset-heavy company,” notes Buss. “We have grain elevators, river terminals, gateway ports, processed food ingredients and ethanol refineries.” Every asset produces a lot of data.
Additionally, Buss oversees 50 business applications and 14 different enterprise resource planning systems (ERPs) — including one that has 36 instances — in an effort to get farmers and supply chain managers the data they need. He jokes, “It creates job security, that’s for sure, [and a] fun data engineering challenge.”
When Buss started at CHS 8 years ago, he contended with slow 1.5 Mbps T1 connections and Citrix-based ERP systems. Now, with the help of a modern data stack — consisting of Fivetran, Snowflake and dbt — he’s ingesting gigabytes of data in real time from farms and assets across America. “It’s easier to manage data quality because we’re able to do point-in-time matches against the data source,” notes Buss. Doing this across dozens of ERPs and applications is no small feat.
Today’s challenges have less to do with bandwidth and more to do with the high volume of incoming data. Using Fivetran HVR, Buss can ensure he won’t miss changes to even a single row across his data ecosystem.
“Fivetran really stood out because it has the compare and repair functionality. [...] It ensures that when we bring the data into Snowflake, we know that it’s matching the source and we’re starting with good integrity.”
- Riley Buss, Senior Manager of IT Engineering, CHS Inc.
The modern data stack powers the world’s largest industry
Buss depends on the modern data stack to power CHS’s data integration, processing and modeling.
“We leverage Fivetran to do CDC off all our data, stream it into [Snowflake] and provide real-time analytics back to our business. That was one of the game changers when we moved from Cloudera to our new data stack.”
- Riley Buss, Senior Manager of IT Engineering, CHS Inc.
His previous architecture relied on batch-style ingestions, lacked modern features and was “more time-consuming and resource intensive.” The modern stack allowed him to move to change data capture (CDC), which made a considerable improvement in performance. With Fivetran, he’s able to handle deduplication and deletes before the data finds its way downstream into Snowflake.
Snowflake enables the team to easily share trustworthy, clean data with external partners. The data is already normalized and available for consumption, solving the problem of providing access to its ERP systems and business applications. Using dbt Cloud, CHS is building automated data products to ensure every product meets its strict data quality standards.
Servant leadership drives innovation at CHS
It takes more than a highly efficient data stack to conquer CHS’s IT engineering challenges. Buss relies on a team of data scientists and engineers to lend their perspectives to each project. When it comes to innovation, he believes that collaboration is key.
“I’m very big on servant leadership,” says Buss, reflecting on his own experience rising through the ranks as an engineer. Now, as Senior Manager, he strongly believes in the importance of empowering each team member to have a voice in strategic planning. “Fostering that collaborative culture is the biggest thing that I try to do from a leadership perspective.”
Rather than looking for pure data wizardry, he’s interested in people who can work with business users, understand the processes and challenges and then partner with them on building a technical solution. “That’s really where we see the best kind of mix of both worlds.”
.svg)
.svg)
.svg)