A new framework for evaluating data teams

Data teams can deliver informational and operational value. Learn what they are and why they matter.

Everywhere you look, people are writing about the return on investment (ROI) of data teams. Data teams can make your organization more decisive and more efficient. Data teams enable other teams and should be factored into their ROIs, or deliver multiple types of ROI depending on function. These takes offer a great survey of opinions on the subject. However, what about looking at things on a role-by-role level? How can this ROI translate into a hiring roadmap and team building?

This series digs into the role-by-role ROI of data teams, giving you a framework to evaluate your current and future team. We will start from a high-level with the return provided by certain functions, and then dig into how to measure the success of each of those functions. We’ll go from individual contributors to managers to product owners. Before we get ahead of ourselves, however, we’ll set the stage with a framework.

The two types of value delivered by data teams

Data can deliver value informationally and operationally. 

Informational value is the insights and ideas that you pull from your data: this conversion funnel shows that experiment A is better than experiment B, this time series graph shows that monthly sales growth varies significantly by quarter. The precise value of this type of information can be a challenge to measure as it’s often tied to how it’s used and to the relevant stakeholder. Insights about A/B testing can make a huge difference for the product team, but only if acted on!

Operational value, on the other hand, is the value that you get from data being where it needs to be and in the proper state. Here at Montreal Analytics, Fivetran automates the extraction of data from many of our clients’ SaaS tools and transformation layers automate the cleaning and standardization of our event data. The value of this information can also be challenging to measure, but at least it’s much closer to software engineering concepts of value and platform stability: Does it have downtime and if so, how much? How quickly are bugs resolved? How quickly are new features delivered? Just like informational value, however, this is only valuable if it is used: what use is a stable, bug-free data pipeline if it’s not powering your business? 

For modern data teams, these two types of value come hand in hand. Some team members – such as a data engineer – may be more focused on operational value while others –such as a data analyst – are more focused on informational value, but both are needed.

This division gives you an important framework through which to measure your data team’s output: Is your biggest strength operational or informational? What about your biggest weakness?

Learn how you can complement a data team with a modern data stack.

Get started

How to sustain the delivery of value

Informational and operational delivery focus on output, but the second major element of a successful data team is its ability to maintain and extend that output.

Just like software teams, a modern data team must keep the future top of mind. Data teams create pipelines to answer questions today, but that’s all for nothing if the system falls apart tomorrow.

Strong data pipelines can be described by three simple attributes: inspectability, maintainability, and extensibility. 

Inspectable data pipelines are easy to understand and not just by their creators. Readability is a priority in inspectable systems, and while it’s important that systems are performant, it’s more important that they’re comprehensible. Inspectable pipelines are clearly documented and are thoughtfully organized throughout. Data engineers should create custom data connector code with comments and sensical function names. Data analysts should create dashboards and visualizations that have strong naming conventions and business-appropriate organization. Inspectability is relevant across the pipeline!

Maintainable pipelines are modular, tested and verified. Downtime and tech debt are two major concerns when we talk about maintainability: how can you design your pipeline to avoid both? Inspectability is a great starting point, but modularity is a great next step. Modular pipelines are faster and simpler to update, and often lend well to testing due to their simpler design. Individual components can then be tested and verified as needed, be that automated testing on code or user acceptance testing on a dashboard.

Extensible pipelines easily accommodate change. They follow strong design rules that make adding new functionality straightforward: this is how things are named, tested, organized and more. Extensibility builds on top of inspectability and maintainability, but it puts more emphasis on the reality of changing business requirements. Will your ingestion code work well if the API changes? Will your dashboard work well if KPIs are redefined? 

Conclusion

Data teams deliver informational and operational value, which depend on data pipelines that are inspectable, maintainable and extensible. In the upcoming posts we’ll use these two frameworks to discuss the various components of the Modern Data Team and the value they each provide. Our first framework helps us understand and classify the output – a common measure of success – while the second helps us put emphasis on the future utility of your pipeline – a much harder-to-pinpoint measure. Stay tuned for our next posts on individual contributors, then on managers and finally on product owners.

A successful data team needs a modern data stack to power its data integration efforts.

Learn how

Montreal Analytics is a Modern Data Stack consulting firm of 45+ people based out of North America. We help our clients on the whole data journey: pipelines, warehousing, modeling, visualization and activation, using technologies like Fivetran, Snowflake, dbt, Sigma, Looker and Census. From strategic advisory to hands-on development and enablement, our agile team can deploy greenfield data platforms, tackle complex migrations and audit & refactor entangled data models.

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Blog

A new framework for evaluating data teams

August 24, 2022
A new framework for evaluating data teams
Data teams can deliver informational and operational value. Learn what they are and why they matter.

Everywhere you look, people are writing about the return on investment (ROI) of data teams. Data teams can make your organization more decisive and more efficient. Data teams enable other teams and should be factored into their ROIs, or deliver multiple types of ROI depending on function. These takes offer a great survey of opinions on the subject. However, what about looking at things on a role-by-role level? How can this ROI translate into a hiring roadmap and team building?

This series digs into the role-by-role ROI of data teams, giving you a framework to evaluate your current and future team. We will start from a high-level with the return provided by certain functions, and then dig into how to measure the success of each of those functions. We’ll go from individual contributors to managers to product owners. Before we get ahead of ourselves, however, we’ll set the stage with a framework.

The two types of value delivered by data teams

Data can deliver value informationally and operationally. 

Informational value is the insights and ideas that you pull from your data: this conversion funnel shows that experiment A is better than experiment B, this time series graph shows that monthly sales growth varies significantly by quarter. The precise value of this type of information can be a challenge to measure as it’s often tied to how it’s used and to the relevant stakeholder. Insights about A/B testing can make a huge difference for the product team, but only if acted on!

Operational value, on the other hand, is the value that you get from data being where it needs to be and in the proper state. Here at Montreal Analytics, Fivetran automates the extraction of data from many of our clients’ SaaS tools and transformation layers automate the cleaning and standardization of our event data. The value of this information can also be challenging to measure, but at least it’s much closer to software engineering concepts of value and platform stability: Does it have downtime and if so, how much? How quickly are bugs resolved? How quickly are new features delivered? Just like informational value, however, this is only valuable if it is used: what use is a stable, bug-free data pipeline if it’s not powering your business? 

For modern data teams, these two types of value come hand in hand. Some team members – such as a data engineer – may be more focused on operational value while others –such as a data analyst – are more focused on informational value, but both are needed.

This division gives you an important framework through which to measure your data team’s output: Is your biggest strength operational or informational? What about your biggest weakness?

Learn how you can complement a data team with a modern data stack.

Get started

How to sustain the delivery of value

Informational and operational delivery focus on output, but the second major element of a successful data team is its ability to maintain and extend that output.

Just like software teams, a modern data team must keep the future top of mind. Data teams create pipelines to answer questions today, but that’s all for nothing if the system falls apart tomorrow.

Strong data pipelines can be described by three simple attributes: inspectability, maintainability, and extensibility. 

Inspectable data pipelines are easy to understand and not just by their creators. Readability is a priority in inspectable systems, and while it’s important that systems are performant, it’s more important that they’re comprehensible. Inspectable pipelines are clearly documented and are thoughtfully organized throughout. Data engineers should create custom data connector code with comments and sensical function names. Data analysts should create dashboards and visualizations that have strong naming conventions and business-appropriate organization. Inspectability is relevant across the pipeline!

Maintainable pipelines are modular, tested and verified. Downtime and tech debt are two major concerns when we talk about maintainability: how can you design your pipeline to avoid both? Inspectability is a great starting point, but modularity is a great next step. Modular pipelines are faster and simpler to update, and often lend well to testing due to their simpler design. Individual components can then be tested and verified as needed, be that automated testing on code or user acceptance testing on a dashboard.

Extensible pipelines easily accommodate change. They follow strong design rules that make adding new functionality straightforward: this is how things are named, tested, organized and more. Extensibility builds on top of inspectability and maintainability, but it puts more emphasis on the reality of changing business requirements. Will your ingestion code work well if the API changes? Will your dashboard work well if KPIs are redefined? 

Conclusion

Data teams deliver informational and operational value, which depend on data pipelines that are inspectable, maintainable and extensible. In the upcoming posts we’ll use these two frameworks to discuss the various components of the Modern Data Team and the value they each provide. Our first framework helps us understand and classify the output – a common measure of success – while the second helps us put emphasis on the future utility of your pipeline – a much harder-to-pinpoint measure. Stay tuned for our next posts on individual contributors, then on managers and finally on product owners.

A successful data team needs a modern data stack to power its data integration efforts.

Learn how

Montreal Analytics is a Modern Data Stack consulting firm of 45+ people based out of North America. We help our clients on the whole data journey: pipelines, warehousing, modeling, visualization and activation, using technologies like Fivetran, Snowflake, dbt, Sigma, Looker and Census. From strategic advisory to hands-on development and enablement, our agile team can deploy greenfield data platforms, tackle complex migrations and audit & refactor entangled data models.

No items found.

Related blog posts

No items found.
No items found.
No items found.

Start for free

Join the thousands of companies using Fivetran to centralize and transform their data.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.