Watch out for "Immortal Time" in conversion rates

Learn about "immortal time" and how it impacts conversion rates
May 2, 2023

For those of us who work with data every day, biomedicine can be a rich source of guidance for how to work with data well, and what biases we need to watch out for. Recently at Fivetran, we were looking at conversion rates of prospective customers, and we ran into a classic source of bias called "immortal time."

What is immortal time? Suppose we conduct a study where we compare patients who took a drug to patients who did not. We measure these patients' survival from when they enter the study. A dead patient can't fill a prescription, so during the time between the start of the study and when they fill their prescriptions, these patients are "immortal." This is a bias that makes the prescription seem to work better than it actually does.

Credit: Catalog of Bias

A cousin of this problem came up at Fivetran when we compared the conversion rate of prospective customers who used transformations to everyone else. The problem is that a prospective customer who has already given up can't set up transformations, so this created a period of immortal time between when they initiated a trial, and when they set up transformations.

Initially, we thought that transformations users were more likely to convert than non-transformations users. But after recognizing the bias, we changed our analysis to look at conversion rates after the first successful sync. This dramatically reduces the amount of immortal time in the transformations group.

Once we made this adjustment, the difference between the groups vanished. We still believe transformations are a valuable product feature, but it doesn't appear that they influence conversion rates. This experience was a good example of how we can use biomedicine concepts to be better analysts of business data.

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Data insights
Data insights

Watch out for "Immortal Time" in conversion rates

Watch out for "Immortal Time" in conversion rates

May 2, 2023
May 2, 2023
Watch out for "Immortal Time" in conversion rates
Learn about "immortal time" and how it impacts conversion rates

For those of us who work with data every day, biomedicine can be a rich source of guidance for how to work with data well, and what biases we need to watch out for. Recently at Fivetran, we were looking at conversion rates of prospective customers, and we ran into a classic source of bias called "immortal time."

What is immortal time? Suppose we conduct a study where we compare patients who took a drug to patients who did not. We measure these patients' survival from when they enter the study. A dead patient can't fill a prescription, so during the time between the start of the study and when they fill their prescriptions, these patients are "immortal." This is a bias that makes the prescription seem to work better than it actually does.

Credit: Catalog of Bias

A cousin of this problem came up at Fivetran when we compared the conversion rate of prospective customers who used transformations to everyone else. The problem is that a prospective customer who has already given up can't set up transformations, so this created a period of immortal time between when they initiated a trial, and when they set up transformations.

Initially, we thought that transformations users were more likely to convert than non-transformations users. But after recognizing the bias, we changed our analysis to look at conversion rates after the first successful sync. This dramatically reduces the amount of immortal time in the transformations group.

Once we made this adjustment, the difference between the groups vanished. We still believe transformations are a valuable product feature, but it doesn't appear that they influence conversion rates. This experience was a good example of how we can use biomedicine concepts to be better analysts of business data.

Topics
No items found.
Share

Verwandte Beiträge

How fast is DuckDB really?
Data insights

How fast is DuckDB really?

Beitrag lesen
Evidence-based business
Data insights

Evidence-based business

Beitrag lesen
Cloud Data Warehouse-Benchmark
Data insights

Cloud Data Warehouse-Benchmark

Beitrag lesen
No items found.
FivetranChat: A homebrewed generative AI story
Blog

FivetranChat: A homebrewed generative AI story

Beitrag lesen
Unlock 5x productivity: How global enterprises scale with Fivetran
Blog

Unlock 5x productivity: How global enterprises scale with Fivetran

Beitrag lesen
A change to our Transformations pricing structure
Blog

A change to our Transformations pricing structure

Beitrag lesen

Kostenlos starten

Schließen auch Sie sich den Tausenden von Unternehmen an, die ihre Daten mithilfe von Fivetran zentralisieren und transformieren.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.