For those of us who work with data every day, biomedicine can be a rich source of guidance for how to work with data well, and what biases we need to watch out for. Recently at Fivetran, we were looking at conversion rates of prospective customers, and we ran into a classic source of bias called "immortal time."

What is immortal time? Suppose we conduct a study where we compare patients who took a drug to patients who did not. We measure these patients' survival from when they enter the study. A dead patient can't fill a prescription, so during the time between the start of the study and when they fill their prescriptions, these patients are "immortal." This is a bias that makes the prescription seem to work better than it actually does.

A cousin of this problem came up at Fivetran when we compared the conversion rate of prospective customers who used transformations to everyone else. The problem is that a prospective customer who has already given up can't set up transformations, so this created a period of immortal time between when they initiated a trial, and when they set up transformations.

Initially, we thought that transformations users were more likely to convert than non-transformations users. But after recognizing the bias, we changed our analysis to look at conversion rates after the first successful sync. This dramatically reduces the amount of immortal time in the transformations group.

Once we made this adjustment, the difference between the groups vanished. We still believe transformations are a valuable product feature, but it doesn't appear that they influence conversion rates. This experience was a good example of how we can use biomedicine concepts to be better analysts of business data.