The following post is by Emily Hawkins, Data Infrastructure Lead at Drizly. Emily authored this post as part of our Data Champions program. If you’re interested in contributing to the blog or learning more about the Data Champions program, please get in touch.

Who we are

Drizly is the world’s largest alcohol marketplace and the best way to shop beer, wine and spirits. Our customers trust us to be part of their lives – their celebrations, parties, dinners and quiet nights at home. We are there when it matters, committed to life’s moments and the people who create them. We partner with the best retail stores in over 1,200 cities across North America to serve up the best buying experience. As a marketplace, retailers are the backbone of what makes Drizly possible. We partner with mom-and-pop liquor stores to big-box chains across the country to bring more and more customers Drizly coverage everyday.

Our data team

Our data team is currently nine people, but we’re growing! Check out our open roles.

Our tech stack includes Fivetran, dbt, Snowflake and Looker. Our data infrastructure has been significantly upgraded over the last year, and we couldn't be happier with where we are now. Check out how we’re using Snowflake shares with dbt to share data with our external partners!

Some history

Pre-2019, one of the main goals at Drizly was to add more stores. As we were growing, we knew more coverage meant we could bring the joy of Drizly’s service to more people. However, we didn’t know where, geographically, adding a store would have the most impact, or if the store added was going to be a successful partner.

Then in early 2019, we started disaggregating stores into their component parts:

• Availability: The area the store covers, the hours and days of week stores are open
• Capacity: The store’s ability to service an incremental order, how many drivers they had available, if they could legally use third party delivery drivers (Doordash, Postmates)
• Selection: The store’s inventory, if they had the items customers are looking for
• Price: The store’s item prices, if they were competitive in their market

Our first metric was the number of stores a user session had available. Then we quickly realized that many user sessions did not feature any available stores, so we made the percentage of sessions where stores had no on-demand availability (i.e., they didn’t have suitable hours, means of delivery or inventory) our metric. A few months later, we discovered that selection matters, in conjunction with stores on demand. Our most recent analysis has been around pricing effects, and how price variance changes the behavior of customers in a marketplace.

I’m going to cover the four main areas of disaggregation, and the importance of each to the success of the Drizly network:

• Coverage
• Availability
• Hours
• Capacity
• Selection
• Price

We will also discuss future directions: Fees, Minimums, ETAs and so much more

Coverage

Where should Drizly add new retailers?  

In mid-2019, Drizly analytics was taking on a new form. Our analytics team was getting somewhat of a restart, and we brought event tracking online with the help of production engineering, which resulted in more powerful data coming in around what each Drizly customer was experiencing on our platforms. We started leveraging dbt to transform this data into usable form. We knew adding new stores was important, but how important? Knowing that the bottom 50% of our stores only did 4% of our business, where should the sales team prioritize adding stores to see the greatest impact? We started thinking of new ways to measure the success of Drizly by using the event data that was now more easily analyzed thanks to dbt.

Our first metric, the number of available stores per session, tried to answer the question, where should Drizly add new stores? Where were potential customers not able to access Drizly at all? Where would adding a new retailer to our network have the most impact? This was the first step in our journey of evaluating the value of a retailer, as well as helping our sales team prioritize outreach to get these valuable stores online.

Availability

Where should Drizly add more retailers?

We iterated on the first metric quickly. We knew that Drizly offers the best experience when customers can order on demand, and get what they want delivered to them right now. We quickly realized we needed to look at on-demand coverage, not just the presence of stores, to get a better picture of the customer experience. We used this data to help us prioritize which areas Drizly needed to go after to grow our marketplace.

The most important thing was providing Drizly service to those that didn’t have on-demand coverage. We looked at zip codes that had a high percentage of sessions with no stores on demand, and realized those were where we needed to add stores, and/or work with current retailers to improve their on-demand availability.

We also found that moving from one store to two stores on demand had a big impact on conversion rate, as did moving from two to three. So now we had a strategy to determine where stores should be added, and the likely impact they would have on the Drizly network.

Customers convert at higher rates, and spend more per session as more stores are available on demand:

Hours

Are retailers open during the right days and hours?

Store hours are a really important part of availability. If a store is not online during the right days or hours, Drizly customers will not have a good experience, or be able to get what they want delivered on demand. We were able to figure out what hours would be most impactful for stores to be online, based on session traffic and other store availabilities in the same area, and provide stores with recommendations and potential GMV (gross merchandise value) impact for increasing their available hours.

Capacity

How many orders can a retailer successfully fulfill per hour?

Capacity as an analytics project is a newer concept at Drizly. Before COVID, stores rarely had too many orders to handle. As states shut down, Drizly usage skyrocketed. Stores hit capacity limits and either shut off, canceled orders, or failed to fulfill them. We had to come up with a way to use our current data on-the-fly to prevent stores from accepting orders we didn't think they could fulfill. This is exactly what we did. Capacity limits are now something we track and measure to ensure stores’ continued success, as well as to ensure customers have a great Drizly experience.

Selection

Do retailers have the right items?

As we explored more of our event data, we realized there were more factors that made certain areas outperform others. We discovered that another big factor that made customers more likely to convert was having great selection. Having great selection not only means having the items you want available, but also being able to get them delivered to you now.

For example, Tito’s Vodka 1.75L bottle makes up about 2% of all Drizly sales, and about 5% of all Liquor Drizly sales. If a store doesn't carry this item, they are missing out on a significant portion of customer orders. In practice, we calculate these percentages at a market level so we can capture differences in item preferences across the country, and recommend items that are popular locally to each store. We can let you guess the exact figures, but the Pareto principle definitely applies to the beverage market.

This metric is what we call the percent of selection by GMV. Each item’s importance is based on the percent of GMV it made up in the past year, relative to each DMA (designated market area). This metric allowed us to start predicting how each store will likely perform based on the items they carry and have available on Drizly. It also helps us assess the health of our network based on if customers had the items they wanted available to them.

Using this metric we can prioritize adding items we know customers are looking for to stores’ inventories, as well as surface item recommendations to our retail partners. We can tell a store, “you have 2,000 items, but you’re not carrying the 4th, 8th, 11th, and 12th most popular items on Drizly.” In general, customers convert at higher rates and spend more per session as selection increases. We have also found that selection drives positive conversion increases, separately, and in addition to, the number of stores on demand.

Price

Are the right items priced competitively?

Our most recent deep-dive concerned pricing effects in our marketplace. This was a really interesting analysis given that there is so much experimentation, research, and theory around behavioral economics and pricing in e-commerce marketplaces. With the results of this analysis, we know that price variance and price index play a significant role in how customers behave and purchase on Drizly.

Price variance is based on the number of unique price points a customer sees. For example, customers will behave differently if they see a Tito’s 750ml bottle at two stores, both for $27 vs two stores, one priced at $27 and one at $30.

Price index is the perceived price of an item in a specific market. For example, a Tito’s 750ml bottle in New York City may have a median price of $30, and is considered a good value, whereas in Dallas, Texas, the median price may be $25, and $30 would be perceived as being priced too high.

We can use this data to improve the customer experience as well as to improve retailer success within the Drizly network.

Conversion rate declines as a store’s price index increases:

What we’re doing now

As we saw continued success with these metrics, we realized Drizly should continue measuring our success against them. Our 2020 goals are based on availability and selection metrics, and these concepts are now pillars of our strategic competencies as an entire company.

All of these metrics offer an endless amount of depth to analyze, and we continue to uncover new insights every day.

Putting it all together

We now have many metrics to explain the different factors that make retailers and the Drizly marketplace successful. Coverage, hours, capacity, selection, item prices + price variance, store minimums, delivery fees, and ETAs all play a role in making the best customer experience on Drizly. Teams across Drizly also now have more concrete metrics to go after, and strategy to go after them in a quantitative and effective way. Putting data behind all of these metrics has helped Drizly set data-driven company goals, and track the impact of these efforts over time.