Background
Over the Summer in 2024, several teams were brought together for a large in-person discovery session around the customer & business paint points around our inventory experience from the back end balance on hand data to the front end experience that our customers see.
From this discovery, we compiled a list of ideas & features to test or gather additional research on. Utilizing a difficulty matrix, each team championed a particular idea to focus their test & learn efforts on. Item Display (my team) chose to focus on surfacing substitutions for low stock items, Checkout & Post Order focused on substitutions for out of stock items, and the available to promise team focused on enhancing inventory projection capabilities.
The Problem
While each teams efforts did show promising improvements for our customers in initial AB testing, there was a concern from design & product leadership that the experience could potentially be disjointed due to each team working only in their respective silos.
I was selected to lead the charge in bringing the teams together to align on a vision, and determine which experiments made the most sense to move forward with in the near future. This ask required me to work with and across 4 additional teams outside of my direct domain team, and each team had its own product manager, lead designer, and engineering team.
Inventory Experience service blueprint at the time of initial discovery (created by Nichole Chaney, PD on Available to Promise Team). Their team ran the back end service that impacted the front end teams across Item Display, Cart, Checkout, & Post Oder.
Discovery Kickoff: Front End Customer Experience
With the teams aligned on bringing our efforts together, I kicked off phase 1 of our discovery with representatives from all 5 teams. In our kickoff we focused on level setting on understanding the new data service (Inventory Projections) being provided by the Available to Promise team, and how we could utilize this new capability to provide a better experience for our customers in a cohesive way.
We began with understanding how the service could be brought to the front end, what best practices say today, what our hopes & fears were, and the beginnings of a roadmap of how we would complete discovery and move on to testing. At the end of the kickoff, it was decided that this initial effort would be focused on the customer experience around Out of Stock Items.
Discovery Phase 2:
Understanding Assumptions & Opportunities for Out of Stock Items
With the focus on Out of Stock items on the front end experience -- our next session we spent some time looking at our current OOS experience, journey mapping with an OOS item lense, & assumption mapping around OOS items.
From there my PM and I helped to organize the stickies onto a matrix, and then each team prioritized their list of opportunities for areas to dig deeper (analytics & AB testing, small UI tweaks, or user research).
How might we let customers know when the items they want will be back in stock?
The item display team initially had the idea of displaying back in stock dates on product cards and our details page. This leaned on the assumption that customers wanted to know when an item would be back in stock, and that this change would be so small that it would do minimal harm to the customer experience in AB testing.
Design Opportunity: Usability Testing & Customer Interviews
While we had a clear understanding of what AB tests could be run for work already in-flight -- the assumptions we had for small UI changes that would potentially "do no harm" lacked actual customer sentiment.
Because of this, design aligned on running usability testing in Q1 and partnering with the user research team for customer interviews.
Unmoderated Usability Testing
We first began with a quick unmoderated usability test to showcase the small UI change of including back in stock dates on the cards & PDP.
I wrote the research plan, created the test, and created the prototype. Another designer from the Post Order team supported me on the analysis of the feedback and compiling the findings into Dovetail.
High-Level Findings
From the usability testing, the majority participants stated that the would not consider waiting or pushing their order based on a back in stock date for an out of stock item. Users stated that if it was an item they really needed, they were more likely to look for an alternative or physically go into our store or another store.
This actually disproved our assumption that customers may be willing to be flexible with their order date (like potentially delaying it) even if a back in stock date was shown.
Moderated Interviews
Then we had the opportunity to moderate 5 customer interviews with the help of the user research team. This allowed us to dive deeper into their responses and also cover a few different entry points for inventory including proposed addition of back in stock dates on product card and the enhanced Cart + Checkout substitutions experience.
For this piece of the work I pulled in the designer from the available to promise team to help me create the research plan & interview script. Then I pulled in the support of the designer over the Cart + Checkout space to help build the prototype, and then myself and the available to promise designer co-led the interview moderation.
Takeaways
While customers stated that they would like to know when an item might be back in stock when it came up during our conversation, when we showed them visuals of a back in stock date -- many customers back tracked their responses and were concerned about the accuracy of the dates. Without an available action to hold or reserve that item for when it was back in stock, the dates introduced more concerns and confusion.
For OOS experiences in other parts of the experience, customers found adding substitutions easy and effective and found the new enhanced experience to be intuitive. However, some customers expressed that they wish they new an item was unavailable sooner in the experience than the Cart or Checkout space. They desired transparency sooner so that they could have better control & visibility of alternatives.
Solution: Button state changes
Given the feedback from customers, it was very clear that even a small UI change like surfacing back in stock dates would not be helpful to our customers and may in some cases cause additional confusion.
But with the desire for transparency, and the business concern regarding blocked sales -- our team aligned on a solution to simply change the button states from Out of Stock to our standard Add to Cart when we had inventory prediction data available.
For example, if a customer is shopping online with us on a Wednesday (when an item is currently out of stock), but they are scheduling their order for Friday (when the item is expected to be back in stock), we would leverage that available data to make the item appear available to our customer and allow them to add to cart.
