An amateur photographer is shopping for a new camera. She strolls into a store, credit card in hand, finds the right section, and starts scanning the shelves. Except that this is 2022, not 2002. She’s not searching a store but browsing a website. And there are not tens of cameras on offer, but hundreds, possibly thousands. How does she choose? And how can marketers predict her decision-making in order to optimize their pricing?
In a physical store, that customer can see all the available products right in front of her. She can inspect them and work out which one is best for her. But anyone who has shopped online knows the vast range of products makes it far trickier to choose. Choice is a great thing, but it can also be confusing – both for customer and retailer. And in a global e-commerce market worth over 5 trillion dollars, online retailers can't afford to confuse consumers.
Retail websites need practical algorithms specific to online shopping behaviors. Understanding customers’ click and search behavior is crucial for helping them make a variety of operational and marketing decisions in the context of web analytics. Understanding the likelihood that a customer will click on a certain item in a certain place on the list, for example, will help the retailer price and rank products in order to maximize its expected total revenues.
Updating the traditional model
So, we wanted to figure out how customers search and make purchasing decisions online. Instead of focusing purely on click rates and display position of each product on the page (its ranking), we set out to drive up revenues by creating an algorithm that learns from customer behavior and to optimize product ranking and pricing.
We wanted to understand how customers search and make purchasing decisions online, to drive up revenues by creating an algorithm specific to online shopping behaviors.
We used the framework of a commonly used online search model known as the Cascade Click model. This model assumes that customers search products in order from the top of the search results downwards, one by one, and that they do not revisit a product they have already looked at. At each product in the list, the customer can decide to click on it to reveal more details, and then to buy the product or return to the list of results. They may also decide to terminate the search without buying anything.
This is a well-established model that captures a lot of information and is simple to implement. Many versions of this model assume that customers have enough time to go through every single product, and then choose the one that has the maximum desirability and maximum utility. But given that online retail has exploded since the traditional models of choice were first developed, we thought this highly unlikely. You might be provided with a list of 400 products, but only end up looking in detail at five of them. In other terms, existing models do not seem to make sense anymore in a world of near-boundless choice.
We set out to establish the parameters of the Cascade Click model and use those parameters to optimize prices and ranking. We observed how customers behaved and then we aggregated those behaviors to make a generalized assumption about how customers will behave. And from that, we create an algorithm that provides the retailer with an optimum price, given those parameters.
Developing algorithms for different online settings
We found that blindly applying a general algorithm without adapting it to the Cascade Click model is highly inefficient for the retailer. Our research shows that doing this would lead to huge losses in total revenue.
We first created a base algorithm using the structure of the Cascade Click model, in which the display position of each product is fixed, and the retailer only has to decide the pricing. This is a stepping-stone to more complex online shopping settings, but also stands alone as a useful tool for retailers that do not allow their customers to filter products. These include the high-end fashion websites of Dior, Kenzo, and Alexander Wang. We then adapted that algorithm to incorporate all the website’s historical click and purchase information.
And finally, we extended the algorithm to consider the ranking and pricing of products in parallel. This takes into account filters applied by the customer. Developing this model was trickier. We started by modifying the algorithm to work for a relatively small number of possible display rankings and re-tunes as the number of possible rankings increase. Then we added a rule that updated the algorithm for websites with a larger number of display rankings.
We found that this algorithm significantly improves performance. Decisions about ranking and pricing are often made separately, by completely different departments, but our research highlights how important it is for retailers to consider these decisions in parallel.
Developing the algorithms further
While we have studied approaches to ranking and pricing products on a webpage, we have not considered assortment decisions – the subset of products a retailer may choose to display. With limited display space in many applications, choosing which products to display is an important decision. Our findings suggest it would be worthwhile to combine this with pricing and ranking.
We would be interested in combining click models with classic choice models, because they can capture other aspects of customer behavior. We would also be keen to work with industry collaborators on our work to date.
