How Online Stores Know What You’ll Buy Next: The Math Behind “Frequently Bought Together”

Amazon and other online retailers seem to know exactly what you’ll add to your cart next. How? In physical stores, products are organized into sections, aisles, and shelves, making it easy to find related items together. This real-world experience was missing from early digital stores, where breadcrumbs were the only way to navigate categories. E-commerce sites have only recently begun to replicate the convenience of finding similar products grouped together, just like in a physical store.

You're back-to-school shopping and add a notebook to your cart. Suddenly, you're shown pens, glue sticks, backpacks, printer ink, and scissors all the essentials, right there in one swipeable carousel. So convenient, way better than hunting them down aisle by aisle at Walmart.

But how do eCommerce sites know exactly what to show you?

🎉 Drumroll... Enter Association Rule Mining the behind-the-scenes magic of data mining that finds patterns, relationships, and "frequently bought together" combos across massive datasets.

Even though these products are made, shipped, and sold by totally different suppliers, this tech connects the dots — making your shopping experience smarter and smoother.

Association Rule Mining helps computers find these patterns automatically from huge amounts of data, so businesses can use them to make better decisions like showing related products online or organizing store shelves smarter.

Think of it like this: "If X, then Y" that's the heart of association rule mining.

The Apriori Algorithm is a classic method used in Association Rule Mining to find items that frequently appear together in large datasets like in shopping carts, web clicks, or user behaviors. So, if we have seen previous patterns where products being bought together are used as guiding tools to create these associations.

The Apriori Algorithm

Explanation

• Purpose: Identify frequent itemsets and derive association rules.
• Steps:
• Identify all itemsets that meet a minimum support threshold.
• Generate larger itemsets from smaller frequent itemsets.
• Derive association rules from frequent itemsets that meet minimum confidence.
• Mathematical Intuition:
• Support: Frequency of itemset in dataset.
• Confidence: Likelihood of consequent given antecedent.
• Lift: How much more likely the consequent is, given the antecedent, compared to random chance.

Mental map

Data Preprocessing & Visualization

The table below shows the likelihood of items being purchased together. For example, if a customer buys a SPACEBOY LUNCH BOX, they are also likely to buy a DOLLY GIRL LUNCH BOX. This pattern suggests that parents may be purchasing lunch boxes for both their son and daughter.

Similarly, if someone buys all these ROSES REGENCY TEACUP AND SAUCER , PINK REGENCY TEACUP AND SAUCER then they are likely to buy Green one as well to complete color combination of the teasets.

Interpretation

• antecedents: The item(s) on the left side of the rule (the "if" part). For example, (ALARM CLOCK BAKELIKE RED ) means the rule is considering transactions that include this item.
• consequents: The item(s) on the right side of the rule (the "then" part). For example, (ALARM CLOCK BAKELIKE GREEN) means the rule is predicting that this item is also likely to be present.
• support: The proportion of transactions that contain both the antecedent and the consequent. For example, 0.028593 means about 2.86% of all transactions contain both items.
• confidence: The probability that the consequent is present given the antecedent is present. For example, 0.604333 means that if a transaction contains the antecedent, there is a 60.4% chance it also contains the consequent.
• lift: How much more likely the consequent is to appear with the antecedent than it would be by chance. A lift greater than 1 means the rule is useful; for example, 14.197612 means the items appear together about 14 times more often than if they were independent.

Support: Proportion of transactions containing the itemset.

• Confidence: Probability of consequent given antecedent.
• Lift: Ratio of observed support to expected support if independent.
• Conviction: Measure of implication strength.

Retailers use association rule mining to optimize product placement and promotions. For example, if analysis shows that customers who buy a SPACEBOY LUNCH BOX often also buy a DOLLY GIRL LUNCH BOX, the store can place these items together on shelves or offer bundle discounts. This increases the likelihood of customers purchasing both items, boosting sales and improving click count on their page.

Try Next:

We have used the Apriori algorithm to discover associations, but there are other effective options such as FP-Growth and Eclat. By creating a pipeline that runs the data through all these algorithms and applies a scoring system, we can select rules that are identified by at least two methods and have strong metrics. This approach increases confidence in the reliability of the discovered associations.

Practical Retail Application: Digital Shelf Optimization

This approach can be used to pre-populate related products in an online system, allowing users to quickly find complementary items when searching for a product. In a physical store, this is similar to placing frequently bought-together items side by side on shelves. By replicating this model digitally, we streamline the shopping experience, saving users time and effort while increasing the likelihood of additional purchases.

References-

• UCI Machine Learning Repository: Online Retail
• mlxtend documentation: http://rasbt.github.io/mlxtend/
• Han, J., Kamber, M., & Pei, J. (2012). Data Mining: Concepts and Techniques.