Have you ever wondered how Amazon, Netflix and Spotify make product recommendations? At the heart of these systems is a technique called Market Basket Analysis.
Defining actions around what products customers buy in bundles is a well-known marketing strategy. Nonetheless, some product combinations are easy to identify, while others need more analysis.
Examples of Market Basket Analysis are everywhere. From product recommendations when shopping online, email campaigns, coupon offers and much more.
Would you like to know how top retailers make relevant product recommendations?
This post will show you what Market Basket Analysis is and how it can increase your sales. Firstly, here’s the table of contents. Feel free to jump to any section that interests you:
What Is Market Basket Analysis?
Market Basket Analysis is a technique that looks for combinations of products in purchases. Most importantly, its basis is that customers who buy a particular group of products are more or less likely to buy another group.
An analysis of this kind:
- It is performed with transactional data. For example, product items of each invoice.
- Establishes the strength of association between products purchased together.
- Allows the businesses to estimate how likely it is that two or more events occur together.
- Produces rules in the form of if-then scenarios. These rules are probabilistic. That is to say, how likely is an event to happen, not that it will always happen.
The Objective of Market Basket Analysis
Market Basket Analysis models aim to identify the next product that might interest a customer. As a result, marketing and sales teams can develop more effective pricing, product placement, cross-sell and up-sell strategies.
It can help predict product sales in specific locations, improving shipping times and warehouse operations.
Therefore, it translates into increased revenues, lower costs and higher profits.
Market Basket Analysis Example
Consider the following example at Amazon:
See the extra products? Do you think the customer will buy them too?
This Amazon feature is powered by software that makes sense of the vast amounts of data they collect.
For the customer, this is convenient, as he or she will not have to return later to buy another item. Also, for the business, this means more sales. Everybody wins.
Furthermore, a business can go beyond showing related items to customers. For instance:
- Offer a discount for extra items at the time of sale.
- Later, send the customer a newsletter or email campaign with attractive product bundles.
- After finishing the sale, offer a coupon enticing the customer to return and buy the extra items.
- If you have a brick-and-mortar store, you could place the items close to one another.
What Is Market Basket Analysis Used For?
Market Basket Analysis produces a list of product sets with their probability.
Such a list can be helpful to:
- Know the attraction or repellence between products. As a result, a business can make better-informed decisions about product placement.
- Rank each set of products by their probability. So, you can learn which associations are prevalent over others.
- Determine which products to place close to one another to increase bundle purchases.
- Plan marketing promotions around products that drive the sale of other products.
- Avoid including both products in the same promotion (Since you know one will drive the sale of the other).
- Increase cross-selling and up-selling opportunities. You can do this since you know which products are top sellers and which ones drive extra sales.
- Helps marketing and sales organizations distribute their resources better (Channel optimization).
- Deliver online marketing campaigns to customers based on their purchasing behavior.
To accomplish this, companies use recommendation engines. These are software components that make recommendation rules available to other systems. For example, an online store could integrate a recommendation engine to make offers or product placements.
In short, Market Basket Analysis is a valuable way to leverage the information you already have to predict what customers will buy.
Where Can It Be Used
The applications of Market Basket Analysis go beyond online businesses and retailers. Furthermore, any activity where identifying patterns or behavior from transactional data can benefit. For example:
- Fraud detection: How likely is a US customer to pay a bill from Russia with his credit card? A business could flag suspicious transactions for further investigation.
- Patients with health risks: You could find patients with a high risk of adverse events based on their health records.
- Web or Email Filter: Association rules sets can identify spam emails. For example, how likely is an email containing the word “Viagra” to be Spam? Association rule learning can uncover the correlation between these two events.
Boost your software development with nearshore teams
BairesDev provides end-to-end software outsourcing extending your in-house capacity and expertise, from definition and design, to development and testing.
Market Basket Analysis and Recommendation Engines
Retailers pioneered this technique to get insights into optimizing shelf space to drive sales. In the early days, firstly, they would use data mining algorithms to generate reports. Secondly, the reports would tell them what association rules were the strongest. Furthermore, they would plan promotions and identify cross-selling opportunities with this information.
Market Basket Analysis is one example of data mining techniques; check out this link for more examples of data mining in real life.
Then came the internet, retailers went online, and new opportunities presented themselves. Accordingly, now online retailers can consider making product recommendations when customers browse a particular item.
Recommendation engines are software applications capable of identifying which related products to show. They are behind features like “people who bought these items also bought.”
Market Basket Analysis is not the only analytics technique used by recommendation engines. They can also take into account users’ past preferences and purchases.
Therefore, techniques used by recommendation engines can be:
- Customer-based: The recommendation engine keeps track of customers’ past purchases and browsed products. Moreover, it also tracks other visitors’ activity.
- Item-based: Show new items similar to or related to the current product.
Some examples of recommendation engines in action are:
- When Netflix shows you movies that you might like, have you noticed how accurate these predictions are? (That you might like these movies).
- When Spotify generates a “Radio Station” based on a current song or album.
- The main page of Amazon uses recommendation engines to show you items similar to those you have browsed in the past.
Market Basket Analysis Basics
A typical goal of Market Basket Analysis is to provide a set of association rules in the following form:
IF [antecedent] THEN [consequent]
The first part of the rule is the “body” or “antecedent,” while the second part of the rule is the “head” or “consequent.” Furthermore, the antecedent and consequent can include many conditions, creating more complicated rules. For example:
IF [‘Smartphone’] THEN [‘Case’]
A more complex rule could be:
IF [‘Smartphone’ , ‘Case’] THEN [‘Screen Protector’ , ‘Replacement Battery’]
Association rules have three reliability measures: Support, Confidence, and Lift.
Support
It measures the frequency of the association rule in the data. For instance:
The number of transactions with ‘Smartphone,’ ‘Case,’ ‘Screen Protector,’ and ‘Replacement Battery.’
/
Total number of transactions.
A 3% support would mean that 3 out of 100 sales include the four mentioned items.
Confidence
Confidence measures how strong an association rule is. Following is an example of how to calculate it:
Confidence = # Transactions with ‘Smartphone,’ ‘Case,’ ‘Screen Protector, and ‘Replacement Battery.’
/
# Transactions with ‘Smartphone’ and ‘Case.’
A 40% confidence means that 4 out of 10 sale transactions with a Smartphone and a Case also include a Screen Protector and Replacement Battery.
In other words, if you buy a Smartphone and a Case, you are 40% likely to buy a Screen Protector and Replacement Battery.
Market Basket Analysis Lift
Lift is the ratio of confidence to expected confidence.
Expected confidence is the confidence of the “consequent” condition. That is to say, the number of purchases with the consequent condition is divided by the total number of transactions. As a result, lift tell us how much better a rule is at predicting the outcome than just assuming it in the first place.
Here’s an example of how to calculate lift:
Confidence
/
(# Transactions with ‘Smartphone’, ‘Case’ / # Total Transactions)
Lift is a valuable metric to determine to rank and prioritize lift association rules. Furthermore, the higher the lift and support, the better. We cover those topics in our post on Market Basket Analysis Lift.
Furthermore, another name for calculating lift between two events, or in this case, products, is two way lift. Moreover, lift can also be calculated between three events. Check out our post about computing two way and three way lift for more on this topic.
Association Rules Algorithms
The most famous algorithm generating these rules is the Apriori algorithm.
The Apriori algorithm is designed to operate on databases containing transactions. For example, items bought by customers.
Firstly, it identifies common individual elements in the dataset. Secondly, it extends the set by adding more and more, as long as they appear often enough. Afterward, it calculates each rule’s “Support” and “Confidence”, making it possible to examine them according to their importance.
Many programming languages include the Apriori algorithm as a package.
It is helpful to set thresholds for support and confidence when executing the Apriori algorithm.
There are other algorithms for generating association rules, such as Eclat and FP-Growth. They also are good at mining frequent itemsets found in databases.
What You Get out of Market Basket Analysis
Once you run the algorithm and generate the association rules, the result looks like this:
The strongest association rules with higher lift, confidence, and support. Therefore, they are the ones you should deploy to the recommendation engine and use to determine store layout, promotions and cross-selling strategies.
How to Implement Market Basket Analysis
Assessing Your Current Capabilities
To implement a Market Basket Analysis, you need a point-of-sale (POS) system already in place. Furthermore, your sales system needs to store the item, unit price and quantity of each sale at the very least.
For most businesses, this information is sensitive. Thus you need to consider secure ways to extract it. For instance, you could set a mechanism to retrieve only the data you need to perform the analysis.
Assess the Transactional Data You Have
Association rules need to appear in hundreds of transactions to be statistically significant. So, you need enough data for a Market Basket Analysis to be meaningful. The Data Scientist you hire can help you determine if you have.
Sometimes terms like Data Science and Artificial Intelligence are used interchangeably. However, they are in different fields. Check out our post where we explain Data Science vs AI.
The good news is that even the smallest businesses already have large amounts of data. Thus, Market Basket Analysis is within their reach.
Also, you must consider the following:
- If you have the adequate means to store and then extract the data.
- Quality of the data: Is it complete, reliable and of acceptable format?
It could help to hire consultants to perform a data quality assessment before committing more resources. In other words, you must obtain incomplete data and improve its accuracy before moving on with the project.
What Software and Infrastructure Do I Need?
You will need hardware and software to capture and use data to build a Market Basket Analysis solution. Moreover, you have two options: Develop an internal infrastructure or go to the cloud.
If you develop internally, you must buy and maintain your servers and IT infrastructure. Secondly, you must purchase or build a data mining and predictive analytics software package. Also, you will need to hire consultants to integrate the software components.
On the other hand, using a cloud service has the advantage that the provider handles server and infrastructure provisioning and maintenance. However, cloud services usually charge by usage, adding operational costs. Still, for small and mid-sized businesses, they are lower than maintaining servers and infrastructure.
You still need to develop or get a cloud software service for an Analytics package. Also, you will need to hire consultants to integrate your sales systems.
Assembling the Necessary Software Components
To build a complete Market Basket Analysis solution, you need an association rules learner:
Association Rules Learner
An Association rules learner is a computer program that identifies association rules in sales data. Above all, it involves a software package that:
- Reads basket and product data.
- Runs algorithms to build rules sets.
- Match product id and names. As a result, the rules become easier to read.
- Calculates price and revenue stats to rank rules.
- Write a file with the association rules.
You can produce and use product combination reports with an association rules learner. Still, this is only the first step.
Deploying the Association Rules to a Recommendation Engine
If you have an online store, you can integrate a recommendation engine to give your customers product recommendations in real time. Hence, taking Market Basket Analysis to the next level.
The system will receive a file with the association rules generated by your learner. Then, when prompted by the website application, it can tell it which related products to show.
You need to link the online application with the recommendation engine to build the solution. For example, you could publish the recommendation engine as a web service.
In its workings, a recommendation engine performs the following tasks:
- Read the current product or basket content.
- Read association rules.
- In rules antecedents, find the closest match to the current product or basket content.
- Find rules with higher support, confidence and lift.
- Return or export to a file the product recommendations.
How Does a Market Basket Solution Look Like
Once implemented, a Market Basket Analysis system could look like this:
Serving its intended purpose requires integrating the sales application and data mining software with the recommendation engine.
Critical Success Factors
The success of a project like this requires the following:
- Accurate, reliable and high-quality data.
- Adequate methods for gathering and storing data.
- Support from top management.
- Teamwork and collaboration between business users and implementation team members, including data scientists and machine learning engineers.
- Necessary computing power.
- Professionals with the right set of skills.
More on Market Basket Analysis
Would you like to learn how to use this data mining technique in practice? Check out our post: How To Conduct a Market Basket Analysis in Tableau.
Also, while Market Basket Analysis provides a set of association rules you can work on, you still need to rank and prioritize your marketing actions around them. Check out our post about Lift in Market Basket Analysis to learn how.
Increase Your Sales With Market Basket Analysis
This article has explored how Market Basket Analysis can help you increase your sales.
Getting started might require some work. Firstly, you need to address quality issues with data. Secondly, you need to plan and execute a project, decide between an internal IT base or going for the cloud, hire consultants, and more.
The potential benefits of Market Basket Analysis outweigh the challenge. Hence, we invite you to explore how this technique can help you take your business to the next level.
What do you think about Market Basket Analysis? Have you tried it out? In what ways do you envision Market Basket Analysis helping your organization?
Subscribe to Our Email List
Did you like our article? Would you like to get free updates directly in your email? Subscribe to our list in the form below.
References
Albion Research LDT. Market Basket Analysis.
IBM Knowledge Center – Market Basket Analysis.
Li, S. A Gentle Introduction on Market Basket Analysis. Published in Towards Data Science.
Rasmus Borregaard Thomsen. Using machine learning in market basket analysis.
Select Statistical Consultants. Understanding Customer Behaviours.
Silipo, R. Learn Market Basket Analysis and Recommendation Engines using real examples.