By Joshua Mews, Objective Computing.
Using a Product Recommendation Engine
Product recommendation systems utilise filtering systems to identify products to recommend. The type of filtering system that is used for a product recommendation engine depends on what is trying to be achieved. Whether you’re after quick wins, or a system that has a full ability to learn and develop from a huge data set, product recommendation engines can be used by many different businesses in many industries.
“A lot of times, people don’t know what they want until you show it to them” – Steve Jobs, former CEO of Apple Inc.
Since the increase in availability of data due to the Internet of Things (IoT), product recommendation engines (PREs) have become a very popular tool for businesses. Product recommendation engines, such as those developed by Amazon and Netflix, have had huge levels of success. Amazon have stated that roughly 35% of their revenue is developed by its recommendation engine, whilst Netflix estimate that up to 75% of viewer activity is driven by recommendation.
In a time where consumers are wanting more content personalisation than ever, product recommendation engines can provide this. Frequently accessing millions of point of data, PREs typically use one of two types of filtering system, collaborative or content-based.
Collaborative System Filtering
Collaborative systems use a filter that identifies search habits of similar users and recommends a product that it believes the user may be interested in; an example of this would be Amazon’s ‘customers who viewed this item also viewed this’ recommendation. The idea behind this sense of filtering being the belief that the best recommendations will come from those with similar interests to themselves.
A common method used for a collaborative filter is the K-Nearest Neighbours (K-NN) algorithm. The K-NN algorithm classifies items in accordance of their locality to their nearest neighbours within a multi-dimensional space. Normally, contributions of neighbours are weighted in accordance with their distance from the object. Once the algorithm is run, you would be able to identify the nearest neighbours, and in turn use this output data to fuel your product recommendation engine.
However, there can be drawbacks to collaborative filtering methods, most commonly the ‘cold start’ problem. The system may initially struggle to produce accurate predictions as new users will have little to no previous data, as collaborative filtering methods recommend based on user’s past searches. This is similar to when new products are added into a system; they will need a substantial number of ratings before they can be recommended to users with similar tastes.
Content Based Filtering
On the other hand, content based filtering methods do not suffer from this ‘cold start’ problem. These systems use a filter that identifies similar keywords that the user is currently using to identify products they may be most interested in. This is why there is no ‘cold start’ problem with this method, as the current search keywords can be used by an algorithm to recommend products.
A common algorithm used within content based filtering is the TF-IDF algorithm. This algorithm shows us the relative frequency of a keyword within a document. It does this by first measuring the search term frequency against results, and then compares this with the relative frequency of the search terms amongst all results. An example of this could be the search term ‘the development of IoT; it’s likely that ‘the’ will be the most common search term, however ‘IoT’ will have the greatest search power due to lack of frequency.
Pandora internet radio is an example of content based filtering in action. This radio station provides music with similar characteristics to one that in initially provided by the user.
Similar to collaborative filtering systems, content based filtering also has its limitations. Because the recommendations are drawn completely from within the system, the system will have a reduced ability to learn; there is also no true user personalisation as it doesn’t take into account any user history, instead using its own history for recommendations.
Larger, more complex product recommendation systems can use a hybrid system approach however. Using a merger of both collaborative and content based filtering approaches, these styles of recommendation engines have proven to have a greater accuracy when compared with single filter systems. These better recommendations mean an increase in sales and engagement for the business.
A rise in the use of bandit algorithms have been used for product recommendation engines more recently. These contextual bandit algorithms have the ability to learn from every single interaction that takes place.
Contextual Bandits work through collecting information about how the context vectors and rewards relate to each other (in this case the product recommended and whether or not a sale / engagement was achieved), so that it can then predict the next best product to recommend.
Bandits need a metric to determine how well a certain strategy is performing. This metric is known as ‘regret’, which can be defined as the difference between a chosen strategies’ success with an optimal strategies’ success. As decisions are made, the bandit learns from its decisions and minimises regret until an optimal strategy has been discovered. In the case on a product recommendation engine, this would mean the best products are being recommended for engagement.
According to how a bandit’s algorithm has been programmed, which run on a compromise between exploiting the current optimal choice and analysing sub optimal strategies, banding may become inefficient. If the initial exploitation rate is set too low, then the bandit may incorrectly identify an optimal strategy. However, if the initial exploration rate is set too high the bandit may take too long to correctly identify the optimal strategy, wasting precious resources.
Look out for our goody bags as you take your seat at the event! They contain an opportunity to win a Fitbit just by identifying the solution to the puzzles!