Content Recommendations That Work: Proper techniques

Introduction

Content recommendations are vital to content strategy success. Recommendation systems help audiences find new content they might like, keep them engaged and spending more time on a platform, and can also increase revenue. For effective recommendations, understanding the underlying principles is critical.

Imagine scrolling through your favorite social media platform and coming across an article or video that catches your eye. You may not have been actively searching for it, but the recommendation algorithm knew exactly what you would be interested in based on multiple factors. That's the power of content recommendations, they can provide personalized experiences for users and help them discover content that they may not have found otherwise.

How popular would have been YouTube without its proper recommendation strategy? Imagine if Instagram and TikTok didn't have a personalized content recommendation feature, users would have to rely solely on their exploration skills. TikTok would never become so popular without its content recommendations, and Instagram's Explore page wouldn't be as engaging without its personalized suggestions.

I'm not saying that their recommendations are perfect, but they demonstrate how content recommendations can be a game-changer for user engagement and, ultimately, revenue.

Everything that we will discuss in this series can be applied not only for web development, but also for any type of content creation where you can use a programming language to write the actual algorithm. So, whether you're a web developer, backend engineer, or just someone interested in how recommendations work, read on to learn about effective content recommendations.

This article is just a first Part of the series “Content Recommendations That Work”. Be sure to follow along and check out the remaining parts to get a comprehensive understanding of how to create effective content recommendations that truly work for your audience.

The next article will be the actual implementation of the Recommendation Pipeline System (RPS) that we will learn here. We'll use a programming language to code the algorithm and explain step by step how to integrate it with your website or platform.

Since there are different types of recommendations, we will be using a variety of techniques in upcoming articles to cover the main ones.

What to expect?

This article will cover all the theoretical knowledge you need to understand how content recommendations work. Since there are different types and systems of recommendations, we'll explore what those systems are and how they work.

We'll take a look at real-world examples of successful platforms that use good recommendation strategies, like YouTube, Medium, and Udemy. Additionally, we'll dive into the Recommendation Pipeline System (RPS) to give you a powerful tool for implementing easy to make recommendations without machine learning.

Understanding Recommendations

Before delving into the details of developing successful content recommendations, it is important to consider the key features of an algorithm that is suitable for generating effective content suggestions. A good algorithm for content recommendations should be able to provide accurate and personalized recommendations that match to the user's preferences.

This means that the algorithm should consider various factors such as user behavior, history, and interests to suggest content that is relevant and engaging for the user. We need to exclude content that the user has already seen or interacted with, and identify patterns in their behavior to predict what they might be interested in next.

Without proper recommendations, users may become overwhelmed by the amount of content available and miss out on pieces that could be valuable to them.

I actually think most people don't want Google to answer their questions. They want Google to tell them what they should be doing next.

Eric Schmidt

To translate the quote from Eric Schmidt, the former CEO of Google, to content recommendations, it's significant to understand that users don't just want a list of content options. Instead, they want to see what they should be doing next.

Types of Recommendations

Content recommendations are always involving, and as a result, there are many types of recommendations available to help users find new content. Moreover, the list of types is constantly expending as technology advances and new content platforms appear.

We are interested in the six types of recommendation systems that can be categorized as follows:

1. Collaborative filtering looks at the preferences and behaviors of similar users to recommend items to a user, based on their past behavior and preferences. Collaborative filtering is particularly useful in situations where there is a large amount of data available on users and their interactions. The weakness of collaborative filtering is that it may not be useful in cases where you don't have enough data about your users, or where users lack similar tastes and interests.
2. Content-based filtering analyzes the characteristics of the items themselves (such as keywords, tags, category) and recommends items that are similar to those that a user has already approved. This method is useful in both cases where there is a lot of data available, and when you have sparse data. There might be weaknesses of this method in that it may not recommend items outside a user's preferences, but it can be fixed by just improving the algorithm.
3. Demographic filtering uses demographic information such as age, gender, and location to recommend items that may be of interest to a particular group of users. For example, if the system determines that users in a particular age range or geographic location tend to prefer certain types of content (such as sports or news), it can prioritize or suggest that type of content to users in that segment.
4. Knowledge-based filtering recommends items based on explicit knowledge of the user's preferences, such as ratings or reviews they have provided. In knowledge-based filtering, the recommendation system may first ask the user to provide explicit information about their preferences, such as by filling out a survey or selecting from a set of predefined categories. The system then uses this information to build a user profile that captures the user's interests and preferences.
5. The Recommendation Pipeline System (RPS) is a fast and flexible framework that builds recommended content based on specific filters. Filters can have any logic that your program needs. The detailed explanation of RPS you'll get later in the article in a separate section.
6. Hybrid filtering combines multiple techniques to provide more accurate and personalized recommendations. Hybrid filtering can combine any two or more recommendation techniques. For example, a hybrid recommendation system may use collaborative filtering to suggest items based on the user's behavior and preferences, and also incorporate content-based filtering to suggest items that are similar to ones the user has liked in the past.

Note. It's worth noting that different authors and researchers may use slightly different terminology or groupings when categorizing recommendation systems, but the general principles and types of recommendation systems outlined above are widely recognized and studied.

In this article, we are not going to focus on these types of recommendation systems. Instead, we'll cover something called RPS, Recommendation Pipeline System. We'll talk about it later in the article, but we are not going to ignore the other ones.

Content-based recommendation system and Collaborative Filtering we will be covering in the future articles of this series, since these are the two most common types used in content recommendation engines. They have their own strong and weak points, and can be effective in different ways depending on the specifics of the platform, dataset, and user needs.

Machine learning

Machine learning is probably the most trending approach for developing recommendation systems in big companies. This approach involves training algorithms to learn patterns and make predictions based on data. Machine learning has the advantage of being able to consider a wide range of data, such as user behavior and preferences, item characteristics, and contextual factors.

However, this approach requires a lot of data and computational power to train the algorithms effectively. It's not enough to just know programming languages, you also need a solid understanding of statistics and data analysis to implement machine learning-based recommendation systems effectively.

It's a complicated topic where you need to have an understanding of machine learning concepts. If you're interested in implementing machine learning-based recommendation systems, there are plenty of machine learning courses on platforms like Udemy and Pluralsight available that can help you get started.

There are even TensorFlow JS courses which focus specifically on building recommendation systems using JavaScript. Lots of them have discounts during certain times of the year, so keep an eye out for those.

I will definitely write an article on how to get started with building recommendation systems using machine learning in the future articles of this series. But first, we should cover the implementation without machine learning because it's still widely used everywhere on the web, even on platforms with large amount of traffic.

Real-World Applications

Recommendation systems are becoming increasingly popular in various industries, from e-commerce to media and entertainment. You probably encounter them daily without even realizing it, such as when you are recommending a product while shopping online or when a streaming service suggests a movie or TV show for you to watch.

Let's take a look at a couple of examples of real-world applications with proper recommendation algorithms.

Medium

One example of a company that uses recommendation algorithms is Medium, the popular online publishing platform. Medium's recommendation system looks at a user's reading history, how much time they spent on an article, and which articles they liked, among other factors.

They mark these articles, and display them on the user's homepage under the “For you” section. If you read articles on Medium frequently, you've probably noticed how accurate their recommendations are. It gets messy when you start reading articles outside your typical interests, but overall the system does a good job at suggesting articles that you’ll enjoy.

YouTube

Another example is YouTube, the world's largest video-sharing platform. YouTube's recommendations are generated by an algorithm that analyzes a user's viewing history and suggests videos that the algorithm predicts the user will be interested in watching.

The algorithm considers a variety of factors, such as the user's watch history, the popularity of the videos, how long users usually spend watching videos of similar topics, and so much more. Some of these factors we will never know, as the algorithm is constantly being updated and refined.

The algorithm uses machine learning to continually improve its recommendations. It analyzes patterns in the user's viewing history and adjusts its suggestions accordingly.

Udemy

Lastly, Udemy definitely has one of the best recommendation systems for online learning platforms. They track your search history, watch history, and course ratings to recommend courses that match your interests and skill level. These recommendations appear on the user's home screen as “Because you searched for” and “Because you viewed” sections.

I haven't found any information about whether they use machine learning to achieve this or not, but their suggestions are usually very accurate and helpful. I like the separation of viewed content, and searched.

These are just a few examples of how recommendation systems can be used to enhance user experiences in various industries.

Implementing recommendations

Now that you have a more in-depth understanding of recommendation systems and their real-world applications, it's time to start thinking about how you can implement them in your app. We'll start by listing everything that we need to consider.

Here are 4 criteria to consider when implementing a smart content recommendation system:

1. Used content

Exclude content that the user has already used so that we can recommend new content. There is no point in recommending liked, viewed, shared, rated content or products that have been added to a wishlist or purchased by the user.

For viewed content, you can make a separate section called “Previously Viewed” or “Recently viewed”, so that users can easily access it if they want to revisit it.

2. Demographic information

In some cases, consider factors such as the user's age, gender, and location, as well as any other relevant demographic information, to help personalize recommendations. Keep in mind that this information should be obtained with the user's consent and handled in compliance with privacy regulations.

Demographic information is especially useful for ecommerce websites and apps, where recommendations can be tailored to fit the user's gender or age. It will be strange if a teenage girl is recommended men's clothing, right? Unless she wants to buy a gift for her dad.

3. Popularity

Consider the popularity of the content, such as the number of views, likes, and shares, to help determine which content to recommend. If many people are engaging with a particular type of content, it's likely that the user will be interested in it as well.

However, keep in mind that popularity should not be the only factor considered. It's important to balance popular content with niche or lesser-known pieces that may still be relevant and engaging for the user.

4. Content Diversity

When creating recommendations, it is important to suggest a variety of different content to the user. This is because if the system only recommends content that is similar to what the user has already seen, it can create a “filter bubble” and limit their exposure to new and different ideas.

If you were searching for t-shirts for a while, and finally, you purchased one, you would rather not be bombarded with only t-shirt recommendations all the time. I would rather see a popup window with a question like “Are you still interested in t-shirts?” and two buttons, “Yes, show me more t-shirts” and “No, I don't need t-shirts”.

Note. Some of these criteria might not be relevant for every website or app, so choose the ones that are significant for your specific audience. For example, it's significant for us to know the gender of the user on a fashion and beauty website, but it might not be as significant for a gaming or tech website.

If you have a Pinterest account, you've probably seen the question box they provide when they are not sure if you want to continue with viewing certain types of images or not. I like this kind of behavior, which not many platforms do.

Recommendation Pipeline System

The Recommendation Pipeline System (RPS) is a framework that allows us to gradually collect recommendation candidates based on the criteria that we provide. This system consists of several terms, here is a brief description of each of them so that you have a reference:

• Data stream in the context of RPS is the object that flows through the filters of the Recommendation Pipeline System to carry information about the state of the system. The purpose of the Data stream is to connect the system with filters.
• A filter in the RPS refers to a handler that takes the Data stream as input, processes it according to a specific set of rules, and returns a new version of the Data stream, or returns the old one.
• An item is the single content item like a post, article, product, video, music track, etc.
• An excluded item refers to the item that we don't want to have in the final recommendations. The excluded item is an identifier (ID) of the content item and not the actual item itself.
• A fetched item is the item that has been taken from a storage system, such as a database, to be prepared for the recommendation. It's basically a data record retrieved from a database.

So, how can you make the most of these criteria and create effective content recommendations? Using the RPS framework, we'll be able to gradually narrow down potential recommendations based on the different criteria that we've identified.

Tip. RPS is a good analogy to a water filtering system, where the water (in this case, data) flows through a series of filters to remove impurities. Similarly, the RPS framework filters through data to gradually narrow down recommendations and ensure that only relevant content is suggested to users.

To make things easier, let's take a look at a couple of schemes that I've prepared for you to help visualize the RPS. The first one is just a Legend for the terms we'll be using, where you can see which drawing represents which term.

The second scheme shows the data flow starting from the system through the filters and ending with the final step:

Don't be overwhelmed by the amount of information presented here. Once you understand the flow, you'll be surprised at how easy it is to implement it. Here is the same scheme for news platform or blog websites as an example down below. For this example, let's imagine that we are viewing a blog post and at the bottom of the post we want to see 5 recommended articles that are related to the current one.

Note. It's important to store items as objects and not as IDs like we do with excluded items because items will be the result of our pipeline system. That's why they need to store all the fields required to display them on the client.

This is how it looks with code:

const result = new RecommendationSystem(5)
    .pipe(new LikedPostsFilter())
    .pipe(new TagsRelatedFilter(this.post))
    .get()

1. We start by defining a Data stream with default values. The Data stream is what will be used for collecting potential recommendations through all the pipes that follow.
2. Next, we pass our Data stream through the “Liked posts filter”. This filter collects all the posts that the user has liked and returns the Data stream with those posts as potential recommendations.
3. Then, we pass our updated Data stream to the second filter, which is the “Tags related filter”. This filter looks for posts that have similar tags to the one we are currently looking at.
4. The final step in this pipeline is making sure we have enough posts to display. If there are still not enough posts to recommend, we just grab random posts, except those that we already have.

If you still find the concept of RPS overwhelming, just remember that it's all about gradually collecting potential recommendations based on specific filter. A filter is just an abstraction that receives the Data flow and modifies it. In the next article, we'll focus on the Recommendation Pipeline System implementation and explore how you can customize it to suit your needs.

Advantages

The biggest advantage of using such a system is that it's very fast and highly customizable to your specific needs. As you will see in the next articles of this series, other algorithms are much slower and require you to move the calculation logic outside the client.

Users will be frustrated waiting until the page finishes loading and recommendations are displayed. Of course, we can still overcome those issues using various techniques, but the Recommendation Pipeline System solves those problems in a more efficient way.

Plus, it's highly customizable, meaning that you can tweak the filters and settings to make sure that recommendations are very relevant to your audience. You can add as many filters as you wish, depending on the specific needs and preferences of your users.

Disadvantages

One potential disadvantage of the Recommendation Pipeline System is that it mostly applicable to finding similar content based on the currently viewed product, post, video, or whatever content the user is interacting with.

RPS is not recommending content what user might be interested in, but rather content that is similar to what user is currently viewing. If you are looking for a system that focuses more on predicting user behavior and recommending content accordingly, other systems that we will be covering in this series will be more appropriate.

Conclusion

Proper content recommendations can make a big difference in user engagement and satisfaction. There are different types of recommendations that you can use based on your website or app's requirements.

There are different types of recommendation, including Collaborative filtering and Content-based filtering, each with its strengths and weaknesses. Nowadays, the most popular recommendation systems are based on machine learning algorithms, which can analyze user behavior and preferences to offer highly personalized recommendations. This is the most efficient way to offer content recommendations that are relevant and engaging to your audience.

While it's being popular, it's not the most used approach because with machine learning, you need a lot of data to train the algorithm accurately, and it requires constant updates. Machine learning is a complex field, to build a powerful recommendation system, you need to have the expertise and resources necessary to handle large amounts of data and complex algorithms, even with libraries like TensorFlow. I will definitely write an article for this series if I see that this topic is interesting to the audience.

Note. There are no perfect recommendation techniques, even the most advanced machine learning algorithms have their limitations. Companies like Google, Apple, ByteDance, and Facebook put lots of effort and money into improving their recommendations and predicting what a user might like, but they still have their flaws and sometimes recommend content that is irrelevant for the user. It happens to me all the time, and I'm sure you also get recommended content that you're not interested in.

More details for the Recommendation Pipeline System we will be covering in the next article. I'll publish it as soon as possible, so stay tuned and don't miss out on learning how to implement this efficient system for your own content recommendations.

Make a change in the world and consider living a comment down below or simply putting a like on this article if you found it useful. Yeah, you need to have an account to do that, but it's a small effort to support creators and share your feedback. You can simply sign in with a GitHub or Google account with 1 button click. See you soon.

Keywords: improve, fullstack, udemy, medium, machine, learning, rps, system, suggest, suggestions, schmidt