Big Data Scientists: Netflix is not maximizing its recommendation systems, are you?

As a Big Data Scientist, it is imperative today to develop recommendation systems which turn insights about customers and prospects into relevant recommendations and quality leads. As a graduate student in Northwestern University's Medill Integrated Marketing Communications program, I have been researching big data issues and have found two articles which address this challenge - using Netflix as an example.

In the article "Putting Big Data in Context", Scott Gnau(@Scott_Gnau), president of Teradata labs pointed out that best decisions and recommendations are made with a combination of data analytics and human intuition. Two mistakes companies tend to make when dealing with big data are: 1. over-reliance on what data tells us; 2. become too enamored with certain types of new data, or look at data in silos. For example, Netflix's recommendation system relies too much on past behavior data while failing to incorporate context data such as real-time emotion and situation individuals are experiencing.

source: Who knew? Big Data is a Social Beast!

Nestor Bally(@NKBailly)  mentioned the restrictions of current recommendation system in "Build A Better Algorithm(With A Little Help From Your Friends)". He said rudimental recommendation algorithms made largely ineffective predictions based on similarities in items and the premise that people with similar historical preferences are likely to share future preferences. However, this led to a low degree of diverse content and a self-amplifying vicious cycle. For example, Netflix does not recommend films based on film fanatics' watching list or what friends are watching, and thus restricts the spectrum of the recommended content. To build a better algorithm, companies should leverage the power of social influence. 

Based on my analysis of these two articles and my classes on social marketing and big data analytics in the Northwestern Medill IMC program, there are three actions you need to consider when developing your social and contextual recommendation system.  They are:

1. Think in context. Consuming content online is an emotional behavior as well as a rational one. You need to get data about people's current emotions, like what Spotify does, and incorporate the contextual data into the recommendation algorithm.  

2. Leverage social. You can provide more personalized recommendations without entering the self-amplifying vicious cycle by integrating the power of social influencers such as experts and friends.

3. Data is not the problem. As a big data scientist, you should not let what data you have restrict what question you can answer. It is critical for your team to think out of the box and disruptively innovate your recommendation system by integrating more varieties of data. 

A better recommendation algorithm can become the competitive advantages of any companies in the long run. HBO just announced that it will join the digital distribution space along with Hulu, Amazon and Apple, Netflix's future will largely depend on whether they can disruptively innovate the recommendation algorithms to provide a more customized and social network for digital content consumers. As a big data scientist, your company may face similar challenges as Netflix. It's time to reinvent your recommendation algorithms and stand out in the big data competition.

Joyce Liu, a marketing wiz and film fan, is pursuing a Master degree of Integrated Marketing Communications at Medill, IMC. I specialize in digital marketing and social analytics. I have helped financial and CPG companies transform their marketing practice in the digital space. Connect with me on Twitter(@Joyce_xinranLiu) and LinkedIn.