BLOGS

Decoding ‘Impulsion of Movie's Content-Based Factors in Multi-Modal Movie Recommendation System’

By IIT Patna
30th November 2023
Brijraj Singh summarises the paper titled ‘ LLM Based Generation of Item-Description for Recommendation System’ that was accepted at the RECSYS-23 conference in Singapore, this September.
In this blog, Brijraj Singh summarises the paper titled ‘Impulsion of Movie’s Content-Based Factors in Multi-Modal Movie Recommendation System’ co-authored by Prabir Mondal, Pulkit Kapoor, Siddharth Singh, Prof. Sriparna Saha (IIT Patna) and Naoyuki Onoe which was accepted at the International Conference on Neural Information Processing (ICONIP) in Changsha, China from 20th-23rd November 2023.

This blog, summarises the paper titled ‘Impulsion of Movie’s Content-Based Factors in Multi-Modal Movie Recommendation System’ co-authored by Prabir Mondal (IIT Patna), Pulkit Kapoor (IIT Patna), Siddharth Singh (IIT Patna), Prof. Sriparna Saha (IIT Patna) and Naoyuki Onoe which was accepted at the International Conference on Neural Information Processing (ICONIP) in Changsha, China from 20th-23rd November 2023.

Introduction

This research paper delves into the realm of recommendation systems, particularly focusing on movie recommendations, a pivotal component of modern streaming platforms with extensive film libraries. The paper highlights a significant limitation in existing approaches, which treat user inputs as uniform, despite the fact that individual users perceive movies differently, influenced by factors such as genre, story, director, and cast. To address this, the authors introduce two novel metrics: TextLike_score (TL_score) and GenreLike_score (GL_score). These scores play a critical role in their Cross-Attention-based Model, which outperforms the current state-of-the-art recommendation systems by considering these nuanced user preferences.
The research is supported by evaluations conducted on two diverse datasets: MovieLens-100K (ML-100K) and MFVCD-7K. Notably, the authors leverage multi-modal data, including audio, video, and textual information, to calculate the introduced scores. Their experimental results affirm that their Cross-Attention-based multi-modal recommendation system, incorporating the Meta_score, effectively addresses user preferences, making it a compelling solution for real-time movie recommendations.
The importance of understanding user preferences in the digital age, particularly for platforms like streaming services, is emphasised. With the proliferation of personal viewing devices and the rise of OTT platforms, the demand for tailored movie recommendations is ever-increasing. Traditional recommendation systems have focused on predicting user-movie ratings based on embeddings derived from text, audio, or video data, ignoring the intricate nuances of user preferences for genres, directors, cast, and storylines.

Findings

To address the limitations mentioned above, the authors propose the introduction of TextLike_score (TL_score) and GenreLike_score (GL_score) as parameters to quantify textual content and genre preferences. Unlike conventional methods, their model applies a Meta_score to user-movie embeddings, ensuring a more accurate representation of user preferences.
The authors’ innovative approach involves a Cross-Attention-based rating prediction model that considers audio and video embeddings of movies, combined with user embeddings. A self-attention-based fusion technique, complemented by multi-head cross-attention, facilitates the merging of these different modalities. Their model’s superiority is demonstrated through empirical evaluation on the ML-100K and MFVCD-7K datasets, reaffirming its effectiveness in real-time scenarios.
The research paper concludes by hinting at future extensions, including multitask settings such as movie genre prediction and user age/gender prediction, alongside user-movie rating prediction, highlighting the potential for even more refined and personalised recommendation systems.

To know more about Sony Research India’s Research Publications, visit the ‘Publications’ section on our ‘Open Innovation’s page:

Open Innovation with Sony R&D – Sony Research India

Skip to content