Harnessing Machine Learning to Predict Success in Film and Television

While the film and television industries are known for their creativity, they remain inherently risk-averse. With rising production costs and a fragmented production landscape, independent companies struggle to absorb substantial losses.

In recent years, there’s been a growing interest in utilizing machine learning (ML) to identify trends and patterns in audience reactions to new projects in these industries.

The primary data sources for this analysis are the Nielsen system, which, despite its roots in TV and advertising, offers valuable scale, and sample-based methods like focus groups that provide curated demographics, albeit at a reduced scale. Scorecard feedback from free movie previews also falls under this category, though substantial budget allocation has already occurred by that point.

Exploring the ‘Big Hit’ Theories

ML systems initially relied on traditional analysis techniques such as linear regression, K-Nearest Neighbors, and Decision Trees. For example, a 2019 initiative from the University of Central Florida sought to forecast successful TV shows based on combinations of actors, writers, and other key factors.

Meanwhile, existing models in recommender systems often analyze projects already deemed successful. This begs the question: how do we establish valid predictions for new films or series when public taste and data sources are in flux?

This challenge relates to the cold start problem, where recommendation systems must operate without prior interaction data, complicating predictions based on user behavior.

Comcast’s Innovative Approach

A recent study by Comcast Technology AI, in collaboration with George Washington University, tackles this cold start issue by employing a language model that uses structured metadata from unreleased movies.

This metadata includes key elements such as cast, genre, synopsis, content rating, mood, and awards, which generate a ranked list of likely future hits, allowing for early assessments of audience interest.

The study, titled Predicting Movie Hits Before They Happen with LLMs, highlights how leveraging such metadata allows LLMs to greatly enhance prediction accuracy, moving the industry away from a dependence on post-release metrics.

By making early predictions, editorial teams can better allocate attention to new titles, diversifying exposure beyond just well-known projects.

Methodology and Data Insights

The authors detail a four-stage workflow for their study, which includes creating a dataset from unreleased movie metadata, establishing a baseline for comparison, evaluating various LLMs, and optimizing output through prompt engineering techniques using Meta’s Llama models.

Due to a lack of public datasets aligning with their hypothesis, they constructed a benchmark dataset from Comcast’s entertainment platform, focusing on how new movie releases became popular as defined by user interactions.

Labels were affixed based on time taken for a film to achieve popularity, and LLMs were prompted with various metadata to predict future success.

Testing and Evaluation of Results

The experimentation proceeded in two main stages: first, establishing a baseline performance level, and then comparing LLM outputs to a more refined baseline that accurately predicts popularity based on earlier data.

Advantages of Controlled Ignorance

Crucially, the researchers ensured that their LLMs operated on data gathered before actual movie releases, eliminating biases introduced from audience responses. This allowed predictions to be purely based on metadata.

Baseline and LLM Performance Assessment

The authors established baselines through semantic evaluations involving models like BERT V4 and Linq-Embed-Mistral. These models generated embeddings for candidate films, predicting popularity based on their similarity to top titles.

The study revealed that BERT V4 and Linq-Embed-Mistral excelled at identifying popular titles. As a result, BERT served as the primary baseline for LLM comparisons.

Final Thoughts on LLM Application in Entertainment

Deploying LLMs within predictive frameworks represents a promising shift for the film and television industry. Despite challenges such as rapidly changing viewer preferences and the variability of delivery methods today compared to historical norms, these models could illuminate the potential successes of new titles.

As the industry evolves, leveraging LLMs thoughtfully could help bolster recommendation systems during cold-start phases, paving the way for innovative predictive methods and ultimately reshaping how content is assessed and marketed.

First published Tuesday, May 6, 2025