Introduction and Background

The movie industry is challenging to navigate, and predicting the success of one’s film can be the difference between making it big or losing millions of dollars. Traditionally, measurements like “star power” and marketing campaigns were used, but they often weren’t reliable. With the rise of digital platforms like YouTube, new data sources have emerged, offering deeper insights into audience engagement and success metrics for movies. This project will explore machine learning and its applications to predict movie success using traditional metadata and transmedial data.

Literature Review

Research has been done to explore different aspects of predicting the success of a movie. Some researchers used sentiment analysis and data mining techniques to predict box office revenue based on the cast, budget, and genre [1]. Other studies have investigated the impact of social media on movie popularity [2]. Recent work shows how transmedial data such as YouTube trailer views can enhance the accuracy of predictions [3]. There is still a gap in research that comprehensively combines both traditional metadata with transmedial data for recent films.

Dataset Description and Link

This project uses the “The Movies Dataset” from Kaggle [4], which contains movie metadata including genre, budget, revenue, and release date. We will seek to incorporate features like YouTube trailer view counts and social media sentiment analysis, time permitting. We will concentrate on movies released up until July 2017 due to the recency of the dataset.

Access the dataset here.

Problem Definition

Studios and distributors require accurate predictions to guide decisions on marketing, distribution, and investment. Current methods often lack the precision needed in today’s evolving media landscape. This project will address this issue using machine learning techniques.

Motivation

Accurate movie predictions offer significant benefits. Studios can optimize budgets, distributors can refine release schedules, and investors can assess project viability. Additionally, understanding what drives success offers valuable insights into audience trends and preferences.

Methods

Data Preprocessing Methods

Data Cleaning

Data cleaning was the core of our preprocessing operations because it involved identifying and removing any missing duplicate or irrelevant data. This was crucial because the Kaggle dataset that we used initially had 8 features, many of which would not help in predicting the target: revenue. Features like posters, initially included in the Kaggle dataset, were of no use to us because just including posters would be too vague. If “poster types” (minimalist, floating heads etc.) was a feature, that could have been useful as trends could have been identified but simply including images of posters would do nothing. So, our first step was to set a completed list of features. We decided on using “budget”, “revenue”, “vote_average”, “popularity”, “genre”, and “production_companies”. By doing so, we were able to use the original dataset while ignoring the irrelevant features. We then also ensured that all numerical features were of type float. This was done using the .astype(float) function. This ensured that any important information was not lost and that comparisons and arithmetic between features could be conducted smoothly. Lastly, all placeholder values were removed from the input data. The former was done through the following code:
df = df[(df['budget'] > 0) & (df['revenue'] > 0)]
This ensured that only data points with a recorded budget and revenue were considered as inputs.
Feature Engineering

Feature engineering helped enhance our model’s performance by transforming existing features. The primary way this was done was by transforming raw data into something that is suitable to the model. One way this was done was by transforming all the data points that had a value of 0 associated with the “popularity” feature. These data points had their popularity replaced with the mean popularity of all data points. This was implemented using the following code:
df['popularity']=df.mask(df['popularity']==0, df[df['popularity'] > 0]['popularity'].mean())['popularity']
This was done because 0 was not a representative value of the data as a whole and too many 0’s would skew the data. Such data points were not removed because all their other features contained valid values which indicated that these points could be used as inputs but without being classified as outliers.
(In code, SciKit Transformers were used after One-Hot Encoding).
One-Hot Encoding

We used both one-hot and ordinal encoding throughout the course of our model. This was because for some features an ordering had to be implicitly defined (ordinal encoding) and for others they just had to be represented as unique discrete values (one-hot encoding). For example, one of our features was “genre” and so categories like horror, comedy, action etc. do not have an implicit order defined between them. Thus, we used one-hot encoding for genres. This was done through the following commands:

genre_names = [[genre["name"] for genre in json.loads(str(genre_list).replace("'", "\"")) if "name" in genre_list] for genre_list in genres_data] mlb_genre = MultiLabelBinarizer() one_hot_encoded_genres = mlb_genre.fit_transform(genre_names) # replace genres in dataset with one-hot encoded df= df.drop('genres',axis=1).join(pd.DataFrame(one_hot_encoded_genres)).fillna(0)

The first line of code traverses the “genre” column to individually capture each genre that is defined within the dataset. Then, we create an instance of a MultiLabelBinarizer object which mimics the principle of One-Hot Encoding while allowing a movie to be classified under multiple genres. The last two lines convert each data point’s genres into an array which contains binary values. Data points which are not classified under any genre have their genre represented by a 0, following the principle of data cleaning user previously.

Machine Learning Algorithms

Linear Regression

The linear regression model implemented for this project was designed to predict movie revenue based on several key features, including budget, popularity, TMDb vote count, release year, and genre categories. The model utilized log-transformed versions of budget and popularity (log_budget and log_popularity) to address right-skewness present in the data and to improve linearity in the relationships. It analyzed continuous variables like budget and popularity as well as categorical genre variables, which was represented through One-Hot encoding. Visualizing this model's choice in regression coefficients revealed patterns in how these factors influence revenue predictions, as demonstrated in the following generated model:

If the following embed does not show an image, please click here.

Learning Methods

This project primarily plans to utilize supervised learning techniques to train our models with labeled data from the dataset. In this case, we have thus far implemented linear regression.

Results and Discussion

4.1: Visualizations

If the following embed does not show an image, please click here.

To better understand our model’s behavior, we visualized the actual versus predicted revenue with a scatter plot, which proved to be very informative. In an ideal model, the prediction would always equal the true value, so points would align tightly along the red diagonal line. With an R2 value of ~0.6, our scatter plot demonstrates a broadly linear correlation, indicating that, as the predicted success of a movie increased, so too did the actual revenue. We noticed deviations at the high end of this scale, as blockbuster movies with large actual revenues consistently outperformed our predictions, indicating an underestimation for those films. This suggests that our linear model did not fully capture the unique factors that gave top films their incredible success. Our predictions for films in the middle revenue range proved to be reasonably accurate, and we examined the distribution of residuals to confirm our findings. The residuals were roughly centered around zero but showed a slight right skew, since our model tended to severely underestimate revenue for high-grossing movies, leading to large positive errors where actual revenue exceeded predictions. Aside from the outliers, there was no strong bias across most of the range; residuals for mid-level movie performances were relatively symmetric in their distribution. Our visualization of the data aligns with our quantitative metrics. The linear model performs well for the bulk of movies, but it currently consistently underestimates top performers, and it overestimates a few of the lower performers. In our case, improvements are needed with capturing extreme cases of movie success, potentially with the addition of new features like YouTube trailer views.

4.2: Quantitative Metrics

The linear regression model yielded moderately accurate results in predicting movie success. For instance, on the held-out test set, the model achieved an R² value of ~0.60, meaning it explains about 60% of the variance in movie revenue/ROI. This indicates that the model is capturing a significant portion of the trend, though roughly 40% of variability remains unexplained. The Mean Squared Error (MSE) was approximately 1016 in squared dollar units for all revenue predictions, while the Mean Absolute Error (MAE) indicated that the average prediction deviated by tens of millions of dollars. An R² in the ~0.5–0.6 range is generally considered a moderate goodness-of-fit. The model is certainly better than a naive guess, but may not be precise enough for high-stakes decisions. The Explained Variance Score was approximately 0.60, closely reflecting the R² value, as there was no significant bias offset in the predictions. These metrics suggest that our linear model has some predictive power, but there is adequate room for improvement. In essence, our model reasonably estimates a movie’s success, but individual predictions can deviate significantly from actual outcomes.

The log-scale statistics obtained are as follows:

R² Score: 0.586
RMSE: 1.669
Mean Absolute Error: 1.136
Mean Average Precision Error: 0.099

4.3: Model Analysis (Linear Regression)

Our linear regression model’s moderate performance is explained by the inherent simplification caused by the nature of linear regression. Since it can only model straight-line relationships, non-linear patterns in the data resulted in an underfit. This was confirmed by our R² value of approximately 0.6, indicating high bias but no overfitting. This is logical, as real-world factors that influence the success of a movie are not purely linear. Social media buzz and virality can both have a huge impact, which violates the linear assumption we initially made. Likewise, interactions between features, such as a high marketing spend combined with a holiday release date, might yield an outsized impact, and this is not accounted for in a simple linear model. Our feature set was also quite limited, as important predictors like cast fame, director track record, or social media hype were absent; the model only drew insight from a slice of the whole picture. As a result, the linear regression worked reasonably well in areas where the relationship between inputs and output was roughly linear and covered by the given features, but it struggled with edge cases. In particular, movies that turned out to be runaway hits or massive flops were not predicted accurately. The model tended to under-predict the revenue of top blockbuster films and over-predict very low-performing films because those outcomes were driven by factors beyond the scope of our linear features. This aligns with the nature of our data – movie revenues have very high variance, where the majority of films earn modest amounts but a few earn hundreds of times more. A single linear trend line does not seem to be able to properly account for outliers, resulting in significant errors for those cases. The general direction of factors affecting revenue was captured, but nuances and complex combinations were not adequately captured, which led to an underfitting in terms of the model’s performance.

If the following embed does not show an image, please click here.

The above model captures the features of the linear regression ranked by their contribution to the prediction of revenue. All coefficients are shown as absolute values. The bar graph suggests that financial investment (budget) and audience appeal (popularity) are the primary drivers of movie revenue, followed by genre and release year as secondary factors.

4.4: Next Steps

Our results, while moderately successful, demonstrated the potential for several avenues of improvement in the future. For instance, we can experiment with more advanced and non-linear modeling techniques. Prior research indicates that decision tree ensembles may significantly outperform linear models, leading us to predict a boost in accuracy with the adoption of such methods in terms of handling the high variance in revenues and the fitting of outliers more accurately. We could also potentially explore neural networks for completeness. We also plan to enrich our original feature set, with the incorporation of features like YouTube trailer views, likes, and potentially critic ratings. These features may even improve upon our original linear model, leading to a greater R2 value. We may also incorporate temporal features, such as the movie’s release date and the time between the first announcement and the actual premiere. Lastly, we will apply rigorous data preprocessing and feature engineering methods to ensure variables are on comparable scales. In our implementation of these next steps, we aim to further increase the predictive accuracy of our model to the level of highly practical predictions.

References

Quader, N., et al. “A Machine Learning Approach to Predict Movie Box-Office Success.” IEEE Xplore, 1 Dec. 2017, ieeexplore.ieee.org/document/8281839 .
him, Steve, and Mohammad Pourhomayoun. Predicting Movie Market Revenue Using Social Media Data. 1 Aug. 2017, pp. 478–484, ieeexplore.ieee.org/abstract/document/8102973 , https://doi.org/10.1109/iri.2017.68 . Accessed 21 Feb. 2025.
Ahmad, Ibrahim Said, et al. “Movie Revenue Prediction Based on Purchase Intention Mining Using YouTube Trailer Reviews.” Information Processing & Management, vol. 57, no. 5, 1 Sept. 2020, p. 102278, www.sciencedirect.com/science/article/abs/pii/S0306457319309501 , https://doi.org/10.1016/j.ipm.2020.102278
Rounak Banik. The Movies Dataset. Kaggle. https://www.kaggle.com/datasets/rounakbanik/the-movies-dataset

Proposal Contributions

Dylan Bruce	Timotheus James	Vikrant Talwar	Alexander Thorne	Tyler Morgan
Created website midterm report page, uploaded dataset, created/coded Linear Regression Model, calculated model statistics, created visualizations w/ matplotlib, explained linear regression & visualizations in midterm report	Linear Regression model analysis, next steps, reference collection, results/discussion	Described and analyzed 3 data preprocessing methods along with a supervised learning method	Proofreading, dataset research, initial visualizations, quantitative metrics	Added model statistic, created Movie_ML_Notebook with ML algorithm

Gantt Chart

Click to access project Gantt Chart

Navigation

Midterm Checkpoint