Quantifying Rumor Mongering in the Baseball Media Ecosystem

In what feels like interminable scrolling of the internet this offseason waiting for something to finally happen, it occurred to me to ask, does any of this rumor-mongering actually tell us anything? It is certainly strange that we as consumers of baseball, a modified game of tag with hitting and throwing a ball, care so much about the internal machinations of billion-dollar organizations and the personal decision-making calculus of people we will never meet. Regardless of this peculiarity, I myself still spend hours a week wondering if George Springer would be willing to play for a team who doesn’t have a guaranteed home stadium for the foreseeable future and subsequently will be located in a foreign country in Canada.

This interest is what feeds the North American baseball media ecosystem and employs thousands of people, from reporters to web designers, social media managers to news aggregators, and many more. I wouldn’t necessarily argue that this content holds no value if it is biased or inaccurate, because the time we spend consuming this offseason content really just satiates our longing for baseball when we can’t watch our favorite teams live. But the question remains, does this content hold any predictive value, or are we just fooling ourselves?

This article is based on data scraped from MLB Trade Rumors, the leading aggregator of rumors around baseball, on December 9, 2020. I pulled the last 2,000 posts that each team was tagged in and analyzed what information we’re actually getting from reading and discussing the rumors and reports inside the baseball media ecosystem. To begin, we can observe the volume of rumors for teams by seeing how many days one would have to go back to reach a cumulative 2,000 posts.

A few teams really stand out of from the pack, with the Mets, Yankees, and Red Sox on the high end and the Rockies on the low end. The Mets rumors only go back as far as December 23, 2015, so the remainder of the analysis focuses on the common time period held by all teams beginning on Opening Day 2016. Correlation over the time period of April 2016 to March 2020 between number of rumors and number of transactions is approximately .35. That means there is certainly some relationship there, but a larger factor appears to be franchise value (.67), which in turn correlates with fan interest. The volume of rumors around a team has much more to say about how many people care about the team and thus report on them than it does about the volume of transactions a team actually makes.

Below is the table which shows the date required to go back to for each team to reach 2,000 posts. It’s even more arresting to see the difference between the top and bottom is in excess of three and a half years.

It is also interesting to observe when these posts are being written for each team. Note again that the next table begins in April 2016, so we see which teams have more written about them in the offseason vs. in-season.

Interestingly, there is not a team which has a majority of posts written in-season despite the fact that it is longer in time than the offseason. I think that says a lot about this industry and why it is so popular. It exists because some number of fans want more content about their favorite team, and the desire only intensifies when there is no new baseball to consume. Looking at the list, there is an interesting mix of teams at the extremes. Colorado not only has by far the fewest posts, but they also have the highest percentage of their posts in-season — in fact they don’t even have the least posts in-season. The biggest difference for the Colorado rumors is that they have a shockingly low number of posts in the offseason, only 80% of the next-closest team. I don’t know much about the baseball consumption or culture of Colorado Rockies fans, but I’d appreciate anyone who could shed more light on why this big outlier exists.

It’s worth noting that so much of Oakland and the Los Angeles Angels’ rumors occur in the offseason. The latter doesn’t surprise me as much me since so much of the past five years in Anaheim have been about trying to put a contending team around the best player in the game, Mike Trout, and also signing Shohei Ohtani. Oakland is a little bit more surprising. If one would like to ascribe a reason for that, it must be one that doesn’t apply to the Tampa Bay Rays, a club below the median in the same statistic and whom I consider to have a similar conversation about them in terms of team-building.

Lastly on the descriptive front for team analysis, let’s examine which clubs most frequently occurred in the same rumors together to get a sense of which teams the baseball media ecosystem seem to revolve around. A quick note about the structure of MLB Trade Rumors: often times they do multi-team posts by division, which doesn’t necessarily mean the teams in that post are actually talking to each other or about the same player.

Unsurprisingly, right at the heart of the diagram making the most strong connections is the Yankees and the Red Sox. They are both tied closely with Baltimore, a division mate, but they also have strong connections with the Giants and the Phillies, which we will see later has a lot to do with one Manny Machado. A small-market team at the center of the diagram is the Miami Marlins, which is likely because of their penchant for trading away star players like J.T. Realmuto, Marcell Ozuna, Giancarlo Stanton, and Christian Yelich. Linger on that sentence for a moment longer because all those baseball players are so good.

Regardless, there are many more interesting links which appear, too numerous to name, but I’d also like to highlight the connection between San Diego and Boston, who made deals around Craig Kimbrel and Drew Pomeranz, as well as both being linked to Machado between 2016-18.

I mentioned Machado twice already because I believe he was so important in explaining the map of team connections on MLBTR. Below is table of the top 40 mentions of players specific to a team. The only player to appear more than twice on this list is Machado. To be fair, he also played for three different teams in this time period, but to be even more fair, two of the teams he played for are not even on the list. He does appear again with the White Sox (another team he’s never played for), the Dodgers, and the Padres with 427, 343, and 330 mentions, respectively.

A potential note of confusion: the Cardinals’ Martínez likely refers to both Carlos and José, which artificially boosts that number, and the Nationals’ Rizzo likely refers predominately to general manager Mike Rizzo, not first baseman Anthony Rizzo.

My last observation is around the high placement of different Mets in the table. Most of this list comprised of high-profile players who were in talks regarding changing team either by trade, free agency, or both. Céspedes signed his deal before this time period began, Syndergaard has never been traded or hit free agency, and Jay Bruce has produced 2.7 bWAR — and only 1.6 of that in two stints with the Mets. A lot is made over the tumult within the Mets organization, and perhaps that is fair, but if a media ecosystem around a team can talk about Jay Bruce more than any other team media ecosystem can talk about all but six opposing players across the league in the past five years, there is clearly a hyper focus and scrutiny on this team which others do not face.

With this descriptive analysis, we can see a lot of the trends in the coverage around baseball, where the focus is, who it’s on, and when it is most voluminous. However, the question remains: how much predictive value does this all have? To dig deeper, I checked the number of times a word appears per post about a team, and whether it was written in the offseason, to try to see what it said about the quality of the club. Using this data between the 2016 regular season to the 2018 offseason, I built a multiple linear regression model using keywords which were most relevant in determining a team’s win total. On the positive side was “luxury tax” and “game,” and on the negative side was “organization,” “seasons,” “trade,” and “surgery.”

There’s one other big one, and that is “payroll.” It matters whether or not this word is appearing during the season or in the offseason, because it turns out there’s a more even spread across teams talking about payroll in the offseason and that actually is not a bad indicator for a team. However, if a team is talking about payroll in-season, that is the single worst indicator of a team’s poor performance. This model isn’t particularly fancy and actually knows nothing about how good a single player is, and yet it is still able to explain almost 40% of the variation in wins between teams. Here is a chart outlining the expected number of wins in 2019 by each team according to the model against their actual total.

There’s definitely a bias to overestimate a team’s win total, as it predicts 2,615 wins across the league despite the fact there are only 2,430 games played in a regular season. However, the general order which the teams are placed accurately reflected where they ended up finishing, save for some big misses on the Cubs and Red Sox. This means the coverage which is being aggregated on MLB Trade Rumors certainly is reflective of how a team performs on the field.

It’s also interesting to note the reoccurring words which are pointing us in one direction or the other. There’s a common theme on both sides, as teams who do well are talking about luxury tax (often times because they are spending into it or contemplating the matter), while teams who are struggling tend to talk more about money and payroll, which is often about cutting it rather than increasing it. While this gives a sense of the narrative around individual clubs and how it connects to the team’s performance, it does not tell us about the validity of the rumors we are seeing.

I looked at the offseasons from 2016 to 2020 and compared the number of posts league-wide in a month and the number of signings in that month and returned a correlation of .4, which isn’t nothing. Here are charts of the volume of signings and rumors each of the past five offseasons over time.

In 2016 and to a lesser degree 2017, rumors and signings actually line up really well with a discordance in March, which is likely attributable to season preview pieces in spring training after most of the signings occurred. However, the past three offseasons have been much more level across the months in terms of signings, but the rumors were still coming at a pace more like the previous years in 2018 and 2019. It appears that in 2020, the rumor mill began to resemble the actual status of the signing market much more in terms of frequency of the months.

This is certainly a phenomenon many people have written about in the past five years, but I can’t help but wonder if a uniform distribution across time for free agents signings is actually exactly what we should expect of this market. I am not an economist, but perhaps the forces which currently exist on both sides of the ledger for coming to deal and holding out longer make any given date in the offseason just as likely as any other to be the date which an agreement is made. It may also be a fact that the trend of players signing later and later in the offseason will keep moving in that direction, but I’d be weary of our bias toward overestimating trends and underestimating regression to the mean.

The final question is, can the information in these posts be used to predict which team a player will sign with before he does? Again, using the words in posts about a player before he signs as well as the frequency a player is linked to a specific team and whether or not it is the most linked team, one can create a powerful logistic regression model. Notably, the higher proportion of rumors which specifically link a team and player, especially if the team is the one most frequently tied to that player, the higher chance of a signing.

The words which were most relevant in predicting a player signing with a team were on the positive side were “comments,” “deal,” “sign,” and “report,” while the negative words were “teams,” “pick,” “baseball,” “DH,” and by far the most impactful single word… “owner.” Notably, on the positive side, when a post actually uses words like deal and sign, the pairing is more likely to come together. However, on the negative side, the talk of an owner being involved dramatically decreases the odds of a team signing a specific player, 2.2 times more impactful on the outcome of the signing than any other word, positive or negative.

To get a sense of how good this model actually is, it was used to predict the 2020 offseason (on which it was not trained) and correctly predicted the status of the signing 86% of the time. However, consider the fact that on average, a player was mentioned in an article with 9.12 unique teams that offseason, and they can sign with only one of them. This means that if the model were to just predict that a team would not sign a player they have been linked to every time, it would be 89% accurate. So more relevantly, the percentage of non-signings the model found (specificity) was 89%, and the percentage of signings it found (sensitivity) was 66%. However, the model only was correct in its prediction of a signing (precision) 48% of the time.

The model wasn’t particularly reliable when it supposed a signing, but it was good at balancing the downside risk of misattributing a signing in order to find a higher percentage of them. This implies our independent variables are indeed providing relevant information in terms of signing. Included in the correctly predicted signing was Gerrit Cole to the Yankees, Jason Kipnis to the Cubs, and Edwin Encarnación to the White Sox.

To conclude, this analysis turned up some interesting facts and trends in the coverage of specific teams and how they interact with other teams. It also shows that the words being used often are reflective of the way a team is performing on the field, and in the offseason, can often be used to fairly accurately predict whether or not a team and a player reach an agreement in free agency. The code used to pull the data from MLB Trade Rumors, pull the transaction data from Baseball Reference and analyze it is available on my GitHub here. I’d encourage anyone to use this data and see if they can mine more from this resource, perhaps with more advanced Natural Language Processing (NLP) techniques such as topic modelling or Word2Vec prediction models.