Does Payroll Matter? (Part I)

Money in baseball has been an infinite source of criticism. In MLB, there is no salary cap as in other major sports, and luxury tax is relatively recent. Media has made us believe that the small fish (e.g. small-market teams) will always be eaten by the big one (e.g. big-market teams). The Kansas City Royals’ performance during the last couple of years, along with the tricky and often misunderstood Moneyball concept, has brought back salary to the newspaper headlines even though it is safe to say the Royals were not even a low-end payroll team. In any case, this post is an attempt to see if popular beliefs regarding money, power and on-field performance pass the numerical test.

There are many interesting questions related to this topic. However I will limit myself to the following during two posts:

1. Is there a relationship between payroll and wins? If so, how strong is it?
2. Has this relationship changed over time? If so, where are the peaks? Where are we now?
3. Will money buy you a ring or a post-season ticket? If so, how much should we spend?
4. Are there truly big spenders? If so, who are they? Have they changed over the years?

Let me start off by stating what my data sources are, and laying out my assumptions so that we are in the same page. My sources for salaries are Baseball Chronology (1976-2006), Sean Lahman database (2007-2014) and Sportrac (2015). For wins and post-season appearances, my references are MLB and the Sean Lahman database. MLB revenue data is from Forbes.

My assumptions and caveats are the following:

1. Payroll values are not adjusted for inflation. Time value of money has not been taken into account.
2. The Houston Astros are considered an American League (AL) team. The Milwaukee Brewers are considered to be a National League team.
3. 1994 strike-shortened season does not have playoff teams or a World Series champion.
4. Payroll is considered to be Opening Day payroll. Payroll is assumed to be constant throughout the season for simplicity. Arguably this may not hold true as winning/better teams will likely be buyers at the trade deadline. Losing teams will likely be sellers.
5. I have not tested for any confounding effect on the variables studied (payroll and wins).

Without further talk, I will get to it.

Question 1: Is there a relationship between payroll and wins? If so, how strong is it?

To answer this question, I found the correlation between yearly payroll and winning percentage for every individual season played from 1976 to 2015. Because payroll values have changed so much in 40 years, I used z-scores or standard scores, which allows us to compare different seasons, regardless of payroll differences.  A payroll number on its own does not mean much and should be compared to the pool of teams on a yearly basis i.e. it is the distribution of payroll in the league that matters. Here’s a link in case you are not familiar with the concept of z-scores; please keep in mind that correlation does not imply causation. Check out the correlation here.

A couple of interesting insights can be drawn from this graph. The first one, quite obvious, is there’s a positive slope there, implying that more money affects wins positively. The second point, though, is that payroll alone does not wholly explain the total number of wins. We inherently knew that. In 40 years, we are able to find teams that satisfied each situation: low-payroll teams that were awful (Houston 2013), low-payroll teams that played over a .600 win percentage (Oakland 2001 and 2002), high-payroll teams that unperformed (Boston 2012) and high-payroll teams that exceeded expectations and went on to win 114 games (NYY 1998). There is a mid-tier team that did extremely well (SEA 2001). These are all outliers, though people can (will?) use every one of these cases to support a preconceived idea e.g. “baseball is a sport and it is attitude and effort that matters,” “money will buy you handshakes at the end of each game,” “big-money teams won’t win because they lack camaraderie,” etc. Therefore, let’s focus on the big picture.

The third point I’d like to highlight is the R-square. The R-square measures how successful the fit line is in explaining the variation of the overall data on a 0-to-1 spectrum. In this case R-square is 0.1905 so it looks like ~19% of the total variation in wins can be explained by the linear relationship between payroll and wins. Also, the slope of the best fit line is 0.0302. This means for a one-unit increment in Z-scores, there is a 0.0303 win-percentage increment. Remember z-score increments are not linear e.g. going from -0.5 to 1.5 requires a different amount than moving from 2 to 3.

However, the potential drivers behind the total number of wins are complex (injuries, roster construction, plain luck, etc.) and the R-square, along with the F-test and P-value, shows that money matters but seems to be overrated. Again, remember that correlation does not imply causation.

Question 2: Has this relationship changed over time? If so, where are the peaks? Where are we now?

We have established that team payroll can predict win percentage with a low confidence level. However, has that always been the case? Was money more important in the 80s than now? The following graph shows the R-square value for every two-year period from 1976 to 2015. It is important to keep in mind that the higher the R-square value, the stronger the relationship between payroll and winning percentage.Check out the R-square of payroll and winning percentage for every 2-year period.

The answer to our question of whether the relationship has changed over time is definitely yes. There are noticeable peaks and valleys. There have been two periods (which I highlighted in green) when money was a better predictor of winning percentage: from 1976 to 1979 and from 1996 to 1999. The first period corresponds to the first four years of free agency. Team owners flooded the league with new money as they went after key players e.g. Mike Schmidt or Reggie Jackson, and payroll increased drastically (60% in 1977, 34% in 1978), as shown below. These have been largely documented (here, here and here). Click here for the payroll growth trend since 1976.

The second period (1996 – 1999) is linked to the Yankees, Orioles (though they dramatically underperformed in 1998), Indians and Braves’ successful expenditure (read: lot of won games) and to the lack of Cinderella stories (perhaps only Houston in 1998 and Cincinnati in 1999). This period was also characterized by, firstly, a league expansion sequel: Tampa Bay and Arizona joined the league in 1998 and, understandably, underperformed. Secondly, MLB revenues year-to-year growth averaged 17% from 1996 to 1999 (not adjusted), so probably teams redirected that surplus to the salary pool. Lastly, in the late 90s, MLB was increasingly becoming a rich-team game. The graph below will show the payroll coefficient of variation for the 1976 – 2015 timeframe. This number, which I will call payroll spread, is simply the standard deviation divided by the mean. This number allows us to quickly assess how spread is the payroll across the league over time. Do you see the trend after ~1985? By 1999, this number had increased continuously for almost 15 years and MLB has had enough. As the power money increased AND the gap widened, MLB commissioned the Blue Ribbon Panel to come up with initiatives to level the field A.K.A. a revenue-sharing program to increase competition. Entertainingly, the correlation of money and winning percentage has decreased steadily but the payroll spread has remained pretty much consistent. I am hesitant to attribute the decline in R-square to the Blue Ribbon Panel or to other factors (read: is this coincidence?). Check out the payroll spread here.

If we go back to the yearly payroll and winning-percentage correlation graphs, you’d notice that I highlighted two periods in red too — from 1982 to 1993 and from 2012 until last season. Those were moments when the correlation of salary power and winning percentage was remarkably low. The first period seems to be closely related to the collusion MLB crisis (check out this link as well). The lowest point was in 1984-1987, when the correlation was only 0.03 and the salary spread was 0.22.

The 2012-onwards period has brought down R-square to a 20-year low (0.06 in 2012-2013). While TV revenue keeps rising, the baseball landscape has changed and new variables are in the mix. There is a redefined revenue-sharing model, we have analytically-inclined organizations, an extended wild-card system and international signings – all these factors have added more complexity to the winning equation, effectively diminishing the relationship between payroll and winning percentage – even with the salary spread still at ~0.40. We are living in interesting times in baseball indeed: If investing money in players doesn’t lead to better on-field results, where do teams need to invest e.g. analytics, managers or front office?

Note: This analysis is also featured in our emerging blog www.theimperfectgame.com