Leverage and Pitcher Quality Through the Eyes of Managers by Peter L'Oiseau October 2, 2020 Much criticism has been levied onto baseball managers and their inability to see past the archetypal dominant closer who closes pitches in save situations. Writers in the statistical community have observed and critiqued the many flaws which come with the save statistic and how it’s perceived by fans, managers, and baseball decision-makers as far back at least 2008 . Accumulating saves is a function of opportunity and degree of difficulty that is certainly not the best way to get at a relief pitcher’s ability to get outs. More objective methods such as ERA and its estimators, like Fielding Independent Pitching (FIP) and Skill-Interactive Earned Run Average (SIERA). are better ways to evaluate a pitcher’s talent, and Win Probability Added (WPA) is better for measuring a pitcher’s importance to winning specific games. This criticism has definitely been heard in the intervening years by people running ball which, can be shown by the number of pitchers who are getting saves on each team and the variance of save totals for a given team. A team with high variance in their save totals means that there is one player who accumulates a lot of saves and some number who have very few, opposed to lower variance representing a more even distribution of saves among pitchers. This variance metric is heavily negatively correlated (-0.74) with the number of pitchers a team has record a save in a given season. This means the more pitchers recording a save on a team, the more likely the distribution is to be equitable and the insistence on using your best pitcher in only a save situation is lower. Based on this analysis, somewhere between 2008 and 2011 was the peak on the capital “C” Closer in the majors. A rather precipitous drop occurred in 2016 and has continued on a downward trajectory to the point where last year saw the most equitable distribution of saves among teams since 1987, excluding the lockout-shortened 1994 campaign. That’s interesting, but it doesn’t tell us what managers are doing instead of strictly using their best relief pitcher in save situations. The analyst critique proposes using your best relief options in the highest leverage moments of the game regardless of inning. One problem with that is ordering of highest leverage to lowest leverage moments can only be done after all the decisions have been made. However, it is not completely unpredictable while in the game, and managers certainly can come closer to aligning pitcher quality to leverage than merely pitching their closers in the ninth. In order to analyze if managers are indeed following through on this alternative, we must first define leverage and quantify it. Fortunately, FanGraphs has done exactly this and offers the numeric leverage estimate for each plate appearance dating back to 1974. Leverage is defined in terms of win probability and its potential to fluctuate from the outcome of a given plate appearance. So, if you have a win probability estimate in State A given at least the inning, position of baserunners, number of outs, and score, you then consider all the possible outcomes of a plate appearance and calculate all those win probabilities if they were to happen. From there, you calculate the variance of the win probabilities associated with all possible outcomes, and that is your leverage. If there is a high variance in the win probabilities a plate appearance can generate, then the leverage is high. For example, if the bases are loaded down by one with two outs in the bottom of the ninth inning, a plate appearance could either win the game (100% win probability), lose the game (0% win probability), or tie (approximately 50% win probability), which would be very high variance as compared to plate appearance in a blowout where no matter what happens the game will still likely end in the same way. Using this definition of leverage, choosing the ninth inning of a close game as a proxy for high leverage isn’t bad. This is a plot of the distribution of leverage by inning where the innings are separated by thirds. We can see the first third of a game is distributed fairly symmetrically in terms of leverage, meaning it is not too high or too low most of the time. The middle innings are more uniformly distributed with more frequent low-leverage at-bats but also more frequent high-leverage at-bats. Finally, the remaining innings have an intense right skew, which means a large portion of the game happens in low leverage, but there is a spike in very high-leverage at bats, particularly in the ninth inning or in extras. It is not unreasonable to prima facie assume that the optimal time to use your best reliever would be in the ninth. But since we do have information on the leverage of each plate appearance in a game along with who is pitching, we can get insight into which managers are most successfully getting their best pitchers into the highest leverage moments and conversely their less talented relievers in lower leverage. In order to evaluate pitcher quality, I used a three-year weighted average of FIP and excluded position players pitching. Additionally, I made the FIP relative to the options available on the team, where a low score is an indication of high-quality pitching. I then investigated each plate appearance in the 2015-19 regular season and observed the correlation between pitcher quality and leverage by inning. Perhaps unsurprisingly, the first few innings show very little correlation between FIP and leverage, but it increases (in negative strength) as the game goes on and peaks in the ninth. However, what did take me aback was how weak the correlation was, peaking at -.25, which is not particularly strong. A little poking around actually leads us back to the first diagram and the distribution of leverage. Since there is a negative exponential distribution, most plate appearance are relatively low leverage in comparison. This requires good pitchers to pitch in lower leverage situations fairly often. Moreover, when a good pitcher pitches well, the leverage doesn’t get higher than it otherwise could. To analyze the true area of interest, I separated the leverage variable into blocks and looked at the average quality of those pitchers. There were five blocks which could be considered blowout territory: run-of-the-mill 4-5 run leads in the middle innings, medium-to-small leads in the middle innings, 1-3 run leads in the late innings, peak house on fire, and finally, “call Goose Gossage” leverage. Bingo! We’ve got something like the correlation we were expecting. What we see is the average pitcher in each of these five situations gets progressively better, which makes a lot of sense. However, the pattern over innings isn’t what one would expect, with peak correlation occurring in the middle innings and a large drop (in strength) in the ninth. The reason why is the fact that we see closers so often in the ninth in any save situation which spans a large range of the leverage values. This does however seem to be a good variable to target when trying to understand if a manager is tailoring his pitching decisions to the leverage rather than the inning. Here is top the list for manager seasons in the past five years and their correlation between team-adjusted FIP and leverage in the late innings, accounting for the manager’s best relief option as well as the range and the variance of those options (negative implies good management). Best Relief Leverage Managers, 2015-19 Team Manager Year Score St. Louis Cardinals Mike Matheny 2016 -0.6242 Seattle Mariners Scott Servais 2016 -0.5781 Tampa Bay Rays Kevin Cash 2015 -0.5364 New York Yankees Joe Girardi 2015 -0.5345 Arizona Diamondbacks Chip Hale 2015 -0.5334 Kansas City Royals Ned Yost 2015 -0.5261 Kansas City Royals Ned Yost 2016 -0.5113 Philadelphia Phillies Pete Mackanin 2016 -0.5015 Pittsburgh Pirates Clint Hurdle 2015 -0.4965 Boston Red Sox John Farrell 2015 -0.4957 St. Louis Cardinals Mike Matheny 2017 -0.4821 Detroit Tigers Brad Ausmus 2016 -0.476 New York Mets Terry Collins 2016 -0.4759 New York Yankees Joe Girardi 2016 -0.4655 Chicago White Sox Rich Renteria 2017 -0.4406 Milwaukee Brewers Craig Counsell 2015 -0.438 Baltimore Orioles Buck Showalter 2016 -0.4291 San Diego Padres Andy Green 2017 -0.4289 Milwaukee Brewers Craig Counsell 2017 -0.4277 Atlanta Braves Brian Snitker 2018 -0.4275 Houston Astros A. J. Hinch 2016 -0.4275 Los Angeles Dodgers Dave Roberts 2017 -0.4256 New York Mets Terry Collins 2017 -0.4209 Cincinnati Reds Bryan Price 2016 -0.4067 Seattle Mariners Scott Servais 2018 -0.4031 Conversely, the bottom of the list looks like this: Worst Relief Leverage Managers, 2015-19 Team Manager Year Score L.A. Angels of Anaheim Mike Scioscia 2016 0.6998 Atlanta Braves Brian Snitker 2016 0.6144 Houston Astros A. J. Hinch 2018 0.573 L.A. Angels of Anaheim Mike Scioscia 2015 0.5285 New York Mets Terry Collins 2015 0.4527 Colorado Rockies Walt Weiss 2015 0.4127 Chicago White Sox Robin Ventura 2016 0.2878 Oakland Athletics Bob Melvin 2015 0.286 San Francisco Giants Bruce Bochy 2016 0.2763 Kansas City Royals Ned Yost 2018 0.2372 Washington Nationals Dusty Baker 2017 0.2003 Chicago White Sox Robin Ventura 2015 0.1969 Oakland Athletics Bob Melvin 2016 0.1955 San Francisco Giants Bruce Bochy 2017 0.1673 Cincinnati Reds Bryan Price 2015 0.1318 Philadelphia Phillies Pete Mackanin 2015 0.1113 Atlanta Braves Fredi Gonzalez 2015 0.1103 Toronto Blue Jays John Gibbons 2015 0.108 Seattle Mariners Scott Servais 2019 0.07462 Cleveland Indians Terry Francona 2015 0.07063 Baltimore Orioles Buck Showalter 2018 0.03112 Arizona Diamondbacks Torey Lovullo 2018 0.02875 Atlanta Braves Brian Snitker 2019 0.02367 Kansas City Royals Ned Yost 2019 0.02061 Toronto Blue Jays Charlie Montoyo 2019 0.01285 Here is the complete list of scores for managers with more than 50 games managed in a season since 2015. Managers by Relief Leverage Score, 2015-19 Manager 2015 2016 2017 2018 2019 Mike Scioscia 0.5285 0.6998 -0.2595 -0.2889 NA Brad Ausmus -0.3638 -0.476 -0.1469 NA -0.09107 Dusty Baker NA -0.2425 0.2003 NA NA Rocco Baldelli NA NA NA NA -0.2937 Jeff Banister -0.08477 -0.2623 -0.3698 -0.2559 NA Rod Barajas NA NA NA NA 0.7869 David Bell NA NA NA NA -0.2858 Buddy Black NA NA -0.09133 -0.2447 -0.3884 Bruce Bochy -0.2596 0.2763 0.1673 -0.1525 -0.07801 Aaron Boone NA NA NA -0.3824 -0.181 Mickey Callaway NA NA NA -0.2526 -0.3298 Kevin Cash -0.5364 -0.04291 -0.1703 -0.3773 -0.1947 Terry Collins 0.4527 -0.4759 -0.4209 NA NA Alex Cora NA NA NA -0.2335 -0.304 Craig Counsell -0.438 -0.2237 -0.4277 -0.2494 -0.1596 John Farrell -0.4957 -0.351 -0.1804 NA NA Terry Francona 0.07063 -0.08552 -0.3348 -0.1803 -0.2911 Ron Gardenhire NA NA NA -0.2035 -0.1687 John Gibbons 0.108 -0.2445 -0.301 -0.08363 NA Joe Girardi -0.5345 -0.4655 -0.2138 NA NA Fredi Gonzalez 0.1103 -0.2941 NA NA NA Andy Green NA -0.3702 -0.4289 -0.27 -0.1938 Chip Hale -0.5334 -0.3837 NA NA NA A. J. Hinch -0.1909 -0.4275 -0.3549 0.573 -0.2157 Clint Hurdle -0.4965 -0.07946 -0.1268 -0.2998 -0.2531 Brandon Hyde NA NA NA NA -0.2115 Dan Jennings -0.3015 NA NA NA NA Gabe Kapler NA NA NA -0.2826 -0.343 Torey Lovullo NA NA -0.2677 0.02875 -0.293 Pete Mackanin 0.1113 -0.5015 -0.07152 NA NA Joe Maddon -0.296 -0.2616 -0.2175 -0.3262 -0.04729 Dave Martinez NA NA NA -0.1418 -0.02162 Mike Matheny -0.2466 -0.6242 -0.4821 -0.2872 NA Don Mattingly -0.03443 -0.2981 -0.3183 -0.3853 -0.2418 Lloyd McClendon -0.3796 NA NA NA NA Joe McEwing NA NA NA -0.3927 NA Bob Melvin 0.286 0.1955 -0.3996 -0.3696 -0.1787 Paul Molitor -0.265 -0.1107 -0.3823 -0.1374 NA Charlie Montoyo NA NA NA NA 0.01285 Jerry Narron NA NA NA NA NA Bryan Price 0.1318 -0.4067 -0.2384 -0.4585 NA Tom Prince NA NA NA NA NA Mike Redmond -0.02058 NA NA NA NA Rich Renteria NA NA -0.4406 -0.1089 -0.2435 Jim Riggleman NA NA NA -0.3917 NA Dave Roberts NA -0.3434 -0.4256 -0.3206 -0.2364 Ron Roenicke 0.1175 NA NA NA NA Ryne Sandberg -0.2677 NA NA NA NA Scott Servais NA -0.5781 -0.3008 -0.4031 0.07462 Mike Shildt NA NA NA -0.3859 -0.1953 Buck Showalter -0.1451 -0.4291 -0.3771 0.03112 NA Brian Snitker NA 0.6144 -0.02649 -0.4275 0.02367 Chris Speier NA NA NA NA NA Robin Ventura 0.1969 0.2878 NA NA NA Don Wakamatsu NA NA NA 0.3326 NA Walt Weiss 0.4127 -0.2612 NA NA NA Matt Williams -0.2489 NA NA NA NA Chris Woodward NA NA NA NA -0.106 Ron Wotus NA NA NA NA NA Ned Yost -0.5261 -0.5113 -0.2712 0.2372 0.02061 Looking at the top and bottom of the list brings some interesting names, with some managers appearing on both. Naturally it’s reasonable to ask how much is this really telling us about the manager. From year-to-year there is some correlation in a manager’s score, but it has decreased through time, starting around .4 correlation in the first three years down to near 0 from 2018 to 2019. Perhaps there was an advantage in late inning bullpen usage being gained in 2015-16 by mangers like Kevin Cash or Joe Girardi but certainly there is more work to do connect points. How a manager affects winning will always be nebulous and largely a function of intangibles, but it doesn’t have to be entirely so. Holding decision makers to a logical plan and attempting to quantify their ability to execute will be another tool in the belt we use to evaluate managers. The relationship between quality of pitcher on the mound and leverage of the game has many nuanced factors, surely, but more work on uncovering these will prove crucial in shining light on which managers are making the best decisions consistently to give their team the best chance to win. Here is my code, available for web scraping and analysis: https://github.com/peterloiseau/manager_analysis  Sheehan, Joe. “Prospectus Today.” Baseball Prospectus, Baseball Prospectus, 11 Sept. 2008, web.archive.org/web/20100213073350/www.baseballprospectus.com/article.php?articleid=8060.