The Ray Searage Effect

Much has been made of Ray Searage, and his ability to get the most out of Pitchers. In April Jeff Sullivan wrote an article on FanGraphs about Ray Searage’s work on Arquimedes Caminero and his rise in fastball velocity. Another article was written on Rant Sports last October about how the Pirates are lucky that Searage has not been offered a manager’s job due to his proven ability to get the best out of his pitchers. There have definitely been numerous examples of pitchers who have improved once they got to Pittsburgh, including Burnett, Liriano, Volquez, Worley, Caminero (as mentioned in Sullivan’s article) and this year J.A. Happ. Happ was the pitcher who motivated me to do this article, since he has had so much success after coming over from Seattle, with another great outing last Friday night against the Cardinals. With all these examples of pitchers improving on the Pirates, it seemed like there might be something here that could be quantified.

cFIP

I wanted to use Jonathan Judge’s new statistic cFIP (FIP in Context) to quantify the pitchers’ success, since it adjusts for ballpark, league, defense and many other things, including opposition quality which many other statistics fail to do. cFIP, much like FIP-, is set to a scale on which 100 is average, and 100 – x means the player was x% above average. If a player is x above 100, they would be x% below average (For example, a cFIP of 90 would be 10% above average, and a 110 would be 10% below average). This stat will account for almost any advantage you can think of when switching teams, so whether it was a hitters or pitchers park, strong or weak division, it should not matter. Not only that, but this article by Judge for the Hardball Times shows how cFIP is better than pretty much every alternative in predicting future performance, and shows what the player’s true-talent level is. If there is a consistent improvement in cFIP for these pitchers, it would point to a change in skill which could be attributed to Searage. On the other hand, if the cFIP did not seem to change considerably, then it would be more likely that either the Pirates were good at finding players who had an unlucky season (which cFIP can show) the year before and the uptick in success could be them preforming at their true-talent level. Either that or as always possible, the Pirates could just be getting lucky. Of course this could also be the case, if the pitchers did see an increase in cFIP.

The Process

First, I found all the pitchers who played one full season with the Pirates and one full season not with the Pirates in consecutive seasons. I grouped them based on whether or not they played with the Pirates on the first of the two seasons. Their Pirates season had to occur in 2011 or later, since that was Searage’s first full season as pitching coach. I limited the group to just starting pitchers who had started at least 10 games both seasons. I found the players cFIP on Baseball Prospectus and put it in an Excel spreadsheet. Unfortunately, players like Happ who switched to the Pirates mid-season could not be included, since cFIP was not recorded for players before and after they were traded, and only for the full season of data. I found the difference in cFIP between the Pirate and non-Pirate seasons (first season minus the second season), and used that to find a weighted difference based on their total games started between the two seasons (cFIP Difference * Games Started). I then averaged all players weighted differences in the group, to get the averaged weighted difference. For example, let’s say pitcher A has 50 total games started with a cFIP difference of 4 and pitcher B has 25 games and a difference of -6. The weighted average would be pitcher A’s games * difference + Pitcher B’s games * difference all divided by total games (You could add in a third, fourth, fifth pitcher and so on). This would turn out to be (4*50) + (-6*25) / (50+25) = 50/75, which is a 2/3% improvement.

Results

Here are the two tables of results with the weighted average difference in the bottom right corner.

Pitchers Joining the Pirates

Name Year Team GS cFIP Total GS Weighted Net cFIP Average cFIP Improvement
A.J. Burnett 2011 NYA 32 102
A.J. Burnett 2012 PIT 31 97 63 315
A.J. Burnett 2014 PHI 34 113
A.J. Burnett 2015 PIT 21 95 55 990
Edinson Volquez 2013 TOT 32 112
Edinson Volquez 2014 PIT 31 111 63 63
Francisco Liriano 2012 TOT 28 92
Francisco Liriano 2013 PIT 26 84 54 432
Kevin Correia 2010 SDN 26 117
Kevin Correia 2011 PIT 26 122 52 -260
Vance Worley 2013 MIN 10 124
Vance Worley 2014 PIT 17 101 27 621
Total 196 856 4.37

Pitchers Leaving the Pirates

Name Year Team GS cFIP Total GS Weighted Net cFIP Average cFIP Improvement
A.J. Burnett 2013 PIT 30 81
A.J. Burnett 2014 PHI 34 113 64 -2048
Edinson Volquez 2014 PIT 31 111
Edinson Volquez 2015 KCA 26 105 57 342
Erik Bedard 2012 PIT 24 100
Erik Bedard 2013 HOU 26 102 50 -100
Kevin Correia 2012 PIT 28 123
Kevin Correia 2013 MIN 31 116 59 413
Paul Maholm 2011 PIT 26 106
Paul Maholm 2012 TOT 31 105 57 57
Total 287 -1336 -4.66

As the tables show, when pitchers joined the Pirates, they gained a little more 4% on the league, but when pitchers left, they lost that 4% and even a tiny bit more. If these results were accurate, it would seem that the Pirates helped their pitchers in a way that could not be attributed to anything on the field, such as defense, since that is accounted for in cFIP. It could have to do with some sort of chemistry or some other sort of edge, that didn’t stay with them when they left. One hypothesis is that it could be attributed to the fact that they are one of the few teams to have a clubhouse traveling statistician who relays information to the players from the front office. I decided to take a little bit further look at these tables, however, and I found some other interesting results.

In the first table, the only pitcher to pitch on the Pirates in 2011 was Kevin Correia. This was Ray Searage’s first year as pitching coach, and you could easily say that he was still learning on the job, and that if he was giving some sort of edge, he had not mastered his skills yet. If you take out players who pitched for the Pirates in 2011, here is the new table.

Pitchers Joining the Pirates 2012-2015

Name Year Team GS cFIP Total GS Weighted Net cFIP Average cFIP Improvement
A.J. Burnett 2011 NYA 32 102
A.J. Burnett 2012 PIT 31 97 63 315
A.J. Burnett 2014 PHI 34 113
A.J. Burnett 2015 PIT 21 95 55 990
Edinson Volquez 2013 TOT 32 112
Edinson Volquez 2014 PIT 31 111 63 63
Francisco Liriano 2012 TOT 28 92
Francisco Liriano 2013 PIT 26 84 54 432
Vance Worley 2013 MIN 10 124
Vance Worley 2014 PIT 17 101 27 621
Total 144 1116 7.75

You can see that the results are changed pretty dramatically, as now pitchers are improving by about 8% compared to the average pitcher. This is very significant, and we will get back to it later. Another change you could make to the Leaving Pitchers table is to take out Burnett, who seems to be an outlier (-2048 cFIP). This could lead to some interesting results, although there isn’t as much of a reason to take him out. After removing Burnett, as well as Maholm who pitched for the Pirates in 2011, you are left with only 3 players, but here are the results.

Pitchers Leaving the Pirates 2012-2015 (minus Burnett)

Name Year Team GS cFIP Total GS Weighted Net cFIP Average cFIP Improvement
Edinson Volquez 2014 PIT 31 111
Edinson Volquez 2015 KCA 26 105 57 342
Erik Bedard 2012 PIT 24 100
Erik Bedard 2013 HOU 26 102 50 -100
Kevin Correia 2012 PIT 28 123
Kevin Correia 2013 MIN 31 116 59 413
Total 166 655 3.95

This time the results change even more significantly then before, as now pitchers improve by 4% on the league when they leave the Pirates. I am not suggesting that you can just remove Burnett from this list, as he definitely counts, but the fact that the results do a 180 reversal by removing one player (Maholm would have made the pitchers improve even more) shows two things. 1) That the data isn’t very conclusive, but also 2) that it looks like there is not much of a trend.

Putting this new information together, you can come to another conclusion. It seems recently that pitchers improve rather significantly when they come to the Pirates, but there isn’t much evidence they regress back to their original performance when they leave. This points directly to the option that Ray Searage is improving these players in ways that stick with them once they leave. There is by no means conclusive evidence with such a small data set and there are many other possible hypotheses, but by weeding through this data, it certainly looks like a strong possibility. The Pirates definitely should be thrilled that Searage has not gotten a job as a manager, even though he may provide more of an advantage as a pitching coach, where he can focus solely on helping his pitchers. If he keeps this up however, and a bigger sample size of data backs up these results, you can bet that he will at least get some interviews for a manager’s job.

Questions or comments are much appreciated.





My name is John Graves and I am in 12th grade in Vermont. I love baseball and have found baseball analytics and sabermetrics very interesting, especially since I enjoy math.

newest oldest most voted
Rob Mains
Member
Member

John, I’m blown away that you’re a tenth grader – this is a very thoughtful article. I assumed you were an adult. This is sophisticated work. I’m a Buccos follower (onthefieldofplay.com) and your piece motivated me to look at some of the Pirates’ relievers. I’m not sure what the proper weighting would be (probably innings pitched), but here are some raw cFIP numbers: Jason Grilli 89 in 2009, 79 with the Pirates in 2011 (he missed 2010 with knee surgery, traded mid-2014 so no “after” stats); Mark Melancon 92 in 2012, 70 with the Pirates in 2013; Antonio Bastardo 87… Read more »

Catoblepas
Guest
Catoblepas

Enjoyed this, but I would’ve liked a little more discussion of what other factors could be causing this. The Pirates are notably aggressive with their shifting, and that might be a potential cause. Not to say that Searage isn’t responsible in some way for that, but learning how he’s responsible beyond just “making pitchers better” would be great.