FIP, xFIP, SIERA are all very good ERA estimators, and their predictability is well documented. It is well known that SIERA is the best ERA estimator over samples that occur from season to season, followed very close by xFIP, with FIP lagging behind. FIP is best at showing actual performance though, because is uses all real events (K, BB, HR). Skill is commonly best attributed to either xFIP or SIERA. ERA is also well known to be the worst metric at predicting future performance, unless the sample size is very large <500IP with the pitcher remaining in the same or a very similar pitching environment.

FIP, xFIP, and SIERA are supposed to be Defense Independent Metrics, and they are. Well, they are independent of field defense, but there is one small error in the claim of defense independent. K’s and BB’s are not completely independent of defense. Catcher pitch framing plays a role in K’s and BB’s. Catchers can be good or bad at changing balls into strikes and this affects K’s and BB’s. Umpire randomness and umpire bias also play a role in K’s and BB’s. It is unknown how much of getting umpires to call more strikes is a skill for a pitcher or not. Some pitchers are consistent at getting more strike calls (Buehrle, Janssen) or less strike calls (Dickey, Delabar), but for most pitchers it is very random (especially in small sample sizes). For example Jason Grilli was in the top 5% in 2013 but was in bottom 10% in 2012.

I wanted to come up with another ERA estimator that eliminates catcher framing, umpire randomness and bias, and eliminates defense. I took the sample of pitchers who have pitched at least 200IP since 2008 (N=410) and analyze how different statistics that meet this criteria affect ERA-. I used ERA- since it takes out park factors and adjusts for the changes in the league from year to year. I looked at the plate discipline pitchf/x numbers (O-Swing, Z-Swing, O-Contact, Z-Contact, Swing, Contact, Zone, SwStr), the six different results based off plate discipline (zone or o-zone, swing or looking, contact or miss for ZSC%, ZSM%, ZL%, OSC%, OSM%, OL%), and batted ball profiles (GB%, LD%, FB%, IFFB%). *Please note that all plate discipline data is PitchF/X data, not the the other plate discipline on FanGraphs, this is important as the values differ*

The stats with very little to absolutely no correlation (R^2<0.01) were: Z-Swing%, Zone%, OSC%, ZSC%, ZL% (was a bit surprised as this would/should be looking strike%), GB%, and FB%. These guys are obviously a no-no to include in my estimator.

The stats with little correlation (R^2<0.1) were: Swing%, LD%, and IFFB%. I shouldn’t use these either.

O-Contact% (0.17), Z-Contact%, (.302), Contact% (.319), OSM% (0.206), and ZSM% (.248) are all obviously directly related to SwStr%. SwStr% had the highest correlation (.345) out of any of these stats. There is obviously no need to include all of the sub stats when I can just use SwStr%. SwStr% will be used in my metric.

OL% (0.105) is an obvious component of O-Swing% (0.192). O-Swing had the second highest correlation of the metrics (other than the components of SwStr%). I will use it as well. The theory behind using O-Swing% is that when the batter doesn’t swing it should almost always be a ball (which is bad), but when the batter swings, there are a two outcomes, a swing and miss (which is a for sure strike) or contact. Intuitively, you could say that contact on pitches outside the zone is not as harmful to pitchers as pitches inside the zone, as the batter should get worse contact. This is partially supported in the lower R^2 for O-Contact% to Z-Contact%. It is more harmful for a pitcher to have a batter make contact on a pitch in the zone, than a pitch out of the zone. This is why O-Swing is important and I will use it.

Using just SwStr% and O-Swing%, I came up with a formula to estimate (with the help of Excel) ERA-. I ran this formula through different samples and different tests, but it just didn’t come up with the results I was looking for. The standard deviation was way too small compared to the other estimators, and the root mean square error was just not good enough for predicting future ERA-.

I did not expect/want this estimator to be more predictive than xFIP or SIERA. This is because xFIP and SIERA have more environmental impacts in them that remain fairly constant. K% is always a better predictor of future K% than any xK% that you can come up with. Same with BB% Why? Probably because the environment of catcher framing, and umpire bias remain somewhat constant. Also (just speculation) pitchers who have good control can throw a pitch well out of the zone when they are ahead in the count, just to try and get the batter to swing or to “set-up” a pitch. They would get minus points for this from O-Swing, depending on how far the pitch is off the plate, but it may not affect their K% or BB% if they come back and still strike out the batter.

So I didn’t expect my statistic to be more predictive, but the standard deviation coupled with not that great of RMSE (was still better than ERA and FIP with a min of 40IP), caused me to be unhappy with my stat.

I then started to think about if there were any stats that were only dependent on the reaction between batter an pitcher that are skill based that FanGraphs does not have readily available? I started thinking about foul balls and wondered if foul ball rates were skill based and if they were related to ERA-. I then calculated the number of foul balls that each pitcher had induced. To find this I subtracted BIP (balls in play or FB+GB+LD+BU+IFFB) from contacts (Contact%*Swing%*Pitches). This gave me the number of fouls. I then calculated the rates of fouls/pitch and foul/contacts and compared these to ERA-. Foul/Contact or what I’m calling Foul%, had an R^2 of .239. That’s 2nd to only SwStr%. This got me excited, but I needed to know if Foul% is skill based and see what else it correlates with.

This article from 2008 gave me some insight into Foul%. Foul% correlates well to K% (obviously) and to BB% (negative relationship), since a foul is a strike. Foul% had some correlation to SwStr%, this is good as it means pitchers who are good at getting whiffs are also usually good at getting fouls. Foul% also had some correlation to FB% and GB%. The more fouls you give up, the more fly balls you give up (and less GB). This doesn’t matter however, as GB% and FB% had no correlation to ERA-. Foul% is also fairly repeatable year to year as evidenced in the article, so it is a skill. I will come up with a new estimator that includes Foul% as well.

I decided to use O-Looking% instead of O-Swing%, just to get a value that has a positive relationship to ERA (more O-looking means higher ERA), because SwStr% and O-Swing are negatively related. O-Looking is just the opposite of O-Swing and is calculated as (1 – O-Swing%).

The formula that Excel and I came up with is this: (I am calling the metric TIPS, for True Independent Pitching Skill)

TIPS = 6.5*O-Looking(PitchF/x)% – 9.5*SwStr% – 5.25*Foul% + C

C is a constant that changes from year to year to adjust to the ERA scale (to make an average TIPS = average ERA). For 2013 this constant was 2.68.

I converted this to TIPS- to better analyze the statistic. FIP, xFIP, and SIERA were also converted to FIP-, xFIP-, and SIERA-. I took all pitchers’ seasons from 2008-2013 to analyze. The sample varied in IP from 0.1 IP to 253 IP. I found the following season’s ERA- for each pitcher if they pitched more than 20 IP the next year and eliminated any huge outliers. Here were the results with no min IP. RMSE is root mean square error (smaller is better), AVG is the average difference (smaller is better), R^2 is self explanatory (larger is better), and SD is the standard deviation.

Wow TIPS- beats everyone! But why? Most likely because I have included small samples and TIPS- is based off per pitch, as opposed to per batter (SIERA) or per inning (xFIP and FIP). There are far more pitches than AB or IP so TIPS will stabilize very fast. Let’s eliminate small sample sizes and look again.

Now, TIPS is beaten out by xFIP and SIERA, but beats ERA and and is close to FIP (wins in RMSE, loses in R^2). This is what I expected, as I explained earlier K% and BB% are always better at predicting future K% and BB% and they are included in SIERA and xFIP. SIERA and xFIP take more concrete events (K, BB, GB) than TIPS. I didn’t want to beat these estimators, but instead wanted a estimator that is independent of everything except for pitcher-batter reaction.

TIPS won when there was no IP limit, so it obviously is the best to use in smaller sample sizes, but when is it better than xFIP and SIERA, and where does it start falling behind? I plotted the RMSE for my entire sample at each IP. Theoretically these should be an inverse relationship. After 150 IP it gets a bit iffy, as most of my sample is less than 100 IP. I’m more interested in IP under 100 anyhow.

Orange is TIPS, Blue is ERA, Red is FIP, Green is xFIP, and Purple is SIERA. If you can’t see xFIP, it’s because it is directly underneath SIERA (they are almost identical). This is roughly what the graph should look like to 100 IP:

Looking at the graph, at what IPs is TIPS better than predicting future ERA than xFIP and SIERA? It appears to be from 0 IP to around 70 IP.

Here is the graph for 1/RMSE (higher R^2). Higher number is better. This is the most accurate graph as the relationship should be inverse.

The 70-80 IP mark is clear here as well.

I’m not suggesting my estimator is better than xFIP or SIERA, it isn’t in samples over 75 IP, but I think it is, and can be, a very powerful tool. Most bullpen pitchers stay under 75 IP in a season. This means that my unnamed estimator would be very useful for bullpen arms in predicting future ERA. I also believe and feel that my estimator is a very good indicator of the raw skill of a pitcher. It would probably be even more predictive if we had robo-umps that eliminated umpire bias and randomness and pitch framing.

2013 TIPS Leaders with 100+IP

And Leaders from 40IP to 100IP

wRC is a very useful statistic.  On the team level, it can be used to predict runs scored fairly accurately (r^2 of over .9).  It can also be used to measure how much a specific player has contributed to his team’s offensive production by measuring how many runs he has provided on offense.  But it is rarely used for pitchers.

Pitching statistics are not so much based on linear weights and wOBA as they are on defense-independent stats.  I think defense-independent stats are fine things to look at when evaluating players, and they can provide lots of information about how a pitcher really performed.  But while pitcher WAR is based off of FIP (at least on FanGraphs), RA9-WAR is also sometimes looked at.  Now, if the whole point of using linear weights for batters is to eliminate context and the production of teammates, then why not do the same for pitchers?  True, pitchers, especially starters, usually get themselves into bad situations, unlike hitters, who can’t control how many outs there are or who’s on base when they come up.  But oftentimes pitchers aren’t better in certain situations, as evidence by the inconsistency of stats such as LOB%.  So why not eliminate context from pitcher evaluations and look at how many runs they should have given up based on the hits, walks, and hit batters they allowed?

To do this, I needed to go over to Baseball-Reference, as FanGraphs doesn’t have easy-to-manipulate wOBA figures for pitchers.  Baseball-Reference doesn’t have any sort of wOBA stats, but what they do have is the raw numbers needed to calculate wOBA.  So I put them into Excel, and, with 50 IP as my minimum threshold, I calculated the wOBA allowed – and then converted that into wRC – for the 330 pitchers this year with at least 50 innings.

Next, I calculated wRC/9 the same way you would calculate ERA (or RA/9).  This would scale it very closely to ERA and RA/9, and give us a good sense for what each number actually means.  (The average wRC/9 with the pitchers I used was 3.95; the average RA/9 for the pitchers I used was 3.96).  What I found was that the extremes on both sides were way more extreme (you’ll see what I mean soon), but overall it correlated to RA/9 fairly closely (the r^2 was .803).

Now, for the actual numbers:

The first thing that jumps out right away is that Koji Uehara had a wRC/9 of 0.08.  In other words, if that was his ERA, he would give up one earned run in about 12 complete game starts if he were a starter, which is ridiculous.  The second thing that jumps out is that most of the top performers are relievers – in fact, 12 out of the top 13 had fewer than 80 innings, with the only exception being Clayton Kershaw.  Also, the worst pitchers by wRC/9 had a wRC/9 much higher than their ERA or RA/9.  Pedro Hernandez, for example, had a wRC/9 of 7.68, and there were 6 pitchers over 7.00.  Kershaw actually has a wRC/9 that is lower than his insane RA/9, so maybe he’s even better than his fielding-dependent stats give him credit for.

But wait!  There’s more!  The reason we have xFIP is because HR/FB rates are very unstable.  So let’s incorporate that into our wRC/9 formula and see what happens (we’ll call this one xwRC/9):

Not a huge difference, although we do see Uehara’s number go down, which is incredible, and Tanner Roark’s – the second-best pitcher by wRC/9 – nearly double.  Also, Tyler Cloyd becomes much worse, and is now the worst pitcher by almost half a run per nine innings.  Kershaw’s wRC/9 goes up by a considerable amount, so much so that his xwRC/9 is now higher than his RA/9.  All in all, however, xwRC/9 actually has a smaller correlation with RA/9 (an r^2 of .638) than wRC/9 does, so it isn’t as useful. 

Now, logically, the people who outperformed their wRC/9 the most would have high strand (LOB) rates, and vice-versa.  So let’s look at the ten players who both outperformed and underperformed their wRC/9 the most.  The ones who underperformed:

We can see that everyone here – except for Koji Uehara, who had the fourth-highest LOB% out of all pitchers with 50 innings – is below the league average of 73.5%.  Only Uehara and Joel Peralta are above 70%.  Clearly, a low LOB% makes you allow many more runs than you should.  But what about Koji Uehara?  How did he allow all those runs (10, yeah, not a lot, but his wRC/9 was way lower than his RA/9) without allowing many baserunners to score and not allowing many damaging hits?  If you know, let me know in the comments, because I have no idea.

Now for the people who outperformed their wRC/9:

Just what you would expect:  high LOB%’s from all of them (each is above the league average).  Stephen Fife and Alex Sanabia are the only ones below 80%.

So what does this tell us?  I think it’s a better way to evaluate pitchers than runs or earned runs allowed since it eliminates context:  a pitcher who lets up a home run, then a single, then three outs is not necessarily better than one who lets up a single, home run, then three outs, but the statistics will tell you he is.  It might not be as good as an evaluator as FIP, xFIP, or SIERA, but for a fielding-dependent statistic, it might be as good as you can find.

Note:  I don’t know why the pitchers with asterisks next to there name have them; I copied and pasted the stats from Baseball-Reference and didn’t bother going through and removing the asterisks.

On Friday the Cardinals acquired Brewers reliever John Axford for a player to be named later. While dominant in 2010 and 2011, Axford’s lackluster performance since 2012 has many Cardinals fans uninspired by the move. In fact, most of the media attention has centered around his public farewell to Milwaukee fans.

Bernie Miklasz of the St. Louis Post-Dispatch offered his own analysis of the deal, calling it a “smart gamble” for the Cardinals. In addition to acknowledging Axford’s well documented HR/FB% struggles, Miklasz highlighted that the former closer has been particularly challenged by an ineffective fastball and poor performance in high-leverage situations.

PITCHf/x data on Axford’s fastball:

Axford’s performance in high-leverage situations:

FanGraphs readers will know that Cardinals GM John Mozeliak and his organization’s pitching staff have developed a reputation in recent years for quietly acquiring mediocre pitchers and helping them reach previously unimagined levels of success on the mound. To the extent that Mozeliak and company have similar designs for Axford, one must ask how they plan to help him reclaim his once dominant form.

The Cardinals may suggest any number of tweaks to Axford’s approach, but smart money has them coaching him to focus on throwing more first-pitch strikes. Jeff Sullivan recently reminded us of the importance of pitching ahead, and its import is surely not lost on manager Mike Matheny and pitching coach Derek Lilliquist. Since 2012, Redbird pitchers rank tops in the majors in terms of throwing first-pitch strikes.

In the same piece, Sullivan also noted that since arriving in St. Louis in July 2012, Edward Mujica has established himself as the league leader in first-pitch strikes, increasing that figure from a pedestrian 60.9% in 2011 to an elite 75.6% in 2013. Doing so has no doubt played a large part in his improved performance in high-leverage innings.

Mujica in 2011 with the Marlins:

Mujica in 2013 with the Cardinals:

While Mujica’s 2011 performance in high-leverage situations was not nearly as poor as Axford’s has been in 2012 and 2013, there exists a similar opportunity for improvement.

Specifically, Axford is getting absolutely crushed when behind in the count this season.

Axford’s 2013 pitching splits:

As they did when acquiring Mujica last year, look for the Cardinals to initially deploy Axford into low-leverage situations in which he can regain his confidence and focus on getting ahead in the count. If successful, one would expect the club to move Axford into higher-leverage situations, particularly if Mujica or Trevor Rosenthal wears down or runs into trouble down the stretch.

As we start a new baseball season, I start a new season of my own. This is my first – of many I hope – analysis and write-up on baseball that I am submitting. I am an avid fan, a numbers geek, an aspiring writer and lastly a bored software engineer. I am also very fortunate. I have a close connection with a former major league player and the ability to leverage his vast experience and knowledge of the game. Hopefully, I can parlay the knowledge I have learned from many years of observation along with the knowledge I have gleaned from my connection to realize my goal as a contributor to the sabermetric community and to the enjoyment of baseball fans everywhere. Here we go!

Question

Is the effectiveness of a sinker dependent on from which side of the rubber the pitcher throws?

I was in Florida in mid March for spring training, talking with a minor league coach when he mentioned that he and a former all star pitcher were in a disagreement about how to throw a sinker. Their debate centers on where a pitcher should stand on the rubber to throw a sinker most effectively. We all understand that a pitcher should not move all over the rubber to become more effective on a single pitch. This would obviously tip off the hitters as to what type of pitch might be coming. But for argument’s sake, a team might have some newly transformed position players learning to throw different pitches. Wouldn’t a team want to know if, for some pitches, it was more beneficial to stand on one side of the rubber than another?

I consider myself a pretty observant guy, but I will have to admit that I never really paid much attention to where a pitcher stood on the rubber. To me the juicy part is watching the ball just after it is released. The dance, dip, duck and dive a pitcher is able to command of the ball is where the action is as far as I am concerned. So watching what a pitcher does before he even starts his motion was asking a little much. Nonetheless, I was certain that with so many pitchers in the majors, that a breakdown of data would show that there was not a singular starting point on the rubber. Every pitcher is different, right?

Setup

I started my analysis by downloading the last 4 years (2009-2012) of PitchFx data. Most of us know this already but by using PitchFx data there are some limitations to analysis. Unlike Trackman, PitchFx initially records each pitch at 50’ from home plate, not the actual release point of the pitch. For PitchFx this data point is called “x0”, and for all intents and purposes this is pretty good data, as for most pitchers their strides are approximately 5 to 6’ from the rubber, and with arms length added in we are talking about a difference of a couple of percentage points from being the same as the release point metric from Trackman. But full disclosure, it is not exactly the release point. Another factor that I didn’t measure is a pitcher’s motion to the plate. Some pitchers throw “across” their bodies and not down a straight line, and even fewer open up their body to the batter (stepping to stride leg’s baseline). Also, there is probably a bit to glean from going between the stretch and wind-up, but again without doing a very in-depth study I assume no factor in the analysis. Lastly, arm length is an unmeasured factor. For example, I didn’t check to see if there were any right-handed pitchers with extra long arms standing on the first-base side of the rubber distorting the data.

I started by combining the PitchFx Sinker (SI) and Two-seam fastball (FT) data into a single database. The reason to combine the data is due to the fact that the grips for each pitch are the same, combine this with a two-seam fastball can and a sinker break the same way (down and in to a RH batter from a RH pitcher), and lastly they are also somewhat synonymous in major league vernacular. Maybe somewhere along the line the pitch was invented twice (north or south), the name given is based on region like when asking for a Coke… it’s a “soda”, a “pop”, or a “tonic” depending on where you are in the states. Maybe in the South it was labeled a sinker and the North it was taught as a “two-seamer”? Either way it’s the same pitch as far as I am concerned, and the etymology of pitch naming is a different topic for a different time.

Back to the question above about every pitcher being different, I was wrong. Using the 2012 data I created a frequency distribution for right-handed pitchers (figure 1), and as you can see there is definite focal area at around -2’ point from the centerline of the pitching rubber (and home plate).

Figure 1 – Right-handed pitchers in 2012

This shows that most pitchers start from about the same side; which I determined to be the right side of the rubber (3^rd base side). I determined this by adding 9” to one-half the length of the pitching rubber (24”) which comes to 21” (9”+12”). Add in arm length and you can see that using an x0 that is less than or equal to 2’ (remember we are using negatives here) should prove that the pitcher is throwing from the right side.  I would like to add that the 9” used above is based on the shoulder width of an average man, which is around 18”. This metric is based on studies on the “biacromial diameter” of male shoulders in 1970 (pg. 28 Vital and Health Statistics – Data from the National Health Survey). I think we can all agree that the 18” is probably conservative by today’s growth standards. I mentioned in the limitations of the analysis written above, I don’t account for arm length or pitcher motion. Therefore I needed to make sure that there are right-handed pitchers who are throwing from the left hand side of the rubber; just not a bunch of super long-armed, cross bodied throwers.  With the data in hand I was able to identify which pitchers had thrown the ball closer to centerline of the rubber and therefore would be good candidates for standing on the left side of the rubber. The first pitcher who had a higher (>-2) x0 value was Yovani Gallardo of the Milwaukee Brewers. Without knowing Gallardo’s motion I needed to go to the video. From the video, you can clearly see that Gallardo starts on the left side of the rubber and throws fairly conventionally, straight down the line to the batter.

I wanted to keep this as simple as possible, breaking up the pitchers in two categories – Left side or Right side. Without looking at video for each pitcher I had to come up with a tipping point for classifying the side based on the x0 data I had available. If we simply take what we determined above and correlate it to the left hand side we will come up with 1 (starting on left side of rubber) and an x0 of 0. But it isn’t quite that simple. The frequency chart shows that there are less than 1000 balls thrown in 2012 with an x0 greater than or equal to 0. Gallardo threw 504 pitches himself in 2012. So we have to increase the scope a bit. By arranging the x0 data into quartiles we see that upper or lower quartile – depending on handedness – is around -1 or 1 (remember we are using negatives) so for a right handed pitcher the x0 splits are:

For left handers:

As I am trying to stay conservative, and the fact that these are not release point numbers I use 1 and -1 as the cut off for classification based on the handedness of the pitcher. Using these numbers provided a pretty clean break in the distributions (90-10%).

Findings

So who was right, the all star pitcher or the minor league pitching coach? Is there an advantage depending on where the pitcher stands on the rubber? Neither – both of them. It’s a tie.

What can I say; my initial analysis is a bit anticlimactic, but not because of lack of effort.  To denote the labels below:

• LH or RH (Handedness)
• RR or LR (Right or Left Rubber)
• B – Balls
• K – Strikes
• P – In play (No Outs)
• O – In play (Outs)
• BackK – Called Strikes
• FT – Two seam fastballs
• SI – Sinkers
• Efficiency – O/(P+O)
• XSide – Cross Side (i.e. RH-LR or LH-RR)
• Same side – LH-LR or RH-RR

The efficiency is so very close. Twelve-hundredths (.12) of a percent is not a lot – 169 outs out of 140678 – but give any Chicago Cub fan five of those outs in 2003 and Mr. Bartman would be an afterthought. Which, I am sure is the way he and all Cub fans around the world would like it. The efficiency is the same, no other way to put it which is the beauty of statistics and sabermetrics. Numbers can say so much, even when they are the equal.

But the analysis wasn’t all for naught, there are some nuggets to glean from the numbers above. As a segue, I am currently watching Derek Lowe of the Texas Rangers pitch on opening night and from the left side of the rubber he throws a sinker and it dips back over the rear part of the plate for a called strike. With all of the similarities within my analysis the most striking observation is the difference in called strikes depending on the side of the rubber. If a pitcher, coach or manager could get a strike or a strike out without the fear of having a batter get a hit or moving a runner forward they would do it every time. With a five percent difference in getting a strike and not having the worry of the ball being put into play would be an interesting thing to know in some tight situations with runners on base. My thought on the difference revolves around the back door being open a little wider when it comes to getting called strikes. With a pitcher throwing X-side you can definitely see a pattern of called strikes on the same side of the plate from which the pitcher throws from. Positive numbers in figures below indicate right side of plate (1^st base side)

With today’s specialization where pitchers are matched up to batters based on handedness, the ability for a pitcher to throw a strike as it tails back over the plate or close to the plate (or maybe not even close for some of the pitches above ) is essential. It appears that umpires are a little more flexible with their perception of the strike zone for these pitchers as well.

Closing

I didn’t get the results that I anticipated when I started this analysis, and that is great! As a society we are determined to have a winner! Just as there is “no crying in baseball”, there are no ties in baseball. Even when there is a tie; like on a close play at first – it proverbially goes to the runner. We can’t settle for a tie…. hockey reduced ties by adding a shootout after overtime.  College football removed the tie by introducing sudden death (hopefully the bowl playoff with help eliminate the subjective BCS tie). With no clear cut advantage (read – TIE) identified in my analysis means that a more in depth analysis could/should be performed to validate. Maybe expanding the percentage of X-side pitchers to 15-20, or identifying when pitchers are throwing from the stretch and removing those instances would alter the results and provide a much needed winner? If after all analytical statistical avenues have been exhausted there’s still not a proven advantage, we can always resort to having the coach and player settle it with a coin flip?