The Risk and Reward of Attempting to Pick Runners Off

Recently, Dave Cameron examined a planned back-pick by Russell Martin and the Blue Jays in Game 1 of the ALDS.  The play didn’t have a chance to happen because Delino DeShields put a 2-1 change up in play.  Not just in play, but on the ground to directly where the second baseman Ryan Goins would have been had he not been breaking for second in anticipation of the pick.  Dave wrote a great article that covered the play in depth, so feel free to go read it here.  In this article, I analyze the strategy of calling for a set pickoff attempt. What I found not only vindicates Martin and the Jays, but also questions one of my longest-held beliefs about pickoffs.

My strategy for evaluating the set pickoff was to calculate the break-even point (BEP) for a pickoff attempt using Run Expectancy (RE), similar to previous analyses on bunting and stealing. To calculate the BEP for a given pickoff attempt, I calculated the RE benefit (to the defense) of an out and the weighted RE cost of a safe call or an error.  This sounds simple enough, but calculating the RE after an error involved some guesswork.

Although errors can result in multiple outcomes, I chose to pick one outcome for each base to simplify the analysis. Thus, I assumed 2 bases for all runners on an errant throw to first, 1 base for all runners on an error to second, and, after much thought, 2 bases for runners on second and 1 base for runners on the corners on an error to third. If you have data that can replace these assumptions, please let me know.  Otherwise, be cognizant of my assumptions when you attempt to make use of the findings.  For example, if there is a slow runner on second, the BEP for a pickoff attempt to a corner will be overly conservative (inflated).  Additionally, I didn’t differentiate between pickoff attempts from the pitcher and the catcher.  The pitcher has a shorter, unobstructed throw, and favorable balk rules when picking to second or third, but still has to deal with the risk of a balk, especially to first, along with the added difficulty of throwing off the mound.  Finally, while calling for a back-pick from the catcher can put a defender out of position, I chose to ignore this factor because a) I assume it is rare for a hitter to find the vacated hole, and b) the defense can choose to avoid contact.

In order to weight the cost of a failed pickoff attempt appropriately, I had to estimate what the error rate would be on attempts.  While we do have data on pitcher error rates on pickoff attempts (around 0.95%), the data are only from throws to first.  Set pickoff plays are more challenging for the defense, so the error rate should be higher than on typical attempts to first.  My solution, in lieu of empirical data from actual set pickoff attempts, was to estimate catchers’ throwing error rates from the 2015 season.  I chose this strategy for two reasons: First, catchers are one of the primary players who can attempt a set pickoff, so it made sense to sample from their performance.  And second, catchers accumulate a large portion of their assists under similar conditions to the pickoff attempt (for example, in 2015 nearly 40% of all catcher assists came from caught stealing).  Thus, I expected catcher throwing error rates to approximate the error rates we would observe on set pickoff plays.

While not a perfect method, I estimated catcher throwing error rate as Throwing Errors / Assists + Throwing Errors + Stolen Bases.  The mean throwing error rate in a sample of catchers (n = 38) who played at least 500 innings in 2015 was 3.6%.  Do you accept that set pickoff plays will result in 3.8 times more errors than typical pickoff throws to first? If not, adjust your own estimates accordingly.

Using the estimated throwing error rate for catchers, the formula for estimating the BEP on a set pickoff attempt is RE cost / (RE cost – RE benefit). In this equation, RE benefit = RE after a pickoff – RE before a pickoff; RE cost = RE before a failed attempt – RE with a failed attempt, and RE with a failed attempt = (RE of a safe call *.964) + (RE of an error *.036).  Using the RE tables found here, I generated Table 1 below.

 

Runners	Outs	First	Second	Third
1 _ _	0	3.51%
	1	3.32%
	2	3.24%
1 2 _	0	3.32%	2.18%
	1	4.21%	1.93%
	2	9.17%	2.33%
1 _ 3	0	2.37%		0.74%
	1	3.47%		1.92%
	2	6.72%		5.99%
_ 2 3	0		1.70%	1.41%
	1		1.93%	1.73%
	2		5.06%	5.06%
1 2 3	0	10.21%	1.97%	1.64%
	1	4.85%	2.78%	2.48%
	2	7.58%	3.92%	3.92%
_ 2 _	0		1.54%
	1		1.43%
	2		1.26%
_ _ 3	0			0.11%
	1			1.74%
	2			5.61%

Table 1.  Success rate required to attempt a pick at each base.

Table 1 presents the BEP for the defense of (successful pickoffs / attempts) X 100.  In other words, Table 1 provides the minimum expectation of success required for the defense to attempt a set pickoff and it be a break-even strategy. Unfortunately, it is difficult to guess how successful set pickoff attempts typically are.  In Dan Malkiel’s study of pickoffs to first, he found that righties and lefties were successful about 2% and 4% of the time, respectively.  However, Malkiel’s study sampled situations with base-stealers on first, so the stolen-base rate was between 17% and 21%.  It’s impossible to know what percentage of successful pickoffs occurred when the runner intended to steal, but it’s safe to say 2% and 4% success rates are a little high if the runner on first isn’t planning on going. Set pickoffs usually work differently than throws to first, since neither the pickoff nor the steal are always expected. Therefore, the data on picks to first can only serve as a point of reference, helping to calibrate expectations rather than serving as predictions themselves.

One way to assess if teams are over- or under-utilizing set pickoffs is to compare their pickoff to error ratios with the BEPs for that metric. Unfortunately, I could only find data for one special case of the set pickoff: a catcher back-pick to first.  In the Malkiel study, successful back-picks were 96% of back-picks plus errors.  If we assume an error puts the runner on third, the BEP for pickoffs/pickoffs + errors is 50%, suggesting that catchers have room to get much more aggressive in attempting to pick runners off first.  Without more data, it’s difficult to comment further on current MLB behaviour regarding set pickoff plays. Nevertheless, the estimates in Table 1 provide interesting insights into the risks and rewards of pickoff plays. Below, I list six lessons that can be gleaned from Table 1.  At least two of these lessons fly directly in the face of my own long-held beliefs, and maybe yours too!

Lesson 1

If, at any time, the defense notices that it has better than a 15% chance of picking off a runner, they should attempt the pickoff.

Lesson 2

Pickoff attempts require greater confidence with two outs, with three exceptions.  Often, the required success rate is over 5%, requiring a fairly egregious mistake by the runner to warrant a throw. The exceptions to this rule are with a runner on first, a runner on second, or a pick to second with runners on first and second.

Lesson 3

A runner on second with no runner ahead of him should probably be targeted frequently.  The BEPs are consistently low for attempting the pickoff to second, while the runner is motivated to be aggressive by the chance to score a run or steal third. Even failed attempts have the favorable by-product of keeping the runner close, a factor not considered in Table 1.

Lesson 4

Throwing behind the runner on first with runners on first and second or the bases loaded is dangerous.  This doesn’t mean it’s a bad play if the runner on first opens the door, but the defense should be really confident to make the throw.

Now for the lessons that go against everything I thought I knew…

Lesson 5

Pitchers should throw over to third with runners on 1^st and 3^rd in a steal situation.  Ever since the MLB outlawed the fake-to-third move, pitchers haven’t been allowed to bluff the throw in hopes of catching the runner breaking from first.  Based on Table 1, it seems strange that pitchers ever faked the throw to begin with.  With no one out, the defense would only need to pick the runner off third 8 times per 1000 attempts, or nail the runner stealing second 3 times per 100 attempts, or a combination of the two to break even.  Additionally, if the runner on first breaks for second it’s an easier throw from third than from first, which was often the result with the fake-to-third move.  While many old-school baseball people will object to throwing over to third, the common refrain “he’s not going anywhere!” doesn’t necessarily apply to the 1^st and 3^rd steal situation.  The runner could be trying to get closer to home so he can steal on the catcher’s throw to second, making it the perfect time to throw over.  Although the third baseman’s positioning will sometimes make a true pickoff attempt at third difficult, the rules do not require the pitcher to throw directly to third.  Thus, teams can make legitimate efforts to get the runner on third when the situation allows it, while other times making throws away from the base solely to catch the runner on first breaking for second.

Lesson 6

The situation that requires the lowest probability of success to attempt a pickoff is when there is a runner on third with no one out.  The defence needs to nab merely 2 runners out of every 1000 attempts to break even. And get this, the BEP on pickoff attempts to third with 0 out is lower than the BEP for typical throws to first, even with the much lower error rate on throws to first (0.95%), and even after adjusting the assumed cost of an error to one base.  Holding probability of success constant, the pickoff attempt to get a runner on third with 0 out is the least risky pickoff attempt possible. The LEAST risky.

Of course, a runner who is on third with no one out should be taking no chances.  But that doesn’t mean a pickoff will never work…