Fantasy Rankings: Why Methodology Matters

By far, the hardest thing about fantasy baseball is the fact that you can’t predict the future. Every year, a Matt Carpenter or a Chris Davis vastly outperforms expectations and wins a fantasy league for somebody, and a Matt Kemp battles injuries all year and makes somebody else tear their hair out. But you learn to deal with that sort of thing, or you take up a less stressful hobby, like Russian roulette. C’est la vie, and all that.

What this article is about, however, is that the second-hardest thing about fantasy baseball is trying to juggle categories. Which is better, Mike Trout’s five-category production, or Miguel Cabrera’s dominance in four categories? How much is it worth to have Billy Hamilton singlehandedly win stolen bases for you while contributing nothing in the other categories? Can you absorb Pedro Alvarez’s batting average hit for the home runs he gives you? Over the years, people have come up with a few different ways to try to answer those questions. Standing Gain Points (SGP) is one popular method. Z-scores are another. There are others, but those are the two I see the most, so they’re the two I’m going to talk about. The point of this article isn’t to compare all of the ranking systems out there and figure out which one is “right.” The point of this article is to call attention to the fact that your choice of ranking system matters, probably more than you think.

Of course, most fantasy ranking systems start with projections. Personally, I like to use composite projections, because I think there’s value in combining projections and smoothing out spots where one system might be exceptionally high or low on a player. You can disagree with the projections – that’s not the point. The point is, you (or your fantasy expert of choice, if you use published rankings) can take the same projections, plug them into different ranking systems, and get substantially different results.

For the purposes of this article, I’m keeping things very simple, perhaps a little too simple. I don’t care about volatility, risk, upside, injuries, etc. I’m assuming that these projections are accurate. And I’m not going to bother with positional adjustment, because I’m lazy and these aren’t the rankings I’m drafting from, and it doesn’t matter anyway. I’m concerned with how using different methods changes players’ rankings relative to each other, not how much to bump Buster Posey up my draft board because I need a catcher. And I’m looking at rankings, not auction values, because that’s another step that I don’t feel like taking right now.

I’m going to look at the shortstop position (specifically the top 14, because I play in a 14-team league) for this article, because I need to narrow things down to a manageable number of players. I’m assuming a standard 5×5 league. And what I’m looking at is SGP (using the formula here), compared to two slightly different ways of calculating z-scores. In all cases, I’m looking at the rankings of each player among shortstops and among all hitters.  Really, though, I’m concerned with the overall rankings because I want to see how players move around – the choice to focus on shortstops is just a convenient way to select a handful of players to look at.

Anyway, on to the fun stuff:

SGP shortstop rankings:

Player Name	AB	H	R	HR	RBI	SB	AVG	ORANK	SSRANK
Troy Tulowitzki	525	157	84	28	91	3	0.300	19	1
Hanley Ramirez	510	146	81	23	81	16	0.287	22	2
Jose Reyes	573	169	88	12	54	26	0.295	32	3
Jean Segura	592	164	77	10	51	37	0.277	39	4
Ian Desmond	568	156	72	20	77	19	0.275	41	5
Elvis Andrus	612	168	80	5	60	35	0.275	47	6
Everth Cabrera	575	149	76	4	42	49	0.259	50	7
Ben Zobrist	580	157	82	15	77	11	0.271	71	8
Starlin Castro	636	177	77	12	58	14	0.278	94	9
Asdrubal Cabrera	539	141	70	16	68	11	0.261	108	10
Andrelton Simmons	578	157	73	14	61	9	0.271	115	11
J.J. Hardy	577	151	70	23	69	1	0.262	116	12
Alexei Ramirez	595	161	63	8	57	21	0.270	118	13
Bradley Miller	522	142	71	14	57	11	0.271	119	14

Looks reasonable. I don’t know. We don’t have anything to compare it to yet. So let’s compare it to z-scores. For this example, I’m going to calculate my average and standard deviation for each category using all players projected for over 300 at bats.

Z-score shortstop rankings using all players with >300 AB:

Player Name	AB	H	R	HR	RBI	SB	AVG	ORANK	SSRANK
Troy Tulowitzki	525	157	84	28	91	3	0.300	16	1
Hanley Ramirez	510	146	81	23	81	16	0.287	22	2
Jose Reyes	573	169	88	12	54	26	0.295	36	3
Ian Desmond	568	156	72	20	77	19	0.275	43	4
Jean Segura	592	164	77	10	51	37	0.277	51	5
Elvis Andrus	612	168	80	5	60	35	0.275	57	6
Ben Zobrist	580	157	82	15	77	11	0.271	65	7
Everth Cabrera	575	149	76	4	42	49	0.259	71	8
Starlin Castro	636	177	77	12	58	14	0.278	91	9
Asdrubal Cabrera	539	141	70	16	68	11	0.261	110	10
J.J. Hardy	577	151	70	23	69	1	0.262	112	11
Andrelton Simmons	578	157	73	14	61	9	0.271	114	12
Bradley Miller	522	142	71	14	57	11	0.271	119	13
Alexei Ramirez	595	161	63	8	57	21	0.270	125	14

Comparing those two tables, the methods agree on the top 14 shortstops. For the most part, these rankings are pretty similar. But Tulowitzki moves up a few spots in the overall rankings, which isn’t insignificant that early in the draft. Segura drops a round or two, and swaps spots with Desmond in the shortstop rankings. Andrus moves down the overall rankings a bit. Everth Cabrera moves down the overall rankings quite a lot, going from a mid-round steal to a guy who’s probably merely a decent value at his ADP.

So we learned a few things there, maybe. But when I use z-scores, I don’t think it makes sense to calculate them using every player who sees significant playing time – most of those will probably never be rostered in your fantasy league. I want to compare fantasy-relevant players to other fantasy-relevant players, not waiver wire fodder. So let’s take the top 200 hitters, as determined by the initial z-score rankings, recalculate the average and standard deviation for each category using only those players, and try again.

Z-score shortstop rankings using the top 200 players:

Player Name	AB	H	R	HR	RBI	SB	AVG	ORANK	SSRANK
Troy Tulowitzki	525	157	84	28	91	3	0.300	14	1
Hanley Ramirez	510	146	81	23	81	16	0.287	23	2
Jose Reyes	573	169	88	12	54	26	0.295	36	3
Ian Desmond	568	156	72	20	77	19	0.275	49	4
Jean Segura	592	164	77	10	51	37	0.277	59	5
Elvis Andrus	612	168	80	5	60	35	0.275	63	6
Ben Zobrist	580	157	82	15	77	11	0.271	64	7
Everth Cabrera	575	149	76	4	42	49	0.259	93	8
Starlin Castro	636	177	77	12	58	14	0.278	94	9
J.J. Hardy	577	151	70	23	69	1	0.262	108	10
Asdrubal Cabrera	539	141	70	16	68	11	0.261	111	11
Andrelton Simmons	578	157	73	14	61	9	0.271	115	12
Bradley Miller	522	142	71	14	57	11	0.271	119	13
Jed Lowrie	538	145	70	15	65	3	0.269	126	14

Again, everything looks pretty similar at first glance. Alexei Ramirez drops off the list in favor of Jed Lowrie, but that’s no big deal. But Tulowitzki moves up another couple spots – he’s pushing first-round value now, even before positional adjustments. Segura and Andrus drop a little further in the overall rankings. Cabrera, who was already worth less using z-scores, is even worse with a smaller player pool. Remember, that rank of 93 is only among hitters – factor in pitchers, and Cabrera, a mid-round steal using SGP, now looks overvalued at his ADP of 106 (though we can’t say that for sure without applying positional adjustments). All things considered, simply changing the size of the player pool had as much of an effect as changing from SGP to z-scores in the first place.

Depending on which ranking method you use, you’re going to place a pretty different value on some of these players (again, with the caveat that I didn’t do positional adjustments). At the top of the shortstop rankings, Tulowitzki could be anywhere from a late second round pick to a borderline first-rounder. Cabrera’s value swings wildly depending on what system you use – he’s either a player to target fairly early, or borderline undraftable where you’d have to take him. Other players, like Hanley Ramirez or Brad Miller, are remarkably consistent across all three methods, but there’s no way to know how much of that is chance.

The natural thing now is to wonder is which of these systems is right. This seems like it should be solvable. I really want there to be an answer to this, a clear way to combine five categories of production into a single overall rank. Unfortunately, I’m not convinced that exists. People smarter than me have come up with a few different ways to reach that goal, and the results don’t agree with each other. Even if they did, the needs of your team are going to evolve as the draft goes on. When you pick whatever method you prefer and compile your pre-draft rankings, the numbers you get are going to look pretty absolute, there in black and white in your spreadsheet. But really, they’re more like ballpark estimates, and they could easily be totally different.