Space, Physics, and Math

Numbers Don’t Lie: Derek Jeter is a Horrible Shortstop

A New Tool Purports to Deliver the Most Accurate Fielding Statistics Ever

October 26, 2009

New York Yankees third baseman Alex Rodriguez batted a .286 in 2009. That’s .283 on Astroturf, .268 at night and .313 for games played away from home. His batting average may be scrutinized ad infinitum, but how his defensive performance stacks up is less clear. Baseball players accumulate a wealth of hitting and pitching statistics throughout their careers, but quantifying how they field has always proved difficult. Now, all that could change. A team of statisticians at the University of Pennsylvania developed a model they say is the most accurate fielding measurement ever published.It’s called the Spatial Aggregate Fielding Evaluation, or SAFE. It is designed to improve upon other models that rate players now, such as the Ultimate Zone Rating, UZR, and the Plus/Minus system.

Models like the UZR and Plus/Minus system divide the field into zones and calculate the average percentage of balls fielded in these zones. Fielders then receive a score based on their aggregate performance in assigned zones relative to the expected total.

For example, if the average second baseman is expected to field 11 out of 20 balls hit to one zone, but actually fielded 13, he would receive a “+ 2” rating for that zone. Ratings for each of his zones are then added to get his final fielding score.

But statistics professor Shane Jensen, who created the new model, contends that the use of zones often misestimates a fielder’s contribution. These models could have a large amount of data in one zone and none in another, which throws off the rating.

SAFE, instead, treats the field as a continuous plane and uses statistical modeling to predict fielding ability. Using this model, Jensen can take existing data and calculate the probability that a fielder would get a ball, even if it wasn’t hit to that particular part of the field .

So, say a player reliably catches balls hit 100 feet to his left, but has never caught a ball hit 20 feet to his left. One would assume he could catch those balls, but previous models wouldn’t give him credit. SAFE, however, looks at all the balls this player has fielded and assigns probabilities he could catch others across the continuous plane.

SAFE even impresses UZR creator, Mitchel Lichtman. SAFE “is definitely the way to go,” he said. “If I had the statistical knowledge that [Jensen] has … I would do it that way.”

While Jensen and his team had published data from the 2002-2005 seasons last year, the data set’s accuracy was difficult to validate without data from more seasons. However last month, at a symposium for sports statistics wonks in Cambridge, Mass., he unveiled results from data up to the 2008 season. These numbers bolster his belief that SAFE is the best fielding metric that has been published.

One of SAFE’s calculations surely pleased the Cambridge crowd and confirmed what analysts have argued for some time: Derek Jeter of the New York Yankees has been one of the consistently worst defensive shortstops playing today.

Jeter “did have sort of a weird blip up in 2008,” Jensen said. “He’s no longer terrible and more like mediocre.”

SAFE also gives statistical backing to other claims that pundits have ranted about. For example, Manny Ramirez of the Los Angeles Dodgers has been one of the worst left fielders, while Andruw Jones of the Texas Rangers and Chase Utley of the Philadelphia Phillies come out on top in center field and second base, respectively.

Jensen also found that winners of the Gold Glove, given to players judged to be exceptional fielders, were “generally no better than other players in terms of SAFE, and a few were actually much worse.”  For example, despite some of the worst SAFE rankings, Jeter managed to win three Gold Gloves from 2004 to 2006, while second baseman Bret Boone won three Gold Gloves from 2002 to 2004 during his time on the Seattle Mariners.

Also, if he still plays like he did with the Rangers, SAFE predicts Alex Rodriguez might be better at shortstop than Jeter.

However, according to Dave Cameron, a baseball analyst for The Wall Street Journal, while it appears “there are some interesting things” with SAFE, most baseball analysts have not given the ratings an in-depth look.

Cameron praises SAFE’s methodology as “a cool concept,” but he laments that SAFE has not gained much traction compared with other fielding metrics due to its relative lack of availability to the public. “When SAFE was first introduced, there was a lot of excitement,” he said.  “It just hasn’t gone anywhere.”

While the UZR and Plus/Minus system are easily accessible and regularly updated, Jensen has been slow to release his results and admits SAFE has remained primarily an academic exercise.

Baseball analysts “have plenty to gain by looking at SAFE,” Jensen said. “But my main audience has been the academic statistics community.” He also cites the large upfront cost to obtain the detailed data he needs as preventative to releasing the data quickly.

Nevertheless, people are noticing his work. Recently, an NBA team contacted Jensen to develop a similar model to predict the shooting ability of players.

Jensen’s next goal is to compare players across seasons. A Boston Red Sox fan, he also plans to keep an eye on Jeter.  “I’m going to have a good look at him to see if he’s any better than I remember,” he said.  “And unfortunately, I think I’m going to be seeing a lot of him this October.”

About the Author

Alex Liu

A Bay Area native, Alex Liu studied toxicology at the University of California, Berkeley and then spent three years developing oncology medication at Genentech. Currently attending New York University’s Science, Health and Environmental Reporting Program, he hopes to bridge the gap between science and public policy. He’s interned with NOVA scienceNOW and CNN’s medical unit, and loves working on all things video. In his free time he enjoys powering through seasons of television shows, traveling, rooting for Oakland sports teams, and stepping out onto the dance floor. You can visit his personal website and follow him on Twitter.



Julie says:

With the increased reliance on statistics in sports (i.e. Moneyball by Michael Lewis), it will be interesting to see how SAFE’s methodology will develop. Thanks for posting – very interesting article.

Scott says:

Leave it to a Red Sox fan to design a system that shows Jeter is the worst shortstop sounds pretty typical to me. I can’t imagine that they would give the worst shortstop 3 gold gloves. This system also makes its predictions based on balls that it determines he should get. Does it take into account bad hops in poor playing conditions or not making a play because you are out of position due to runners on base? Jeter also plays everyday, often playing through pain and injuries, not to mention playing in the toughest division in baseball. Ultimately his numbers will make him a first ballot hall of famer.

Carol says:

Yep, I agree with Scott.

I had some hater “friend” of mine who clearly is jealous of DJ send this link to me.

Seems that the worst shortstop in baseball can still get World Series rings?

Maybe that Jensen needs to do a algorithm on how come the Red Sox can’t seem to win but every 100 years!

Scott says:

Jeter won his fourth Gold Glove Award today

Mike says:

The Gold Glove is a popularity contest as meaningless as the Heisman Trophy. Of course Jeter won it.

David says:

So Scott… you’re saying Jeter is the ONLY shortsop who gets bad hops and has to play through pain? Come on now. The system may not account for those situations, but unless Jeter has consistently terrible luck, every shortstop will have the same problems with bad bounces and injuries. And it seems to me that the Gold Gloves are largely based off of how many errors you make and how many spectacular plays you make. If Jeter isn’t reaching the balls, he isn’t making errors, and balls that may have been routine for other shortstops may look “spectacular” for him. Therefore, the Gold Glove system is flawed. This SAFE stuff looks like pure genius. ESPN should start using it.

Cody says:

Well actually shortstops that call a dome home wouldn’t have to deal with bad playing conditions more than often. Also, Other shortstops don’t have to play through pain they can sit out. Anyways it just doesn’t make sense this continued hate on one of the best players to play in the last 15 seasons.

Ray says:

Jeter had the best fielding percentage in the American League this year and the fewest errors of his career.

Cassie says:

jeter is a fucking all around shortstop of all time.people can hate all they want but come on.. he’s gonna be a damn legend. so try wasting your time on something else. nuff said.

JR says:

The only people that defend Jeter are Yankees fans, while plenty of neutral observers that don’t have a feeling on the matter can call it without bias. Jeter haters should be equally distrusted, but the allegation that they are “haters” because they say he is terrible is tautological. Surely some of the many analysts that say Jeter is phenomenally overrated are unbiased, and surely their point of view is more legitimate than an uneducated, biased one.

@Ray: this is the first time he has ever done that. Congrats to him. Perhaps he is getting better with age.

@Cassie: you did not make a coherent argument in the slightest.

@Cody: it is not helpful to assert that he is one of the best of the past 15 seasons when the discussion is about that very proposition.

@Carol: yes, terrible players can win championships, particularly when they’re surrounded by great players. and great players cost money. and the yankees spend the most. baseball is a business; the label “sport” should be reserved to leagues that have salary caps.

The fact is, if SAFE reliably predicts who ANALYSTS consider to be the best fielders at their respective positions every year for the next 100, all of you will still report that you “know” Jeter to be the best. What he’s best at is not baseball (according to fielding percentage, other than in the 2010 season, and every analyst I have ever heard), but at selling purple/pink/yellow yankees caps to little rich girls that think he’s cute, which allows Stein to throw around more money than most groups of 3 other teams, which brings home championships, and which further proliferates the insane delusion that he is a quality MLB shortstop.

rich says:

You idiots are ridiculous about jeter being terrible and this nonsense about statistical probability bullcrap. Ever heard of watching the man play. Jesus, you nerds forgot about that huh? Im not even a Yankee fan but to call Jeter the worst shortstop in baseball is just so ignorant that only a hater would spew that dumb shit out of his mouth. The reason yankee fans defend him is because they watch this legend, yes legend, play every day and if you did the same like I have (Im A MET FAN AND HATE TO ADMIT IT!!) you would understand

rich says:

oH yea by the way, i bet the idiots calling him the worst shortstop never picked up a glove or a bat in their life, i can count on that

Joe says:

I knew since the start of Jeter’s career that he was lower than avg at fielding. I don’t need stats and the gold gloves are a farce.

I’m not sure where you’re getting your info, but good topic. I needs to spend some time learning much more or understanding more. Thanks for great information I was looking for this info for my mission.|

Charlie says:

Dumb it down. A bad defensive player, at the most important defensive position in front of the plate, could not have won as much as Jeter did.

Leave a Reply

Your email address will not be published. Required fields are marked *


The Scienceline Newsletter

Sign up for regular updates.