Numbers Don’t Lie: Derek Jeter is a Horrible Shortstop

New York Yankees third baseman Alex Rodriguez batted a .286 in 2009. That’s .283 on Astroturf, .268 at night and .313 for games played away from home. His batting average may be scrutinized ad infinitum, but how his defensive performance stacks up is less clear. Baseball players accumulate a wealth of hitting and pitching statistics throughout their careers, but quantifying how they field has always proved difficult. Now, all that could change. A team of statisticians at the University of Pennsylvania developed a model they say is the most accurate fielding measurement ever published.It’s called the Spatial Aggregate Fielding Evaluation, or SAFE. It is designed to improve upon other models that rate players now, such as the Ultimate Zone Rating, UZR, and the Plus/Minus system.

Models like the UZR and Plus/Minus system divide the field into zones and calculate the average percentage of balls fielded in these zones. Fielders then receive a score based on their aggregate performance in assigned zones relative to the expected total.

For example, if the average second baseman is expected to field 11 out of 20 balls hit to one zone, but actually fielded 13, he would receive a “+ 2” rating for that zone. Ratings for each of his zones are then added to get his final fielding score.

But statistics professor Shane Jensen, who created the new model, contends that the use of zones often misestimates a fielder’s contribution. These models could have a large amount of data in one zone and none in another, which throws off the rating.

SAFE, instead, treats the field as a continuous plane and uses statistical modeling to predict fielding ability. Using this model, Jensen can take existing data and calculate the probability that a fielder would get a ball, even if it wasn’t hit to that particular part of the field .

So, say a player reliably catches balls hit 100 feet to his left, but has never caught a ball hit 20 feet to his left. One would assume he could catch those balls, but previous models wouldn’t give him credit. SAFE, however, looks at all the balls this player has fielded and assigns probabilities he could catch others across the continuous plane.

SAFE even impresses UZR creator, Mitchel Lichtman. SAFE “is definitely the way to go,” he said. “If I had the statistical knowledge that [Jensen] has … I would do it that way.”

While Jensen and his team had published data from the 2002-2005 seasons last year, the data set’s accuracy was difficult to validate without data from more seasons. However last month, at a symposium for sports statistics wonks in Cambridge, Mass., he unveiled results from data up to the 2008 season. These numbers bolster his belief that SAFE is the best fielding metric that has been published.

One of SAFE’s calculations surely pleased the Cambridge crowd and confirmed what analysts have argued for some time: Derek Jeter of the New York Yankees has been one of the consistently worst defensive shortstops playing today.

Jeter “did have sort of a weird blip up in 2008,” Jensen said. “He’s no longer terrible and more like mediocre.”

SAFE also gives statistical backing to other claims that pundits have ranted about. For example, Manny Ramirez of the Los Angeles Dodgers has been one of the worst left fielders, while Andruw Jones of the Texas Rangers and Chase Utley of the Philadelphia Phillies come out on top in center field and second base, respectively.

Jensen also found that winners of the Gold Glove, given to players judged to be exceptional fielders, were “generally no better than other players in terms of SAFE, and a few were actually much worse.”  For example, despite some of the worst SAFE rankings, Jeter managed to win three Gold Gloves from 2004 to 2006, while second baseman Bret Boone won three Gold Gloves from 2002 to 2004 during his time on the Seattle Mariners.

Also, if he still plays like he did with the Rangers, SAFE predicts Alex Rodriguez might be better at shortstop than Jeter.

However, according to Dave Cameron, a baseball analyst for The Wall Street Journal, while it appears “there are some interesting things” with SAFE, most baseball analysts have not given the ratings an in-depth look.

Cameron praises SAFE’s methodology as “a cool concept,” but he laments that SAFE has not gained much traction compared with other fielding metrics due to its relative lack of availability to the public. “When SAFE was first introduced, there was a lot of excitement,” he said.  “It just hasn’t gone anywhere.”

While the UZR and Plus/Minus system are easily accessible and regularly updated, Jensen has been slow to release his results and admits SAFE has remained primarily an academic exercise.

Baseball analysts “have plenty to gain by looking at SAFE,” Jensen said. “But my main audience has been the academic statistics community.” He also cites the large upfront cost to obtain the detailed data he needs as preventative to releasing the data quickly.

Nevertheless, people are noticing his work. Recently, an NBA team contacted Jensen to develop a similar model to predict the shooting ability of players.

Jensen’s next goal is to compare players across seasons. A Boston Red Sox fan, he also plans to keep an eye on Jeter.  “I’m going to have a good look at him to see if he’s any better than I remember,” he said.  “And unfortunately, I think I’m going to be seeing a lot of him this October.”