Sunday Rockpile: Measuring Defense

When I glanced over the recently announced nominees for the 2013 Gold Glove awards, my mind went time travelling back to high school and into a particular argument I had one day with a girl who was convinced that there isn't really "defense" in baseball. Being a novice to sports concepts, she was very confused about how baseball players play defense because "they aren't guarding anybody."

From that moment until this one, I had given very little thought to her confusion and chalked it up to a misunderstanding of the concept. But looking over some recent Gold Glove winners and seeing Eric Young Jr.'s name on the ballot, I suddenly felt a kinship with the young lady from my story; is it possible that many baseball researchers, myself included, are living in the dark ages when it comes to understanding defense?

It's easier to measure defense when you are guarding a particular individual as you can measure that individuals production and see whether or not the defender had positive of negative impact. In baseball, however, things get much more complicated.

The way I see it, there are two unique challenges to overcome when attempting to decipher defensive value each with two main roads currently available to travel. Does one use the eye test or the metrics? And which is more important; what a player can do or what a player has done?

Eye test vs. The Metrics

There are major flaws in using solely one or the other of these methods to determine the value of a player in the field. The eye test leads easily to instances of confirmation bias and the insulation of information. In other words, using the eye test can often drastically limit ones ability to draw legitimate conclusions because inevitably each set of eyes will have varying levels of access to each player.

If a center fielder makes an incredible play in a Rays vs. Astros game (with nobody around to hear it) does it still make a sound?

The fact is, in baseball it's nearly impossible to watch every player on every team an equal amount over a 162 game schedule. On top of that, our eyes often lie to us depending on what we were looking for in the first place and human beings have proven a propensity to remember the occasionally spectacular over the consistently very good.

This partially explains how Eric Young Jr. ended up on this year's list of Gold Glove nominees. A lot of eyes saw him make a few spectacular plays that would likely not have needed to be so if played by a left-fielder with better anticipation and route running abilities. Conversely, relatively few eyes witnessed how lost he looked on so many plays in the Coors outfield.

They rarely show Sportscenter highlights of a guy fumbling around in the corner of the outfield, after a bad jump and a bad route, not being able to pick the ball up before finally making a weak throw, missing the cutoff man, and turning a maybe double into a no-doubt-stand-up triple.

But a guy flying around at 80 miles an hour and making diving catches? Now that's good TV!

I remember watching highlights of outfielders making incredible diving catches with my dad as a kid and being super excited about how amazing they were. But "Andruw Jones," my dad told me, "isn't on the list because he wouldn't have needed to dive for any of those balls, he'd be in the spot waiting for it to fall into his glove, having a sandwich."

With all of these limitations to the eye test, it is only natural that defensive statistics are growing in number and scope. Much like with hitting and pitching,there has been a wave of new defensive statistics in the last few decades in MLB. Compared to those other areas, however, defensive stats are still in the starting blocks and may never be able to get out.

I'd like to think there will be a way to more accurately measure defense, but I wonder if ultimately those more accurate measurements will still end up only telling half of the story.

Player Ability vs. Player Production

One of the things that makes this whole task so complicated is that value can be determined by either what a player has already accomplished or, conversely, what a player is likely to accomplish given his skill-set. Think of last season's version of Carlos Gonzalez. Many defensive metrics hated his performance, and several of us conducted eye tests that backed that up, largely due to a season in which he was explicitly taking it easy in order to avoid injury.

Carlos did not actually produce at a high level in 2012 defensively most of the time (i.e. he didn't save that many runs) but in all reality he didn't become a much worse defender, just a less productive one.

The fact that there were several plays he did not make does not mean he was no longer capable of making the next play if severely needed. In a way, this makes giving the Gold Glove award kind of like crowning the "batting champion." If you do it based on a seasons statistics, you aren't really honoring the best possible defender so much as giving an award for the player who tallied the most number of points that particular season.

In a similar way that someone can lead the league in batting average but not be the most feared hitter in the league, player's can have incredible defensive statistics but still not necessarily be the most well equipped to make the next needed incredible play.

Measuring what has happened does not always tell us what will happen, Hume taught us that.

This is why scouts get paid to do what they do. Production can be deceiving, it's based on any number of variables unrelated to the player's abilities.

When speaking of defensive statistics these variables include how many balls happen to be hit in your direction, how many base runners attempt to advance against your throwing arm, how the defense was positioned before the pitch, the weather, and oftentimes the abilities of your teammates.

What a scout does, then, is to rate a player's abilities in order to project future performance regardless of these variables. If one player has a better combination of skills, one might theorize, then it doesn't matter how many more plays the less skilled player has made, the more skilled player is more likely to make the more challenging play more often.

So doesn't it make a certain amount of sense to give the only award given for defense not to the player who made the most plays, but to the one who's skill-set you would take in a blind draft? Defensive metrics are making strides and Frangraphs notes:

In recent years, though, we've seen two fielding metrics rise above the rest of the field and establish themselves as reliable: the Dewan Plus/Minus system (AKA, Defensive Runs Saved) and Ultimate Zone Rating (UZR). Both systems don’t always agree, but when compared with each other and taken in large sample sizes (~3 years), you can get a good idea of a player’s fielding abilities.

Cynical translation: the best defensive stats available are essentially useless at helping to determine...

The Gold Glove

If defensive metrics are in the dark ages, the awarding of the Gold Gloves are in the stone age. Defensive stats may be misleading over a single season or unsatisfactory in a number of categories but at least it's in the right realm of the game.

The fact that offense is so regularly taken into account for Gold Glove recepients is just plain dumb beyond belief. It's embarrassing for MLB how inept it has been at honoring it's best defenders. There may not be a perfect answer, but let's at least stay on the right category of accomplishment.

The Academy did not give Daniel Day Lewis the best actor in a comedy for his work in Lincoln no matter how good, or even sporadically funny he was. The NBA regularly honored Ben Wallace as one of it's best defensive players during seasons in which he averaged approximately one shot attempt per harvest moon.

No matter how amazing he truly is, or believes he is, Kanye West will never win the Grammy for best female vocal.

So why should professional baseball players be awarded the highest defensive honor in the game based, in any way, on their ability to do something else entirely?

Maybe the whole voting process should be reworked, but as I pointed out above, it's not like anyone has come to a perfect answer yet. I hope some grand amalgamation of eye tests and new statistics and combining skill-set value with production will catapult us into a renaissance of measuring defense.

Also, good work on deeper understanding of the nuance of each position (we didn't even get into the complications of the extra duties given to first baseman and catchers that can't be measured in things like Range or UZR) is currently being done by the guys at Fangraphs, and I'm excited to see what comes of their research.

They write:

Defense statistics have long been the bane of saberists everywhere. While offense and pitching stats are easy to quantify because they result from distinct, calculable events, how do you easily measure a player’s range, positioning, arm strength, and error rate? These sort of measurements are complicated and involve lots of work to ensure accuracy.

Clearly, no conclusions were reached here, but I hope we can use this more as a starting place to figure out exactly what the parameters for the discussion should be. Before we go looking, we need to know what we are looking for. Where does the real value of defense truly lie? And how can it be accurately rewarded?