I don’t want to use this as another debate about how good or bad Jacoby Ellsbury is or anything, but this is a chance to look at the different metrics and where they come from. I’m going to center on four major metrics and what they attempt to measure as best we know.
First up is John Dewan’s plus/minus measurement from The Fielding Bible. Only leader boards are available for free and the rest is in the yearly Fielding Bible, so access requires you pay for the data. This has to be one of the most involved as each players ranking involves video scouts watching every play a player makes and grading him against his peers. The resulting plus or minus value is based on how many more or less plays he make than the rest at that position.
This system has a less direct effect on scoring, but how to compare players defensively. Taking a look at 2008 you have Adrian Beltre as the best third basemen in baseball with a +32. On the other end you have Edwin Encarnacion who was a -21. This number is not a run value as I understand it though and more of a comparison tool. It intends to say that Beltre made 53 more plays defensively than Encarnacion in 2008.
The plus/minus system plays into another Dewan system called DRSor Defensive Runs Saved. It takes the the plays that added or subtracted to their plus/minus and assign run values to them. This gain or loss of run values results in a total value of expected runs. Let’s see the explanation straight from John:
“Let’s say there’s a man on first with one out. The expected runs at that point are .528. The next play is a ground ball to the shortstop. He boots it for an error and we now have men on first and second with one out. The expected runs went from .528 to .919. That’s an increase of .391 (.919 minus .528) runs. The play itself, the error, cost the team .391 runs. We don’t have to follow it through and count the rest of the inning. We know what the value of the ending state is and can use it.”
Similar to the idea of some of the others below, but again this uses actual scouts to watch each play and access if the player should have made the play or not.
Next up is the one that has taken our collective attention lately, UZR. This was developed by Mitchel Lichtmen and freely available on FanGraphs.com. This is why the stat has grabbed so much attention as the site has become one of the most popular sites for stats in the past few years.
The stat takes zones into accounts and equates values to balls hit to the players in these zones. Each type of hit has a certain chance of being caught by the average fielder. If the player can get to more than the average he will gain value in his “range factor”. Then you factor in errors, arm for outfielders and double play skills for middle infielders.
The UZR score has been poorly defined by many and while it has huge limitations I think some don’t know what those actually are. Some question the assumptions of zones to the player since players are positioned differently all the time. Next up is the one that Tom Tango and Jeff Zimmerman have discussed recently dealing with sample size. Based on this I like to think about UZR as a similar metric to ERA. One and even two seasons of data has enough variability to be questioned.
The last one I want to look at today is PMR or Probabilistic Model of Range created by David Pinto. This is a very similar metric to UZR, but with a few changes. Let’s again listen to the creator of the metric give an explanation:
“I calculate the probability of a ball being turned into an out based on six parameters: direction of hit (a vector), the type of hit (fly, ground, line drive, bunt), how hard the ball was hit (slow, medium, hard), the park, the handedness of the pitcher, the handedness of the batter.”
The difference between the two systems is how they reach the final number, but they are fairly similar. David Gassko put the differences best in his article in 2006.
Pinto’s approach is very different from Lichtman’s, though what the two systems are trying to do is very similar. Under the UZR system, a ball is assigned a probability of being caught by a certain fielder, and then that probability is adjusted based on the various factors listed. PMR uses empirical probabilities, meaning that it looks at each ball in play that was the same type of batted ball, hit in the same direction, with the same “hardness”, in the same park, thrown by a pitcher of the same handedness, and hit by a hitter of the same handedness, and assigns its ratings based on the probability of that specific type of ball in play being made into an out by each fielder.
To put this simply UZR looks at a broader group of data points and then adjusts for certain factors like park, batters handness, pitchers groundball/flyball rate and base/out situation. PMR tries to narrow the data points down by all these limitations before calculating the run values.
I don’t think any of these have it perfect, but the point is we look at them all and try to push them in the right direction. The fact that this discussion is happening more often should only help the best metric come forward and be refined. Will it be one of these 4 or the others like FRAA or DER? No one can say, but I can say for sure that fielding percentage will continue to be viewed as less relevant all the time.