I don’t want to use this as another debate about how good or bad Jacoby Ellsbury is or anything, but this is a chance to look at the different metrics and where they come from. I’m going to center on four major metrics and what they attempt to measure as best we know.
First up is John Dewan’s plus/minus measurement from The Fielding Bible. Only leader boards are available for free and the rest is in the yearly Fielding Bible, so access requires you pay for the data. This has to be one of the most involved as each players ranking involves video scouts watching every play a player makes and grading him against his peers. The resulting plus or minus value is based on how many more or less plays he make than the rest at that position.
This system has a less direct effect on scoring, but how to compare players defensively. Taking a look at 2008 you have Adrian Beltre as the best third basemen in baseball with a +32. On the other end you have Edwin Encarnacion who was a -21. This number is not a run value as I understand it though and more of a comparison tool. It intends to say that Beltre made 53 more plays defensively than Encarnacion in 2008.
The plus/minus system plays into another Dewan system called DRSor Defensive Runs Saved. It takes the the plays that added or subtracted to their plus/minus and assign run values to them. This gain or loss of run values results in a total value of expected runs. Let’s see the explanation straight from John:
“Let’s say there’s a man on first with one out. The expected runs at that point are .528. The next play is a ground ball to the shortstop. He boots it for an error and we now have men on first and second with one out. The expected runs went from .528 to .919. That’s an increase of .391 (.919 minus .528) runs. The play itself, the error, cost the team .391 runs. We don’t have to follow it through and count the rest of the inning. We know what the value of the ending state is and can use it.”
Similar to the idea of some of the others below, but again this uses actual scouts to watch each play and access if the player should have made the play or not.
Next up is the one that has taken our collective attention lately, UZR. This was developed by Mitchel Lichtmen and freely available on FanGraphs.com. This is why the stat has grabbed so much attention as the site has become one of the most popular sites for stats in the past few years.
The stat takes zones into accounts and equates values to balls hit to the players in these zones. Each type of hit has a certain chance of being caught by the average fielder. If the player can get to more than the average he will gain value in his “range factor”. Then you factor in errors, arm for outfielders and double play skills for middle infielders.
The UZR score has been poorly defined by many and while it has huge limitations I think some don’t know what those actually are. Some question the assumptions of zones to the player since players are positioned differently all the time. Next up is the one that Tom Tango and Jeff Zimmerman have discussed recently dealing with sample size. Based on this I like to think about UZR as a similar metric to ERA. One and even two seasons of data has enough variability to be questioned.
The last one I want to look at today is PMR or Probabilistic Model of Range created by David Pinto. This is a very similar metric to UZR, but with a few changes. Let’s again listen to the creator of the metric give an explanation:
“I calculate the probability of a ball being turned into an out based on six parameters: direction of hit (a vector), the type of hit (fly, ground, line drive, bunt), how hard the ball was hit (slow, medium, hard), the park, the handedness of the pitcher, the handedness of the batter.”
The difference between the two systems is how they reach the final number, but they are fairly similar. David Gassko put the differences best in his article in 2006.
Pinto’s approach is very different from Lichtman’s, though what the two systems are trying to do is very similar. Under the UZR system, a ball is assigned a probability of being caught by a certain fielder, and then that probability is adjusted based on the various factors listed. PMR uses empirical probabilities, meaning that it looks at each ball in play that was the same type of batted ball, hit in the same direction, with the same “hardness”, in the same park, thrown by a pitcher of the same handedness, and hit by a hitter of the same handedness, and assigns its ratings based on the probability of that specific type of ball in play being made into an out by each fielder.
To put this simply UZR looks at a broader group of data points and then adjusts for certain factors like park, batters handness, pitchers groundball/flyball rate and base/out situation. PMR tries to narrow the data points down by all these limitations before calculating the run values.
I don’t think any of these have it perfect, but the point is we look at them all and try to push them in the right direction. The fact that this discussion is happening more often should only help the best metric come forward and be refined. Will it be one of these 4 or the others like FRAA or DER? No one can say, but I can say for sure that fielding percentage will continue to be viewed as less relevant all the time.
Loading ...
Great post. But you should clarify that, contrary to your explanation, UZR (Ultimate Zone Rating) is not related to Range Factor. RF is put outs and assists per 9 innings and is an interesting but simplistic and unbalanced stat.
Also, for a better explanation of plus/minus, check out the guide from the Fielding Bible's site. In short, it assigns a probability of making an out for a ball hit in a given direction at a given speed, and then subtracts that from a fielder's score if he doesn't make the play or adds (100 – probability) if he does make the out. It rewards guys when they make tough outs and penalizes them for goofing on routine balls.
http://www.billjamesonline.net/fieldingbible/over...
Troy, what kind of agreement is there between PMR and UZR? If the two processes use pretty different methods and come to similar results when judging actual players then that would suggest that both are measuring something real, right?
They are different, but the data is similar. They have some agreement, but I wouldn't say that proves anything since they get there in opposite ways. I will look for it, but there was a study on their correlation and they had some differences at second base and center field on agreement.
Obviously, the several systems look at different facets of the same diamond (no pun intended, but not a bad analogy either).
The descriptions are esoteric because the level of analysis of each "opportunity" is esoteric and, actually, subjective, in that two equally talented "scouts" reviewing the same play could come up with two very different interpretations of that play. Multiple witnesses of a crime almost never give the same story. Even well trained pathologists with the best equipment, looking at cancer cells, often get it wrong. Offensive stats are so simple: how many hits? how many runs? how many rbi? How often does he get on base? Same with pitching. How many GO/AO, how many ER, how many K's. IMO, unless defensive stats can gain this level of intuitive simplicity, or find a way to translate the math to street level, they will be resented.
About a 1,000 years ago, in order to get the College Of Cardinals to elect a new Papa, the people of Rome locked them into the gallery and wouldn't let them out until they got the job done. Perhaps MLB could get the best of these bright statisticians into a room in Bermuda, along with some open minded scouts, and translators, and over a couple of months come up with standardized defensive stats that are considered accurate, intuitive, easy to understand, and discuss. They could be introduced in a one year trial, modified, and become etched in stone.
Just remember that teams are funding some of these studies for private use. This is then a reason we can't be clear of the best statistic or method. I think it will take a large undertaking by someone on the outside to get even better. The video systems they are installing to track fielders I think will be the beginning of a much more accepted defensive metric.
How does Placido Polanco lead the world in UZR among 2B's for 2009, yet he ranks 7th in RZR?