Jan
14

The stats we use: UZR

By

Have you ever read an article on this site, only to encounter a strange acronym that you don’t understand? For the most part they’re either inside jokes or advanced metrics. The increasing amount of data available makes it easier for us to take raw numbers and put them into context, allowing us the ability to compare players using stats that give us not only numbers, but context. These advanced stats tell us not one thing — OBP, for instance, tells us just one thing and ignores other factors — but many things that go into a player’s value.

Over the next week or so we’ll discuss the most commonly used stats on this site. Many of these require heavy math, and we know that can turn off many people. This series of articles will attempt to explain what goes into these stats without getting into any of the heavy math. We’ll include as many resources as possible, however, in case you want to dive into the calculations yourself. By the end of the series, we’ll replace our woefully outdated and partly inaccurate guide to stats.

Linear run estimators

You might have heard the term linear weights, and maybe it intimidated you. For our purposes, however, the term linear run estimators works better, though it means the same thing. Using data from actual baseball games, a linear run estimator assigns a certain run value to each offensive event. On average, the event is said to be worth that many runs. Here’s a simple table to describe the value of each hit type (from Tango):


Event Value
Single 0.46
Double 0.75
Triple 1.03
Home run 1.40
Walk 0.30
Steal 0.19
CS -0.44
Out -0.27

There are other linear models that put these events into context, such as the base-out situation, but for the main UZR calculation we need only worry ourselves with the linear estimator.

Field zones

To determine a fielder’s range, we need to determine a zone for which he is responsible. A baseball field can be broken into 78 zones, of which UZR uses 64. They’re labeled by the position or positions they’re closest to, and also by depth. You can see a diagram of all field zones at Retrosheet.

Determining league average

If I told you that a certain player, say Herp Aherpaderp, hit 18 home runs, what could you make of that data point? Not much, really, unless you also knew the league average number of home runs that year. If Herp Aherpaderp played in the NL in 1920, he would have led the league. If he played in the NL in 2001 he would have been tied for 51st. Comparing players to their peers is important in rating them.

What we need to know here is how many hits and how many outs passed through each zone. Thanks to baseball stats companies, like Baseball Info Solutions and STATS, those numbers go on record. We further need to know which player made the plays in that zone. For example, if we’re looking at Zone 56, the one between short and third, we want to know how many outs the shortstop recorded, and how many outs the third baseman recorded. Finally, we want to know the run value per hit to that zone, which we can find using the above table.

Assigning credit and fault

At this stage we encounter math, so bear with me as I explain. Obviously, we want to know the rate at which balls in play in a particular zone turned into outs and hits. Using that and the league averages, we can then determine how many more or fewer balls a player got to than the league average player. Then, using the run value per hit, we can calculate how many runs the player cost or saved his team.

To make things a bit clearer, we’re just trying to determine which player was responsible for which hits. So if there are 1,000 hits and 1,500 outs in Zone 56, we want to know how many of those outs the third baseman converted, and how many the shortstop converted. Using this ratio, we can determine the responsible party for the hits. So, if the third baseman made 70% of the outs recorded from Zone 56, 1,050 in this example, he’s also responsible for 70% of the hits, or 700. That’s the baseline we apply to individual fielders.

Then we find the value of the balls the player caught and add subtract the value of the balls the player allowed for hits. This gives us the number or more or fewer balls the player got to. Multiply that by the run value per hit, and you have UZR runs.

What about errors?

Errors can be tricky when calculating range, because when a fielder makes an error he usually got to the ball, but could’t field or throw it cleanly. To this end, UZR initially counts errors as outs, again because the fielder got to the ball and therefore has the range to get to that ball in the future. But we still need to factor in errors somehow.

Instead of calculating the error rate for each zone, UZR calculates it by position. It uses the total number of errors committed by players at a certain position and determines the rate by dividing it by the total number of ball players at that position got to. We can then determine how many more or fewer errors a particular player made, and then, multiplying by the run value of an error, we can determine how many more runs he cost or saved his team.

That counts only reached-on-errors, ones which caused the batter to reach first base. There is another type of error, non-ROE, which means a different calculation, since it means a runner moving up rather than a runner reaching base. That’s taken care of with UZR, too.

Adjusting for other factors

Obviously, other factors play into how a defender fields a ball. First up is Park Factor, a metric I’m assuming we’re all relatively familiar with. UZR breaks down park factor into positions, including the infield as one position. The idea here is to make small adjustments for how certain parks play. If an infield plays badly — has high grass, has messy lips — that factors into defense. So does outfield space. Those all get factored into UZR.

Batted ball speed is seemingly the most important adjustment. A third baseman might be able to not only field a tapper between him and the shortstop, but have enough time to set his feet and throw. On a sharp grounder, however, the play becomes more difficult. Game stringers — people who watch the game and record every event — classify ground balls as soft, medium, or hard, and fly balls as easy, medium, and hard. Those all factor into UZR as well, with each zone getting a weight for each batted ball speed. It takes into account the difficulty of catching a lightly hit fly ball to a shallow zone, as well as a hard fly ball to a deep zone and everything in between.

Batter handedness plays a part, too, since that can cause fielders to adjust. Also, batters of a certain handedness tend to hit balls harder to certain zones and softer to others. This adjustment is made so that, for example, a shortstop defending against a left-handed batter doesn’t get extra credit for fielding a ball in the 6M zone — i.e., shortstop up the middle — when he might have been positioned there in the first place.

Then there’s pitcher ground ball ratio. This doesn’t make a huge difference, since on average a pitching staff has a, well, average GB/FB ratio. It still gets factored, though, to ensure accuracy.

Finally, we get to the base-out situation that I mentioned earlier. Again, this has to do with positioning. Middle infielders are more likely to get to their middle zones with a runner on first, since they’re playing closer to the bag for the double play.

Criticism

Earlier this week, Mike Rogers at Bless You Boys examined UZR, noting its ups and downs. He makes good points as to the limitations of UZR. The main point is the subjectivity of the batted ball type. What one stringer sees as a medium hit fly ball another might see as hard. Also, limiting the data to just three classifications might provide simplicity, but it also detracts from accuracy. And, as far as accuracy goes, UZR doesn’t always agree with other defensive metrics, most notably John Dewan’s plus/minus.

Yet despite its limitations, UZR remains the best tool we currently have to measure defense. When using it, however, Mike points out five rules we should abide by.

1. 1 year of UZR data is on par with about 50-55 games worth of offense.Would you judge Miguel Cabrera’s talents at the plate on just his games from April 1st through June? I wouldn’t, and neither would you (or so I hope). So don’t do it with defense. Personally, if I have three years of UZR data for a player, I’d rather have four. If I have four years of UZR data, I’d rather have five. I don’t believe that you can have enough.
2. One full year of defensive data is at least 1200 innings worth of data.
3. Do not use UZR per 150 games (UZR/150; found on Fangraphs’ player pages) if at all possible. It’s way too misleading.
4. If Player A is a -10 one year, +10 the next year and then +0 the next year, he’s likely an average fielder. Large swings in year-to-year data isn’t out of the norm, but you should always use an average (preferably, a weighted average) and be conservative with it.
5. When possible, use multiple defensive systems to grade a player (UZR, John Dewan’s Plus/Minus system, etc).

The biggest shortcoming with using multiple years of data, as I see it, is that if a player is in physical decline we still might rate him positively because of previous years’ data. But that’s just a minor quibble.

To remember

The most frequent criticism I hear of UZR is that we can better assess defense by just watching. Observation, combined with a knowledge of the game, should allow us to assess the defensive abilities of a player. Unfortunately, as Mike mentioned in his defensive stats post, our eyes can deceive us. Our memories get distorted, and the effect gets multiplied as we become further removed from the event. The data used in calculating UZR was observed by human eyes and recorded as such — usually by multiple people per game, to weed out bias.

In other words, UZR is based on eyeball data. It just takes a heap of such data and compiles it into a workable statistic. It tries to factor in all those contextual questions we have after seeing raw data — like how Herp Aherpaderp hit 18 home runs. Well, when did he hit those 18 home runs? In what park did he hit those home runs? Were a bunch of them cheapies that barely cleared the wall? With these questions reasonably answered, or at least accounted for, we can get a better idea of a player’s defensive abilities.

Resources

The best and, in my opinion, only place to get started is by reading explanations by the man who created UZR. That’s Mitchel Lichtman, co-author of The Book. Here are his two UZR primers.

UZR Part 1
UZR Part 2

Categories : Defense

132 Comments»

  1. Does Herp Aherpaderp play LF?

    /Obvious’d

  2. Bo says:

    Tough to get completely behind a def stat that says Tex is a below avg 1b.

    • Steve H says:

      Yet everyone who puts value in UZR agrees that it does not properly evaluate 1B and C.

      • And most educated baseball people who watch Tex (with their own eyes, mind you) say he’s good but not amazing. He’s solid, but not Mattinglyesque.

        A good/solid but not amazing/incredible defensive 1B getting a UZR that’s plus or minus 2-4 runs doesn’t seem at all anachronistic to me.

        • The Honorable Congressman Mondesi says:

          Oh stop, Tex is so good at first base that it’s impossible for any other first baseman to ever have been better than him.

          (shoots self in head with finger-gun)

          • Furthermore, it’s first base. It’s probably harder to post an extremely positive UZR there anyway, due to the nature of the position. Franklin Gutierrez can put up a 10+ UZR in CF, because his amazing range is actually useful there to make plays other centerfielders can’t make. Tex probably never even has enough chances to be THAT much better than his peers at preventing runs at first. (He does have enough chances to be far worse, though.)

            There’s fewer balls hit to first base than most of the other positions on the diamond anyway. His main job as a firstbaseman is to catch the ball thrown to him, not to field the ball hit at him.

            In a nutshell, there’s two types of defensive first basemen: Bad ones, and non-bad ones. There’s not really any “amazing” defensive first basemen, guys who are worth multiple wins for defense alone.

            (Note: I haven’t factually crosschecked this with UZR data. I may be totally wrong. Just my initial hunch.)

            • The Honorable Congressman Mondesi says:

              This is totally off-the-cuff: I totally agree with what you said in the first paragraphs, and I think we all agree UZR is probably not as effective a tool to measure first base defense as it is elsewhere on the field partly for those reasons… But I disagree a bit with the second part. Yes, first base defense is less important than shortstop of center field defense, but I don’t think it’s so unimportant that you can split first basemen into bad ones and not-bad ones. Guys who are truly good at fielding that position add a lot of value to their teams. They provide defense down the baseline, which can take away extra-base hits here and there, and they can improve the entire infield defense by turning errant throws into outs and helping to turn double plays. I actually think there’s probably not a great deal of difference between the bad ones and the average ones, but the truly good ones I think add more than we usually allow for to a team’s defense.

              • Maybe I didn’t phrase that well.

                What I mean to say is this:

                Truly “great” defensive players at first base have their impact lessened a bit by the nature of the position they play. Their runs impact as excellent defenders is minimized, at least in terms of runs allowed/prevented due to range, which is the critical and central concept measured by UZR, by the task assigned to them. Or, in other words, if you lined up all the first basemen in the league in a row, ordered from best to worst, the actual run prevention gap between the best defensive first baseman and the midpoint average defensive first baseman is SMALLER than the run prevention gap at any other defensive position. If the best defensive shortstop in the league saves you 15 runs over an average shortstop, the best defensive first baseman probably saves you only 4-6 runs, tops, over the average first baseman.

                There’s a reason teams hide bad defenders at first: it’s a position where poor range is easier to mask, because you use your range less at first.

                Roughly.

            • Chris says:

              The bigger problem with UZR and 1B is that it doesn’t take into account fielding throws from the infielders. That’s also something that can be relatively easily seen by watching the games because any throw within (roughly) 10ft of first is fieldable while those further away are not.

        • BF says:

          Please give a link to a quote of an “educated baseball” person that says Tex is just “good”.

          The guy IS “amazing” on picks, diving stops and throws to any base. And he makes every routine play with 2 hands no less. If you couldn’t see this watching the Yankees this year you need to get your head out of your spreadsheet.

          • Steve H says:

            Teixeira is also a “plus glove,” which is critical for the Angels, who have built a great run-prevention team with strike-throwing pitchers and good defenders all around the diamond. Keith Law

            Hmm, “plus glove” /= amazing in my book.

            • BF says:

              That’s your book. But “Plus Glove” does not equal just “good” either.

              And “great run prevention” would lead one to believe his UZR would be positive.

              In the last 5 years, his UZR has been negative in 4 of those years. So a plus glove is a detriment to your team in 4 of 5 years? This is a quality statistical indicator?

              The statistic is faulty. If you can’t see that Tex’s defense is a POSITIVE to winning baseball games, contrary to UZR, then I can’t help you.

              • But Tex’s defense wasn’t ever STRONGLY negative. Just mildly negative. That’s the critical point.

                His 10+ 2008 looks more and more like an outlier. Tex is probably just merely “good” and not amazing. Being a -2 to a -4 is probably still well within the realm of “good”.

                It’s an intensity issue, IMO. Being a few runs below average doesn’t make you a bad defender, or even a non-good defender, exactly.

              • JobaWockeeZ says:

                If you can’t see that Tex’s defense is a POSITIVE to winning baseball games, contrary to UZR, then I can’t help you.

                Because your memory is clear of selective bias right? No way it’s wrong. Nope.

                And by the way if it’s faulty, look at the formula as given to us in the post then fix it. I’m tried of the people complaining about UZR without doing anything else.

              • The Honorable Congressman Mondesi says:

                “Plus glove” = “good” in my book, too, fwiw. I certainly wouldn’t think that it means “amazing.” If he meant Tex is an amazing defender, he most likely would have chosen a different term.

                And he didn’t say Tex provides “great run prevention,” he said the Angels built a “great run prevention team” and then added Tex’s “plus glove” to that team.

                And UZR says Tex is a plus defensive player.

                To sum up: Huh? I get that you like Tex, we all think he’s a good defensive first baseman… But what’s your point here? And can you make it without distorting the evidence to fit your preconceived conclusions?

          • Mike says:

            I agree. Anyone who thinks Tex is responsible for saving only 4-6 runs per season must have rocks in their head. 40-60 is more like it.

    • Moshe Mandel says:

      No, if you had read the entire post, you would know that it does not say that all.

    • This will end badly.

      ———-

      (Question to the board: Does this qualify as a violation of my self-imposed ban on replying to SalBoGrantLanny comments? I’m not actually replying to him, IMO, I’m more making a general observation to nobody in particular, or possibly to the entire board, that his comment will open a line of statements and responses that will likely be slightly rancorous, what with the thinly veiled name-calling and claims of idiocy from both sides of the debate.

      What say you? Should this be the only reply I allow myself to make to SalBoGrantLanny, or should I not even say this? I await the verdict of my peers.)

      • Rose says:

        IMO, if you’re trying to truly avoid responding to his posts…you shouldn’t even post this on there. My reasoning, and this is certainly no knock to you or anybody else, but some people genuinely look for your posts…and more often than not, respond to them. THAT is where it continues on to where you probably wouldn’t want to go. Especially since, it’s BoGrantPhilTylerLannyCurlyMo’s original post…so he’ll have free reign on reading and replying as much as possible…where he might overlook an entirely separate conversation.

          • king of fruitless hypotheticals says:

            i think, however, that if you don’t actually say anything that resembles an original thought AND its not in response, by at least 2 tiers (there are two layers of nesting between you and him) then certainly it should be ok.

            for example:

            Bo: UZR sucks because Tex =/= teh pwn
            Moshe: Did you even read the article?
            JGS: UZR isn’t perfect…
            TJSC: ‘Just when I thought you couldn’t be any dumber…’

            Of course you’ll have to be very good with your quote or links, and watch out for double nesting, replying to replies that were replies to the original, etc.

            Just know that we’ll flame you mercilessly if we think we can, whether you’re abiding by the rules or not :)

      • The Honorable Congressman Mondesi says:

        You couldn’t express this thought through an Anchorman quote, Eric?

  3. JGS says:

    in terms of adjusting for errors, do the game stringers treat plays that were ruled base hits but should have been errors as errors and penalize accordingly? If they don’t, it would seem like players could get unfairly punished by subjective official scorers

  4. CountryClub says:

    What’s your take on teams like the Yanks, Sox and Rays having their own defensive ranking system? A lot of teams apparently aren’t big fans of UZR.

    • Moshe Mandel says:

      Most teams are using similar PBP systems. They may have some tweaks to the formula, but they use similar data and generally treat it similarly.

    • Mister Delaware says:

      Teams probably have better internal data feeding proprietary systems. UZR relies on points of the field but doesn’t account for variables like actual positioning and hit speed and stuff. One of those ‘better than the rest but still lacking’ stats. 10 years from now, UZR will be obsolete. In a good way.

      • Moshe Mandel says:

        I dont think so. I’m not sure, but I think teams buy their PBP data from outside sources similar to what Fangraphs and other sites with this data do.

        • Mister Delaware says:

          Yeah? I sort of figured there had to be atleast the beginnings of a Field f/x system, just not one released to the public yet like Pitch f/x is.

      • UZR relies on points of the field but doesn’t account for variables like actual positioning and hit speed and stuff.

        Yes, UZR does.

        • Mister Delaware says:

          Poorly phrased on hit speed: its done at a subjective level open to all the same biases as errors. There’s nothing set delineating a “hard” ground ball and a “medium” ground ball if its a close call? As for positioning, I had no idea that a ball in zone X was only a demerit to the fielder if he’s playing in a close enough zone. Like if an unnamed SS has a tendency to shade more up the middle, he’d still be responsible for balls in a more commonly covered zone to his right.

    • pete says:

      it makes sense. I would think (or hope, anyway) that the yanks have several scouts in every stadium, or at least people watching every game, who write down observations on EVERY play, and then that information is compiled through an in-house system, and is then cross-checked with things like UZR and Dewan +/-.

    • A.D. says:

      Figure they have systems similar, however with their own scouts they might be able to remove a bit of the objectiveness of the “hard” hit vs “medium” etc.

  5. DP says:

    Is there going to be an article on the inside jokes?

  6. Drew says:

    I refer to it as “Uzzer.”

    /Jon Miller’d

  7. I “know” Mike Rogers; he’s a good guy.

    Anyway great article. Hopefully, everyone reads it through.

  8. The Honorable Congressman Mondesi says:

    This is a great idea, Joe (and Ben and Mike, not sure how much of a collaborative effort the post or just the idea were). Maybe you guys could add these posts to the “Guide to Stats” tab at the top of the page, so they’re there for quick reference?

  9. JMK aka The Overshare's Garden Apartment Complex says:

    I don’t buy any of this, Mr. Pawlikowski (if that is your real name). We all know real articles aren’t written on spreadsheets.

    When I need to find out a proper way to get natural lighting in my basement, you’re the man I’ll see.

    Sheesh.

  10. pete says:

    This is probably the best description of UZR I’ve ever read. I didn’t realize so much hands-on analysis went into it. Still, I can’t wait until the park camera systems are all installed and we can better define defense.

    On the eyes deceiving us point – what people often forget when watching baseball is that pretty much everybody who makes it to the majors is an exceptional athlete with incredible hand-eye coordination. When a player frequently makes amazing plays, we think of him as an amazing fielder, because in our mind only an amazing fielder could make these plays. Relative to the human average, this is generally true – when Tex makes a great play, it’s generally a play that most of us most certainly could not make. But when compared to the rest of the 1Bs in the league (especially in the AL, where DHs don’t play 1B), it’s not quite as impressive, because most of those guys makes plays like that all the time too. We just don’t see them very often. Furthermore, watching highlight reels, no matter how extensive they are, is a poor way of gauging a player because it only shows the good side of a player’s abilities. To really gauge a player’s defense, it’s better to watch all the hits that drop in front of him if he is an OF, or all the grounders that get by him if he is an INF, and do so for every other player at his position in the league for comparison (all while somehow keeping track of all of the information). We of course are incapable, as human beings, of doing this. UZR is not, and while its system is imperfect, it does do a pretty good job of it. That is why smart people trust UZR over their own recollection of a player’s defense.

    /ramble’d

  11. Evilest Empire says:

    I can now finally believe in UZR since I can see the zones with my own two eyes.

  12. Ed says:

    What we need to know here is how many hits and how many outs passed through each zone. Thanks to baseball stats companies, like Baseball Info Solutions and STATS, those numbers go on record.

    This is one part I’m a little more curious on. I’ve heard that you can get raw fielding data from multiple companies to base your UZR stats on. Supposedly for a small number of players, you end up with really different UZR scores depending on the data set chosen.

    Does anyone know any more about this?

  13. pete says:

    my one gripe with UZR is the data-set issue. While 3+ year samplings are a fine way to judge players in their physical primes who have fully matured at their positions (say ages 25ish-32ish), but it’s very important to be able to judge a player in decline. Usually UZR will let you know if a guy had a bad fielding year, but, like w/ JD, you can’t be sure if it was standard UZR fluctuation or decline. From watching him play, he does appear to be a lesser fielder than he used to be, but it’s still hard to know if UZR is backing that up or just coincidentally agreeing with my unreliable memory.

    In other words, requiring a 3-year data set makes the whole system unreliable if I’m trying to rate a player whose 3-year data set includes his age 34, 35, and 36 seasons, where he is likely to be in decline, and if not, certainly likely to begin declining soon thereafter.

    • A.D. says:

      Well, when a 30 something year old position player has a bad year at the plate you don’t know if it’s decline or a down year.

      Look at Delgado a in 2008, looked as if his bat was cooked in the first half, then breaks out.

  14. CT Yankee says:

    In a future post perhaps you can explain how these metrics are validated. As a statistican, I am skeptical of composite variables.

  15. bobmac says:

    Anyone have some rope?

  16. [...] “advanced” doesn’t mean it requires an overly complex calculation. Last week we examined UZR, and that might have given off the wrong impression. UZR is complex out of necessity. A baseball [...]

  17. [...] we’ve started writing about the stats we use. One concept we saw in both current entries, UZR and wOBA, is linear weights. The idea might sound complex, but it is not. The idea is to assign a [...]

  18. [...] previous editions of this series we’ve discussed UZR, a defensive statistic, and wOBA, an offensive one. Today we’ll move onto a pitching one. It [...]

  19. [...] that we have discussed our favorites among defensive, offensive, and pitching stats, we’re going to move onto a more general one. Today I’m [...]

  20. [...] I covered in the UZR primer, the statistic is primarily concerned with assigning credit and blame on balls in play and [...]

  21. [...] fielding ability. The fact is, his range has gone from mediocre to bad to worse. Since 2002 (when UZR was first recorded), he has had two seasons in the positive: 6.4 in 2009, and 0.9 in 2002. Every [...]

  22. [...] various defensive stats — -20.6 UZR, -28 DRS, -12.1 FRAA, and -18 TotalZone — absolutely crush him at shortstop, like worst defensive [...]

  23. […] year, guys who can’t hit but can play some solid defense. The club sported a collective +12.5 UZR and +21 DRS in 2013, rates that are only slightly above-average (13th and 10th in MLB, […]

  24. My family members always say that I am wasting my time here
    at web, except I know I am getting know-how everyday by reading such nice
    articles.

Leave a Reply

You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> in your comment.

If this is your first time commenting on River Ave. Blues, please review the RAB Commenter Guidelines. Login for commenting features. Register for RAB.