What does Statcast’s catch probability tell us about Jacoby Ellsbury and Brett Gardner?

Barring injury or a last minute Spring Training trade, when the 2017 regular season begins, the Yankees will have Brett Gardner in left field and Jacoby Ellsbury in center field. That’s been the regular arrangement for three years now. The Yankees will have some things to figure out once prospects like Clint Frazier or Dustin Fowler are ready, but that’s not a pressing issue.

Both Gardner and Ellsbury are 33 and will turn 34 later this year. Gardner in August, Ellsbury in September. They’re at the age — beyond it, really — when everything usually begins to slip. Offense, defense, speed, everything. Soon-to-be 34-year-old baseball players are rarely as productive as they were in their 20s. Such is life. The Yankees will have to navigate their declines in the coming years.

Interestingly enough, the various stats say Gardner and Ellsbury both had their best defensive seasons in several years in 2016. That surprised me. I though the opposite would be true. The quick numbers:

You’ll have a hard time convincing me Gardner cost the Yankees nearly 40 (!) runs in the field from 2013-15 as FRAA alleges, but that’s why it’s good to look at several metrics. Generally speaking, the four main defensive stats say Gardner and Ellsbury were better in 2016 than they were in 2015 and on a rate basis from 2013-15. That’s the direction the numbers are pointing.

If you’ve watched the World Baseball Classic at all, you know there’s a new Statcast metric out called Catch Probability, which is pretty much exactly what it sounds like: how likely is it this ball will be caught? Here are the nuts and bolts of catch probability, via MLB.com:

With Statcast tracking the exact start position on the field for each fielder and also measuring the hang time of each batted ball, the two most important pieces of data to define the difficulty of a catch opportunity are: 1. How far did the fielder have to go? 2. How much time did he have to get there?

Accordingly, each tracked batted ball to the outfield is assigned an expected Catch Probability percentage — relative to comparable catch opportunities in the Statcast era — based on distance needed and opportunity time. The more time a fielder has to react to a ball and the less distance needed to reach it, the higher the Catch Probability.

Seems simple enough, right? This is only the first pass at a catch probability metric, remember. I’m sure there will be ballpark and other adjustments added as time goes on. Catch probability drops batted balls into five buckets:

• One Star Outs: Catches made at least 91% of the time.
• Two Star Outs: Catches made 75-90% of the time.
• Three Star Outs: Catches made 51-74% of the time.
• Four Star Outs: Catches made 26-50% of the time.
• Five Star Outs: Catches made 0-25% of the time.

One Star Outs are your routine cans of corn. The plays every outfielder should make even if he’s, say, late career Carlos Beltran or Matt Holliday. Five Star Outs are the most difficult plays. The fly balls and line drives that rarely get caught by even the best defenders. The math may be gory behind the scenes, but catch probability is easy to digest on this end.

We have two years of Statcast data available and therefore two years of catch probability. The defensive stats in the table above tell us both Gardner and Ellsbury were better defensively in 2016 than 2015. Does catch probability agree? Let’s look. (Shout out to the indispensable Baseball Savant for the data.)

Brett Gardner

Those Three Star Outs jump out at you, eh? Gardner went from making those catches, the ones that are made 51-74% of the time, at a 50.0% rate in 2015 to a staggering 94.4% rate in 2016. Only two players had a higher Three Star Out catch probability last year: Mookie Betts and Desmond Jennings, who were both at 100.0%.

Therein lies part of the problem: sample size. Jennings played only 65 games last year due to injury and he had only only six Three Star Out catch opportunities. Gardner, who played full-time both seasons, had only 14 Three Star Out opportunities in 2015 and 18 in 2016. He made seven of those plays in 2015, hence the 50.0% catch probability. Last year he made 17 of 18.

So, with that in mind, here again are Gardner’s catch probabilities, this time with the number of opportunities added to provide more context:

The number of catch opportunities varies wildly from player to player. Adam Eaton had 65 One Star Out opportunities in 2016. Gardner had 36. They both played everyday, but one guy had nearly twice as many cans of corn hit his way than the other. Obviously the pitching staff plays a part in this. New York’s pitching staff generated way more strikeouts (23.1%) and ground balls (46.9%) than Chicago’s (20.5% and 43.1%), hence fewer opportunities for Gardner than Eaton.

The sample sizes cause us some problems. I’m hesitant to read too much into so few data points. Gardner’s Four Star Out catch probability dropped from 46.7% in 2015 to 21.4% in 2016, but we’re talking about 29 batted balls total across two seasons. We wouldn’t attempt to analyze 29 at-bats spread across two years, would we? Can’t do the same with defense. Anyway, I promised to look at both guys, so let’s get to Ellsbury now.

Jacoby Ellsbury

Yeesh, look at that Five Star Out catch probability. Ellsbury made one such play in 28 opportunities last year. One! As with Gardner, there aren’t enough data points here to say anything definitive about Ellsbury and which way his defense is trending at this point of his career, but gosh, one catch in 28 opportunities? These numbers are a record of what happened on the field, remember. If a hitter goes 1-for-28 at the plate, it doesn’t mean he’s a true talent .036 hitter, but the 1-for-28 happened and it hurt the team.

Keep in mind Ellsbury hurt his knee in May 2015 and missed close to two months, and it’s possible if not likely the injury hampered him in the field after he returned. It sure seemed like the injury threw him out of whack at the plate. The same is possible in the field. Even then, Ellsbury’s catch probabilities were pretty good in 2015. Like Gardner, Ellsbury performed worse in three of the five catch probability categories from 2015 to 2016. And that means … I’m not sure. It could be normal year-to-year fluctuation.

* * *

As with the other defensive stats like DRS and UZR, it seems you need a sample of several seasons for catch probability to be reliable. I do think it’s a better measure of single-season defense than the other stats because Statcast more accurately measures the batted ball trajectory, the defender’s positioning, stuff like that. DRS and UZR are estimating.

So, while Ellsbury’s 1-for-28 effort on Five Star Outs in 2016 may not accurately reflect his true defensive ability given the limited amount of data, it did happen, and it did cost the Yankees runs. My eyes told me both Gardner and Ellsbury were still above-average defenders last season, Gardner moreso. Neither was as good as we’ve seen them in the past, I don’t think, and that makes sense given their ages. The various defensive stats say the opposite is true, that they were better than they’d been in previous years. I was hoping catching probability would clear that up for us, but alas. It’s just more information to consider, not a definitive answer.