What’s in a projection system?

You might have noticed throughout our 2010 season preview series that we’ve been combining various projection systems at the end of each article. Each system handles projections differently, so we wanted to get an average, just to see if we can weed out some more bias. Yet the same caveat still applies to the averaged projection: it’s just a projection. It’s not saying that this is how Player A will hit in 2010. It’s saying that, based on the methodology, this is the best idea we have of the player’s potential production.

As John Sterling often says, you just can’t predict baseball. There are so many moving parts, so many variables, so many unknowns that predictions simply cannot take them all into account. You can compensate for the unknown, but you can’t factor it into predictions and projections. Thankfully, projection systems aren’t trying to predict anything. Instead, they’re taking the available data and putting it through a process which outputs its best idea of a player’s future performance. But, because of all the factors it cannot consider, these projections are often inaccurate.

Then why have them? Because it’s better than assuming a player will repeat his numbers from the previous season. Few players put up the same numbers year after year. Production fluctuates. Players get unlucky and players go on hot streaks. A pitcher can throw a perfect game and then allow five runs in his next start. Projection systems try to smooth this out, taking all available data and processing it in order to give us an idea of a player’s next-year production.

Projection systems have their biases, too. PECOTA, for example, hammers older players. That does make a degree of sense, because production tends to decline as a player ages. Not every player, though, declines along the same path. So when PECOTA projects a rough year from Jorge Posada, it’s just using the data available as it relates to the player, a 38-year-old catcher. That doesn’t mean Jorge will necessarily decline along the same lines.

All this is to back up Rob Neyer‘s disbelief in the projections for a few Yankees’ veterans. There’s an article, written by a notorious pot-stirrer, that basically says, “here are the PECOTA numbers, the Yankees could be in trouble.” It’s pretty benign, really. PECOTA projections have been out since late January, so we’ve all had a change to look over them (or at least those of us with a BP subscription). We know that other projection systems aren’t bullish on the veteran Yankees. Yet I’m with Neyer when he says

I don’t know enough about the guts of PECOTA to rail against it. Instead I’ll just say that I don’t believe that Jeter is going to steal 10 bases this season, and that I don’t believe Mariano Rivera will save only 22 games. I will say, too, that if your system says those things, it’s probably worth checking under the hood just in case one of the belts is running a little loose.

In other words, projection systems use general principles to project individual players. While there’s certainly merit in the exercise, it sometimes can’t nail down the outliers. That might be what’s at play here.