On sample sizes, too small and just right

We talk a lot about small sample sizes on River Ave. Blues. For stat-minded analysts, it can easily become a default excuse for a player’s performance. Sabathia not pitching up to expectations? Small sample size. Blue Jays leading the AL East? Small sample size.

It is a phrase that sounds good, but most people have no idea what it means. In fact, even people who know what it means aren’t too comfortable with the concept. After all, there is no magic tipping point when a small sample size becomes a just-right sample size.

To that end, two pieces hit the Internet yesterday that warrant a reading by fans looking for more insight into the sample size process. On the one hand, Dave Cameron at FanGraphs writes about the proper way to use small sample sizes. On the other, Jonah Keri warns about reading too much into small sample sizes. “As unsatisfying as it is to say it,” Keri writes, “sometimes stuff just happens.”

As the Yankees gear up to take on the Angels tonight, the Bombers find themselves on the precipice of a frustrating April. They’ll finish the month either at or two games above .500 and could easy have found themselves at 15-7 instead of 11-11 or 12-10. That’s just part of the sample size issue.

Some players are off to bad starts; some are off to good starts. When the dust settles in the end, those small sample sizes will morph into season-long representations. For now though, when a player can go 5-for-5 and raise his average nearly .040 points in the process, we’re still firmly in the realm of a small sample size. These numbers can inform our discussion, but in no way can draw many, if any, conclusions from them.