Mailbag: Quality Starts

Saturday Night Open Thread
What's Your Countdown?
(AP Photo/Rob Carr)

David asks: What do you think about QS % as an important measure of starting pitchers? Given the Yankees’ ability to score runs, and their strong bullpen, I’d place a high value on a pitcher’s ability to deliver quality starts, i.e., to be “consistently good.” On that basis we of course notice or confirm a number of things from 2011 AL pitchers with 140+ innings:

  • CC Sabathia was of course consistently good at #10
  • Michael Pineda was #8!
  • A.J. Burnett‘s problems are NOT the occasional blow up; he’s consistently ineffective (worst in the league)
  • Freddy Garcia wasn’t too far behind CC in consistency at #17
  • Ivan Nova was middle of the pack at #27 (in the neighborhood of Gio Gonzalez, John Danks, Alexi Ogando, Phil Humber)
  • Hiroki Kuroda was #14 in the NL, as a solid number two should be

I think that “quality starts” themselves are kinda silly, or at least the term “quality start”  is silly. Three earned runs (why don’t we count unearned runs? they count on the scoreboard) in six innings is a 4.50 ERA, and that would have been a 116 ERA+ in 2011. If they changed the name of the stat to something like “decent start” or “winnable start,” them maybe it would be easier to swallow. That’s how I think of a quality start, it’s a winnable game for the Yankees with their offense and bullpen. My extension, QS% would just tell you how often a certain pitcher threw — not will throw — a decent or winnable game.

Quality starts are in no way predictive, they’re an output stat. They tell you the end result of the game without telling you how it happened. Did the pitcher throw eight innings of one-run ball with four hits, one walk, and nine strikeouts? Or six innings with three earned runs, two unearned runs, ten hits, three walks, and two strikeouts? In terms of quality starts, those two are the same thing. They’re kinda like the pitching version of RBI. We know the run came in, but we don’t know how it came in. Quality starts and RBI don’t tell us how likely the player is to do it again.

Like I said, I consider a quality start to be a winnable game for the Yankees, and that’s basically all I look for out of the back of the rotation, the Burnett’s and Hughes’ and Garcia’s in 2012. If the Yankees get a quality start out of those guys half the time, they’re well ahead of the fifth starter curve. I wouldn’t consider QS% to be a great measure of success or a high one percentage to be all that great without knowing more. It’s a great quick reference thing, but I have a tough time putting more stock into it, kinda like OPS. We have better metrics these days, but the old ones are still okay for a quick glance.

Saturday Night Open Thread
What's Your Countdown?
  • Monterowasdinero

    Question about that picture and CC’s grip. What pitch is he throwing?

    • Mike HC

      Nice pick up on the grip. Don’t know what he was throwing but I imagine it was nasty, ha.

    • vin

      Changeup, using the circle-change grip. That grip helps him get “inside” the ball, which gives it the down-and-away action when thrown to a righty.

      • Monterowasdinero

        Thanks. I knew it was an offspeed pitch. CC’s change to righties is a big part of his success and doesn’t get enough credit. He can throw it in hitter’s counts too which is huge. It should be emphasized more in the handbook of successful pitching especially in this era of splits.

        • RetroRob

          Perhaps he can give his new, equally basketball-sized rotation mate a few tips this spring on the art of throwing a change.

  • I am not the droids you’re looking for…

    Exactly. The threshold is arbitrary, imperfect in approach, and not super useful. I look at QS the other way around. It doesn’t tell me a pitcher pitched well, or how well, it simply says they didn’t pitch a disaster of an outing* which is why AJ is dead last!

    * Even this is possible though since QS doesn’t take into account unearned runs. Setting aside for a moment how terrible the distinction bt earned and unearned runs is when measuring pitcher effectiveness, given “the way it works” a pitcher could give up 20 runs in 7 innings and have it be a QS with a mis timed error in the mix…

    • Mike HC

      Pitching to a 4.5 era or below in the AL East is a decent start, but in the NL West, where half the starters seem to have era’s under 3.0, ha, not so much.

    • Soriano Is A Liar

      Or, a pitcher could go 9 innings with 4 runs, while giving up only 5 hits, no walks, and 11 K’s, win the game 6-4, and we’d have to be like, “Well that wasn’t a quality start”.

      • Steve (different one)

        This is my biggest beef with the stat. The definition should be changed to “at least 6 innings with the game ERA of less than or equal to 4.5”. Doesnt make sense that 6/3 is a QS but 9/4 isn’t. Unless the assumption is that bullpens never give up runs.

        • Yazman

          Great point! That would be a better stat.

          Also, as others mentioned, it’s important to note that QS does NOT = 4.5 era, but rather the range is 0.00 – 4.50

          I submitted the letter, but will admit the stat is LEAST useful when evaluating pitchers for the post season, where you really need “outstanding” starts…

  • Big will

    He is throwing a circle changeup

  • Mike HC

    QS are straight meaningless. Let’s be honest here. Why not a stat for “super quality starts” – 7 inning 3 runs. Or an endless amount of other arbitrary stats.

    • Mike HC

      Maybe if they changed the name to “Below Average, Above Replacement Start” I would like the stat more.

      • Sayid J.

        because a 6 inning 3 run start is not below average

        • Mike HC

          Seems like it is to me if the avg era in the mlb was around 3.9 last year. So six innings 3 runs gives you 6 innings of below the league avg era.

          • Steve (different one)

            But that is only the bare minimum to qualify. Seems to me that there are 40 ways to qualify for a QS (0/1/2/3 ERs, and 6/6.1/6.2/7….8.2/9 IP). 39 of those ways yield an ERA of less than 4.50. Obviously those are not uniformly distributed, but I would bet that the average ERA of all QS is much lower than 4.50. A pitcher who gives you 30 QS is probably quite unlikely to be below average.

    • Bubba

      I would agree with you if you only look at it as an individual stat. If you look at its team impact such as burning out the bullpen or putting the offence behind it probably has some merit.

  • ryan

    It’s probably a good tool to tell you who sucks more so than who’s great. Like if a pitcher ranks last in this category (AJ) your pitcher puts you at a disadvantage.

    • CMP

      Agree. It seems like a shittiness threshold. If you can’t give your team “quality” starts on a pretty regular basis, you really don’t belong in the rotation.

      • Cris Pengiucci

        While I generally agree, as was pointed out, it’s a shittiness threshold that shows past performance. While not likely, it is possible a pitcher will be less shtty going forward (not sure that makes anyone comfortable if he’s your pitcher, like AJ). :-)

  • Virgil Earp

    And where was Phil Hughes, 2nd to last ahead of AJ? This is why the rotation should be CC, Kuroda, Pineda, Nova, Garcia. Period. AJ and Phil are not as good as those 5. Put Phil in the pen and remove AJ from the roster.

    • Bo Knows

      You do realize Hughes was out a couple of months right?

      • Sayid J.

        You do realize Hughes being out a couple months doesn’t affect a rate statistic right?

        • Bo Knows

          You do realize that a small sample size is not good when using rates or percentages because they can be heavily influenced by outliers

          • Adam Parker

            Sayid surely knows this, and would never put his hatred of Phil Hughes* above his knowledge of small sample sizes.

            * Disclaimer: I have no idea if Sayid hates Phil Hughes.

  • CJ

    I think there is something to a QS % as a pitching measure. A three tier measurement of all starts dominant/quality”winnable”/poor. Dominant may be 7 or more innings 2 or less earned runs and poor could be 5 or less IP or 5 or more ER. This would measure frequency of the occasional blow up just didn’t have it days and the lights out days on the mound with the decent quality starts. This would sort of give an explanation of wins and ERA.

  • CMP

    The.whole idea of the stat it’s stupid anyway . Why is 6 innings and 3 runs “quality ” but 5 innings and 1 or 2 runs isn’t?

    • CP

      Because there has to be a cutoff somewhere. It’s not perfect, but no stat is.

    • ryan

      I mean 5 innings is technically a start right? If you pitch 5 you qualify for a win, shouldn’t that be the cut off? I’d prefer 5 shutout innings to 6 innings 3 runs. Maybe if you only pitch 5 it has to be 2 runs or less to be “quality”…. lots of ways to go with this.

    • bpdelia

      Well because 5 innings is unnaceptable and 5 ip 1 r starts are rare. Either a) the guy needed 110 pitches to get through five and left half the game for the bullpen (bad) or he got hurt.

      So what if its arbitrary its actually quite useful and quite frankly i think it hits the sweet spot pretty well. 6 innings is the minimum a decent start shoulg go. 4.5 era is basically the decent threshold.

      As far as its predictive value everyone is so quick to write it off and maybe you are right but do we know this?? Seems to me that leaderboard would be pretty damn stable.

      Just using fip and era don’t give you an idea how long a pitcher throws and doesn’t weed out inconsistency.

      I think the email writer has a pretty good point.

      • Mike HC

        Except if a guy pitches 30 starts and has a 4.5 era, that is not a “quality year.” Last year the average AL Era was 4.08 and average NL Era was 3.81. So that is a pretty poor year. Nothing “quality” about it.

        • Spiff

          For starters? Or for all pitchers.

        • bpdelia

          Yeah but here is the insane things about that

          A) the Stat wasn’t invented last year. Its context neutral obviouslyn. Sure in 2003 that 4.5 looks slot better

          2) amazingly not EVERY quality start is EXACTLY 6 ip with EXACTLY 3 er. Its merely a baseline Stat that tells you how many times a guy had a decent start. That’s all. And in my opinion the baseline is pretty good. 6 ip and 3 er is pretty decent. 6 ip and 4 er really isn’t acceptable and 5 ip is not acceptable. If the Stat ass called DS (decent start) there would be slot less controversy.

    • thumper

      Because in the mid 80s a journalist by the name John Lowe “invented” the stat. A writer invented the save rule as well.

      What have we learned from this – baseball writers are some of the last people I want coming up with statitics.

    • Rookie

      Exactly, CMP. Exactly. The problem is the word “quality”. ERA or WHIP relates much more to quality than “quality” starts do.

  • CJ

    Pineda at 8 is very exciting. He can win a lot if games if he turns out regular quality starts as he works on dominant stuff.

  • ryan

    Hughes made 8 starts out of 14 with 2 runs or less and at least 5 innings. you could chalk up his 1st 3 starts to dead arm and say when he came back he made 11 starts with 8 being above “Quality” starts.

    • CJ

      Great point in favor of Phil.

  • RichYF

    Mike, I think you just gave yourself some homework. Why not define “winnable” start (similar to QS, but add in UER or whatever else you want, quality start, and then even one more tier for “ace-type” start (or something like that) and “AJ-type” start (bottom of the barrel of course). Maybe running those numbers will tell a different tale. It could be interesting, or it could be useless…

    • CJ

      I say ” dominant start” 7 or more innings and 2 or less earned runs. Leave the unearned runs out, all measures have some flaw.

      • Adam Parker

        I agree having another tier called “dominant start” but the word dominant implies more than 7 innings, no? Maybe 8 innings, 2 or less earned runs?

  • CJ

    See this measure does mean something, I thought AJ was weighed down by starts where he was blown up.

  • Pasqua

    QS has to be one of the laziest “stats” ever created.

    • CJ

      I would say incomplete rather than lazy. As a % frequency measure- dominant % + quality % + poor % =100%. I think it is an interesting statistical measure

  • Dave

    Seems to me the nature of the question is less about the value of the QS statistic and more about how we measure consistency. Mike says there are better tools out there. What stats should we use to measure consistency?

  • Mister Delaware

    When 4 ER over 9 is viewed as failing where 3 ER over 6 is succeeding, its the stat that failed.

    • CJ

      You’re right that should also be a “quality start” my proposal is define/measure dominant starts, define and measure poor starts and everything in between is a quality start. DS + QS + PS = 100%

  • Steve (different one)

    It’s an OK stat as long as you don’t try to make it mean something more than it does.

    Its obviously lousy for parsing top pitchers and drawing distinctions between two pitchers who are superficially similar.

    But here is where it does have value:
    Measuring the effectiveness of your entire rotation. I bet if you sorted entire teams by QS%, the list would roughly show contenders on top and pretenders on the bottom. If you look over entire seasons, a lot of the noise in the stat (unearned runs, treats 8 shutout innings the same as 6 IP 3 ER the same) would wash out. Since 6 IP 3 ER is the minimum to qualify, the more QSs you have the more likely the total ERA is to be under 4.50. Obviously this wont be perfect, but as a quick and dirty, it’s probably decent.

    I would think it is also an important stat for evaluating your BACK end starters. These are guys, as RAB has said over and over, like Garcia where you honestly don’t care if HOW they are getting their results as long as they are getting them. If Garcia is giving you QS 8 out of 10 starts, who cares if his FIP is a run higher than his ERA?

    • 28 this year

      It might be helpful for evaluating starters to say whether they did their jobs over the season. But its useless for predicting what backend starters are going to do going forward. Nothing about QS suggests an ability to keep on doing that. There’s no underlying thing about the stat that offers predictive value. Its a measure of results with no future diea.

  • Steve S.

    There should be a new stat for guys like AJ Burnett, Joe Saunders and Joe Blanton.

    You could call it “Quantity Starts”

    • Steve S.

      They’re kinda like the pitching version of RBI. We know the run came in, but we don’t know how it came in.

      BTW-Loved that

      • CJ

        Some would say the run that score is all that matters. You’ve seen players high five eachother for a run scoring double play.
        Rarely do we see % success scoring a runner from 3rd with less than 2 outs.
        I think that is a critical statiscal measure.

        • Ted Nelson

          I don’t think anyone would say that a run scored that results in two outs is the same as a run score that results in no outs… so I don’t think anyone would say that the run scoring is all that counts. That players high five over a run scoring doesn’t mean that they’re happy the guy grounded into a double play rather than one of several better results.

  • Kosmo

    Some stat had to replace the CG. With the advent of setup men and bullpen specialists from the six inning on, the game has become more oriented towards the bullpen. If you really want to know how well a SP performed per start check the boxscore or game logs. Even though it´s not the perfect stat QS does indicate to some degree if the pitcher is pitching “deep“into the game and that he kept his team in the game. 3 runs over 6 innings is a winnable game. This stat has nothing in MO to do with league averages , 6 innings per start, 3 runs equals a 4.50 ERA. A great pitcher will give you maybe 15 great starts per season, games considered above league average and then maybe 10 league average starts and the rest of the time he´s lucky to survive the 5 inning.

  • MikeW

    While I agree that “quality starts” is a weak at best stat what I find interesting is the way the Yankees starters line up – Pineda, CC, Kuroda, Garcia, Nova and a distant last – Burnett. (Hughes did not throw enough innings to qualify). It would not surprise me if by September the rotation lines up pretty close to what quality starts indicates. I would hope it will line up like this – CC, Pineda, Kuroda, Hughes and Nova … Garcia either traded or in the BP and for whatever he can bring Burnett traded. I know the popular line of thinking is trade Burnett for salary relief but if I were Cashman and I could get a valuable player in return I would skip the salary relief and pay a greater % to gain that useful piece. But I do think that the Yankees (and perhaps Burnett) would be better off if Burnett was elsewhere. At a minimum this would probably give Hughes his last chance to prove he is a solid AL East pitcher.

  • Genghis

    Many of the arguments here are silly. First, the “semantic” discussions– should it be called “quality” or “acceptable” or “winnable”– are pointless. The more significant arguments–whether it’s valid stat– miss the forest for the trees. The arguments against QS are essentially sabremetric in nature–i.e., is QS a good predictor of future performance, compared to other stats. The conclusion the sabremetric folks make is correct– it is inferior to other available stats as a predictor, although not entirely useless in isolation (the fact that CC does well in QS is obviously not random)– but it is beside the point. QS is not intended as a sabremetric stat– it is a “popular” stat. It’s intent is to provide insight into the general performance of a pitcher in a way that is meaningful to the average fan. “Popular” stats have requirements that sabremetric stats don’t– they have to be understandable and easily calculable by an average person. FIP fails as a popular stat for that reason. What QS has done is attempt to fit a void in popular statistics in measuring the performance of starters. THe most important thing for a starter is to be able to consistently go reasonably deep into games with reasonable performance. QS is a fair attempt at a popular stat expressing that. It could be made into a better measure, but the tradeoff would be additional complexity.

    • CMP

      Yeah but the point that a lot of people have been making is that pitching 6 innings with a significantly higher than average league ERA is not a laudable achievement therefore saying a pitcher has X qualities starts doesn’t mean much.

      To me, it’s the equivalent of giving a position player a quality start if he went 1-4 with a single, no walks and made no errors or run saving plays in the field. It’s not horrible but it’s certainly nothing great either.

      • Tom

        But for back end starters isn’t that what you want?

        People are quick to point out that 4.50 ERA is worse than league average, but you know what….

        #4 and 5 starters are generally worse than league average.

        So while I would not measure the CC’s, Verlander’s and Felix Hernandez’s of the world with this stat…. to look at how potential 4/5’s line up on this stat is not as bad and useless as people seem to think.

        It’s also worth noting that the average AL starter ERA was 4.21 last year, yes 4.50 is worse than average but is it really “significantly” worse? And until 2010, 4.50 was actually either right at or better than AL league average starter ERA.

        It’s not a great stat, but it also is not the “useless” stat people make it out to be. (It’s also used better as a descriptive as opposed to a predictive stat)

  • MarkL

    It seems to me that with the NL rules, a quality start would be much harder to come by with the possibility that the pitcher might be taken out regardless of how well he’s doing If the situation comes up where he might be pinch hit for. How could someone who’s pitched 5 shutout innings not be considered a quality start if the situation arises where he’s taken out through no fault of his own. Throw away the whole stat.

  • Urban

    It is what it is.

  • RetroRob

    An okay stat as a very, very initial look at a pitcher before digging into the numbers. When the Yankees first picked up Freddy Garcia a year back, I looked at his QS stats, which were encouraging, and then went deeper. It probably tells us something more valuable than RBIs does, but that’s not the highest of standards. The biggest gripe from those above seems to be how “quality” is defined.

    Like the saves stat, QS is not going away since players use it during the negotiation process.

  • RetroRob

    Oh, and it’s pretty funny that people are complaining about the Quality Starts stats, saying it does mean much of anything, yet then pointing out that it’s not a surprise AJ is at the bottom.

    Logic fail.