The Leverage Argument: A SABRmetric-phobe’s guide

Fan Confidence Poll: July 5th, 2010
Halfway through, a status report
Photo Credit: Charles Krupa, AP

There are few things in baseball more frustrating than a blow lead, and the Yankees’ shaky bullpen has broken our hearts on more than one occasion this season. Joe Girardi generally does a pretty good job managing his bullpen, but there’s always a better way to deploy your best relievers.

Written by Rebecca of This Purist Bleeds Pinstripes and You Can’t Predict Baseball fame, this guest piece breaks down the concept of using your best reliever in the most important spot of the game for everyone who’s adverse to baseball’s statistical revolution.

Q: What is the leverage argument?

A: The leverage argument states that a manager should use his best pitcher at the game’s most critical points.

Q: So you mean, like, CC Sabathia should be pitching the ninth inning?

A: Well, no. A CC Sabathia who’s pitched eight other innings is probably not as sharp as a Mariano Rivera who hasn’t pitched any that day, and the ninth inning is not necessarily the most important part of the game.

Q: What do you mean the ninth inning isn’t the most important part of the game?

A: Well, in many games, a team will take a lead early in the game, in the first or second inning, and then lead and build on that throughout the game, so that by the eighth inning, they might be up by six runs, than say the one run they led by in the second.

Q: So then you’re saying Mariano Rivera should pitch the second inning?

A: Again, no. Unless your fifth starter is the April 2009 version of Chien Ming Wang, your starting pitcher is always going to be more valuable than the reliever. The leverage argument focuses primarily on what occurs after the starting pitcher has been removed from the game. This will, hopefully for your team, take place in the latter three innings, and not in the second or third inning, unless something’s gone horribly wrong.

Q: All right, hold up a sec and help me out here: how do you determine what the most important part of the game is?

A: Well, if you want to be technical about it, you can go and look at the WPA graphs on Fangraphs (they’re the graphs the RABbis post in the recap after every game with that line that goes ziggy zag zig wheee or zag zig zag oof, and the bars underneath, where the higher the bar, the more crucial that play was in that game).

However, if anything remotely math-y or graph-y rubs you the wrong way, the most important points of the game are pretty much any point in which the lead is in danger of changing from one team to the other. Like on Saturday, the Yankees had the bases loaded with no one out, that was a high leverage situation, because the likelihood that Toronto’s 2-0 lead would go bye-bye became much higher than it would be had the bases been empty and there been two out.

Really, most of the time it’s pretty easy to tell when it’s a high leverage situation–the crowd will get loud because their team is rallying or groan because Joba’s busy imploding again, for example.

That said, here’s the key thing, the thing so important that I’m bolding it: the importance of the ninth inning is nil if you can’t get through the eighth, and the eighth is not important if you can’t get through the seventh.

Q: Okay, I see your point, but you can’t have Mariano pitch the seventh, I mean, what about the saves?

A: Saves are a stupid stat. Did you know that in the game Texas won in 2007 30-3 against Baltimore, they got a save because the same guy, who wasn’t the starter, pitched the final three innings?

Q: But still! I mean, you can’t have Mariano pitch the seventh, eighth and ninth, the dude’s bloody forty years old!

A: Understood. I didn’t say the leverage argument wasn’t without risk. If the most critical point of the game happens in the seventh inning–like it did in Game 5 of the 2009 ALCS, then the leverage argument says you have Rivera pitch the seventh. It means that Joba or Robertson or pitcher X pitches the eighth and the ninth, but I revert back to the bolded statement above: the eighth and ninth won’t matter (as much) if you blow the lead in the seventh inning. The idea is that you get the lead, and keep it: walk offs and dramatic home runs are awesome and everything, but to be in that situation means that you’re relying on an awful lot of pure luck.

Q: Okay, so the Yankees are up 1-0 in the seventh in Game Seven of the World Series and the Dodgers are sending up Russell Martin, Blake DeWitt and Clayton Kershaw. The Yankees should go to Mo?

A: Well, not necessarily. Those three, right now, are the bottom of the Dodger lineup (and we presume this game is at Dodger Stadium). The situation is important, but if you’re winning 1-0, and it’s game seven, chances are CC is busy being CC and cruising. You know that in the eighth inning the Dodgers have Furcal, Eithier and Kemp due up (theoretically much better hitters than the bottom three), and thus that is perhaps more likely to be the more crucial situation.

That said, once Martin or DeWitt reach base, the complexion changes because then, barring ye olde GIDP, Furcal or Eithier will bat in that inning. That’s when you ring the phone and tell Mariano to start tossing.

Q: But what if DeWitt hits a home run?

A: It’s baseball, stuff happens. The leverage argument is about playing probabilities, and doing your best to use those probabilities to your advantage.

Q: That sounds much too much like sabermetrics.

A: Well, it is. That said, you don’t need statistics to tell you that Albert Pujols is more likely to hit Chan Ho(me run) Park than Mariano Rivera or that Chad Gaudin will probably find Juan Pierre and easier out than, say, Manny Ramirez.

Q: Okay, I get you, but this still seems really risky. What if Joe Girardi did go to Mariano in the eighth on Saturday and then Joba blew the lead in the ninth?

A: Alas, it’s one of the risks you’re going to run. That said, think about it like this: would you prefer to be tied 1-1 going into the ninth, or have a 1-0 lead going into the ninth?

The real point, here, however, is what happens when your team is on the road in extra innings. Most managers (still) won’t use their closer on the road in extras, saving them for save opportunities that may never actually materialize. Yet, us Yankee fans got lucky on the last Yankee road trip: two extra inning games preceding off days, Joe Girardi elected to use Mariano Rivera to pitch the bottom of the ninth, instead of other relievers he had at his disposal. The Yankees were lucky enough to score runs in the tenth inning both times, and because the off day allowed him to do so, Girardi went back to Mariano for the tenth inning.

The Yankees won both games.

Q: All right, you got me hooked. Where should I go?

A: Fangraphs and their WPA charts are my favorite, but you may also find Baseball Reference or the Leverage Indexes posted at The Hardball Time useful as well. Mind, these guys get very into it, actually calculating numbers to determine which situations (ie, how many men on base, how many out, the inning, etc) are more critical than others. It’s a baseball stat nerd’s dream, but you don’t need to get that complicated to understand the point.

Q: So what’s the point?

A: The ninth (or even the eighth) inning is not necessarily the most important inning of the game, and boxing in relievers as “the eighth inning guy” or “the closer” without allowing for an occasional adjustment as circumstances warrant can come back to bite you in the ass. Almost everything about a bullpen and the game of baseball is malleable; use it to your advantage.

Q: You’re still mad at Joba, aren’t you?

A: Yes.

email
Fan Confidence Poll: July 5th, 2010
Halfway through, a status report
  • gc

    I understand every single argument that’s being made, but the realist in me says that there is no way any manager or team will ever decide to do things other than the way they are currently doing them. Right or wrong, that’s just how I feel. The one time the “closer” is brought in to get those two or three most crucial outs in the seventh inning of a big time game, and then another lesser pitcher coughs up the lead (and the game) in the ninth, the media and the fans would go *beyond* ballistic. Even if it was the right call at the time and “it’s baseball and stuff happens.” Realistically speaking, all the shouting from the rooftops will most likely fall on deaf ears on this one. I don’t see things changing. Just a lot of sabermetricians arguing amongst themselves. Yeah, the arguments are worth having, and maybe someday somebody will be brave enough to institute it as policy, but my guess is he won’t have a job for long when “stuff happens” and the fans and media call for their heads in outrage.

    • http://soxandpinstripes.net Angelo

      Also, in addition to this statement, it is much easier to conform (do what everybody else is doing) than it is to be bold and take your own approach.

      Which is part of the reason why the media usually agrees on things that may seem stupid. Conforming is a part of human nature, sadly sometimes.

  • http://MattOnEarth.blogspot.com Matt on Earth

    I found this entry to be particularly interesting. The whole “leverage argument” seems so intuitive in a way. Basically the concept suggests winning the game as soon as one can by mitigating the most dangerous situations with the teams best options, rather than pegging certain players into particular roles (which may or may not ever become relevant).

    Yet, in the world of baseball, it’s not surprising that this mode of thinking causes many people to cringe. The idea of Mo not closing, or Joba not setting up Mo, seems to cause many fans immediate angst. Baseball has enforced the idea that the further into the game a reliever is called upon, the more valuable he must be. By putting Mo in the 8th inning, somehow his value becomes diminished (especially if he doesn’t pitch the 9th as well). Of course, how good is Mo if Joba blows a 1 or 2 run lead and leaves the team trailing?

  • Kevin Ocala, Fl

    Great mental exercise. But, how is the manager supposed to know when these “high leverage situation” are coming? Relievers need some time to warm-up. The alternative is to burn the relievers out by warming them up too often…

    • http://www.lennysyankees.com lenNY’s Yankees

      Looking at the batting order usually helps… If 2-3-4 are due up in the eighth, it’s bound to be a higher leverage situation than the ninth.

  • http://www.tynansanger.com Ethan Stanislawski

    Even as a guy who loves what Sabermetrics has brought to understanding the game, I have difficulty justifying the leverage argument in practice as opposed to theory. I do think that at the field level, players need more clearly defined roles to be as effective. This is trickier in the bullpen, but it’s probably why no “closer by committee” system has worked. I’m not complaining (thanks 2003 ALCS!), but I do think that if you started putting Mo in the 7th, 8th or 9th, depending on the situation, he probably wouldn’t be as effective. If for nothing else in that it would remove his robotic consistency.

    • 24fan

      Another problem is that Mo can’t pitch in every game. So say you use Mo in the 7th of a 2-1 game, and in the bottom of the 7th the Yanks offense goes off and you are leading 10-1 going into the 8th. Then you have effectively burned on of Mo’s appearances in a game where the Yanks blew out the opponent. Not to say the leverage argument isn’t right on many points, but there are a lot of practical issues that make it questionable, or at least difficult to really know that it will work practically.

      • ROBTEN

        Another problem is that Mo can’t pitch in every game. So say you use Mo in the 7th of a 2-1 game, and in the bottom of the 7th the Yanks offense goes off and you are leading 10-1 going into the 8th. Then you have effectively burned on of Mo’s appearances in a game where the Yanks blew out the opponent.

        The problem with this example is that it works the other way as well. Say the Yankees blow a series of games in the seventh or eighth inning, and thus Mo doesn’t pitch at all because you’re saving him for an all-important time that never arrives. We’ve all seen stretches where Mo doesn’t pitch for a week or more and ends up pitching in a less meaningful situation simply because he needs work.

        It is also equally plausible, since we are speaking hypothetically, to imagine a scenario in which Mo entering a game that’s 2-1 in the seventh is what enables them to go on and blow out an opponent because they kept the game 2-1 when it mattered (i.e. when the opposing teams best hitters were coming up).

    • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

      Even as a guy who loves what Sabermetrics has brought to understanding the game, I have difficulty justifying the leverage argument in practice as opposed to theory. I do think that at the field level, players need more clearly defined roles to be as effective.

      I’ll agree, and here’s my retort:

      You can still abandon the rote “7th inning guy-8th inning guy-9th inning closer” set role pattern by instead going to the “#1 fireman-#2 fireman-#3 fireman” set role pattern.

      I.E., Mo’s the #1 fireman. He knows he’s going to go in there when the game is most on the line. Joba’s the #2 fireman, he knows he’s going to back up Mo and pitch right after Mo puts out the fire to keep it put out (or to handle the lower-level fire in the 7th inning that’s high leverage but low batter skill, like the Martin-DeWitt-Kershaw situation described above). Robertson is the #3 fireman.

      They know their roles and they know when they pitch. They know their times to pitch are determined by leverage situation, however, rather than by inning time.

    • ROBTEN

      This is trickier in the bullpen, but it’s probably why no “closer by committee” system has worked.

      I tend to think that the “closer by committee” has generally not worked because most of the time managers are trying to use lesser pitchers to piece together a bullpen. In other words, insofar as teams are generally wedded to the save stat and tend to put their best non-starting pitcher in that role, they go to the “closer by committee” approach not to try something new, but because they are trying to deal with the absence of any strong bullpen arms. In this scenario, they could name any one of the pitchers “the” closer and it probably wouldn’t make much difference.

      • http://www.tynansanger.com Ethan Stanislawski

        ROBTEN That’s a decent argument. When Nathan went down for the Twins earlier this year there was a lot of “closer by committee” talk, but they went with Rauch because he had “9th inning experience.” He’s kept that role, but like the rest of the Twins bullpen, he’s been good enough that there’s no reason to change the status quo. The ’03 Sox, who invented the term, had a much worse bullpen, and their guy with the most “9th inning experience” was…Byung-Hyun Kim. Scott Williamson was doing the closer duties by the ALCS, but it was probably the confusion/distrust of the bullpen that kept Pedro in the game.

        tommiesmith, again, that may be the way to do things. If that’s the way things are 20 years from now, I owe you a Coke.

  • Randy V Larde

    Can’t really see your point here. Or at least not see it as being very valid. Sure it’s easy to be reactionary after the fact and say that if Mariano had come in the seventh instead of Chan Ho Park the Yankees would have won a game but the problem is that if you were to do that everything else that followed would then also change. All of sudden situations would follow in the eight and ninth innings and Chan Ho Park would come in to blow the lead and people would be bitching about that.

    The analysis also fails to consider that all innings are not statistically equal. Simply put a deficit in the first inning is not as important as a deficit in later innings because a team has more chances to make it up and accordingly blowing a lead in the seventh isn’t as big a deal as blowing a lead in the eight and blowing a lead in the eight is not as important as blowing a lead in the ninth. That is why every major league team brings their closer in in the ninth inning. It is by definition the most important inning because after it there are no more chances unless we go extras and that’s a whole other story.

    • http://www.puristbleedspinstripes.com Rebecca-Optimist Prime (Optimovelist Primus)

      Answer me this:

      Say you’re playing the Red Sox in the ALCS, Game Seven. You have a 1-0 lead.

      In the seventh inning, the Red Sox have Youkilis, Ortiz and Drew due up; in the ninth inning it’s Nava, McDonald and Ellsbury.

      Who do you want Mariano facing?

      • Little Bill

        I want Mariano pitching in the 9th because he’s the closer and that’s his role. Joba’s the 8th inning guy and Robertson/Marte are the 6th/7th inning guys. If you deviate from this plan you’re just asking too much from Joe Girardi.

        • pete

          Sarcasm?

        • http://www.lennysyankees.com lenNY’s Yankees

          LOLz

      • Randy V Larde

        First point is that when you’re in the seventh inning you can guess who will be up in the ninth inning but you really have no idea.

        You’re argument seems built on the idea that a team needs to hold on for dear life to get a W. What if in the bottom of the 7th the Yankees were to score five runs? Now you’ve used Mariano for nothing.

        Now granted in a short series this may not be that big of a deal. But to do this for a whole season? Not too hard to see that it’s not exactly a brilliant strategy.

        • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

          What if in the bottom of the 7th the Yankees were to score five runs? Now you’ve used Mariano for nothing.

          That doesn’t make sense. You used Mariano to keep the lead close during the pivotal situation so that your five runs scored post-Mariano made the lead wide enough that you can throw inferior pitchers in the 8th and 9th.

          • gc

            And then what if your lesser pitchers cough up the lead and you end up tied again in the 9th? A whole lotta what if’s going on. Too many, IMO, for us to see any significant changes in the way teams and managers do things in the foreseeable future.

            • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

              Life is a series of “what ifs”. The presence of more potential “what ifs” from a change in strategy is never a good reason to keep doing something in a non-logical or inefficient way.

              • gc

                That’s great. And as I said above, I don’t necessarily disagree with you. Now let’s you and I sit here and wait for things in the sport to actually change. Wake me when it happens, because I fear it will be a long long time.

                • poster on a different computer

                  This comment had no point to it.

        • pete

          this is just so so so so so so wrong. If the yankees score 8 runs in the bottom of the 7th, who gives a shit that Mo’s unavailable?

          • Ran DV Larde

            I’d give a shit. Especially if it’s the third straight day he’s pitched and would be unavailable the next day.

            What people should understand here is that baseball is a season. The manager’s challenge is to use his closer both as much AND as little as he can.

            Mariano wouldn’t still be notching saves in his forties if he was being used like Scott Proctor in this thirties.

            • ROBTEN

              But, why would this mean he would be pitching every day? If the Yankees are up 2-1 in the ninth for four straight games, they wouldn’t use Mo this often and we would still end up seeing someone else close the game.

              In other words, even if we assume that the ninth is the most important moment in a game, it still doesn’t mean that Mariano can pitch it every time either.

              • Ran DV Larde

                He wouldn’t be pitching every day but he would be certainly pitching much more.

                I’d also argue that there is something to be gained by letting your “lesser” pitchers pitch in high leverage situations. It’s how you build a bullpen.

                In the playoffs high leverage situations come in all innings. You aren’t going to get by with one guy. We tried that already and it didn’t work.

              • pete

                that. I’m not suggesting Mo be used any more than he is, just that he be used better. The goal, contrary to what Ran seems to think, is not to use Mariano as little as possible. It is to use him as much as possible without overusing him, and it is to deploy his talents towards optimal results. And those results should be team wins, not Mariano saves.

      • YankeeJosh

        Obviously I’d want Mo facing the better batters. But there is a long time between the 7th and 9th innings. Let’s say for example Mo gets Youkilis, Ortiz and Drew in the 7th, then Joba has a rocky 8th, and Youkilis, Ortiz and Drew are up again in the 9th. Now you are screwed because you burned Mo already.

        I think there can be a compromise. Have a 9th inning closer and then use the rest of the bullpen based on leverage situations.

        I think your argument works better when it’s 8th inning vs 9th inning, when the leverage situation is a little clearer. The problem is that a manager can’t predict when high leverage situations will arise, and you don’t want to burn your best too quickly.

    • http://www.nomaas.org K.B.D.

      [b]It is by definition the most important inning because after it there are no more chances unless we go extras and that’s a whole other story.[/b]

      Not really. By that definition, it is the last inning. And last does not necessarily mean “most important.”

      • Ran DV Larde

        Your lost in the woods. The temperature is dropping below freezing. You have nine matches to start a fire and save your life. Question– which match is the most important?

        • http://www.nomaas.org K.B.D.

          I busted out my “Surviving the Wild: Life Percentage Added” manual for this one. The fifth.

          But seriously, this isn’t a useful situation to compare to a baseball game.

        • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

          Question– which match is the most important?

          Answer: The one that starts the fire, irrespective of what order it is chosen. If you try the first eight matches and none of them light, and then you try the ninth one at it also doesn’t light, then none of them were important. If you try three matches and the third one lights it, then matches 4 through 9 are all less important than match #3.

          This analogy sucks.

          • Ran DV Larde

            It doesn’t suck if you’re the one lighting the match.

            Compare the amount of pressure from trying to light the first one to trying to light the ninth one after the first eight have failed.

            • http://www.nomaas.org K.B.D.

              Who was arguing the pressure of the ninth inning isn’t high?

            • pete

              you’re confusing “pressure” with significance. Pressure can, of course, be caused by leverage, but it can also be caused by potential backlashed caused by potential failure, which should not have any bearing on the way the game is played.

              Of course, it will, because if a manager puts Mo in in the 7th and somebody blows the game in the 9th, it won’t matter to the fans and media that the team has been much more successful over the course of the season because of its strategies, it will only matter that this move led directly to a frustrating loss. And a frustrating loss is, to the fans and media, worse than an unfrustrating (or less frustrating) one. Which is, of course, bullshit.

            • pete

              and it’s the same bad logic if you’re lighting the match or not – if the first 8 don’t light, and the 9th doesn’t light, then, as tsjc said, the 9th is just as insignificant as the others. If the 5th match lights, however, then it’s your most important match.

    • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

      That is why every major league team brings their closer in in the ninth inning. It is by definition the most important inning because after it there are no more chances unless we go extras and that’s a whole other story.

      Incorrect.

      The ninth inning is by definition the LAST inning (usually). It is not by definition the “most important” inning. “Most important” is a very subjective term. The last inning can be the most important, and in many scenarios is the most important, but it is not ALWAYS the most important, because that concept of “importance” can be defined in a host of different ways.

      That’s the entire point of leverage analysis. Looking at the historical record of the millions of past baseball games we can see what game situations are the most pivotal, where a probable victory turns into a probable defeat (and vice versa) or where the turn doesn’t happen and a lead becomes virtually unsurmountable. That moment of potential change or non-change can be argued to be the most important moment, and it often does not occur at the end.

      It’s like studying the great wars of history. The final battle of a war is always important because it envolves the capitulation and declaration of winners/losers, but more often than not, the battle that actually crystallized who was going to be the eventual winner and eventual loser happened weeks/months/years before the final battle. The die was cast; the winner and loser had reached points of no return where only an unprecedented miracle or unpredictable collapse could possibly alter the predicted final outcome.

      • Ran DV Larde

        It’s nothing like studying the great wars of history it’s a baseball game.

        What you have here is a problem- Baseball teams lose games with their best pitchers not on the mound.

        Followed by a simplistic solution- Let’s always have our best pitcher on the mound.

        And a hypothesis – That the problem would than be solved

        What you don’t seem to account for is the possibility that this temporary solution (the games not over after all) could also bring about a whole slew of other problems both for the individual game, the individual player and the entire season that would outweigh the benefits of the solution.

        • Jim S

          My response if you’re going to ignore TSJC’s analogy: It’s nothing like lighting matches, it’s a baseball game. Even as an observer it seems like you’re losing this argument because you can’t admit that being wrong is a possibility.

          • Ran DV Larde

            Well, it would be pretty hard to be wrong in this instance considering the argument is that hypothetically doing things a completely different way would be more effective than a system that for the most part is pretty effective as it is.

            • Jim S

              You ignored the analogy and refuse to accept definitions of words; instead falling back on “this is the way it’s done. that’s good enough”

              • Ran DV Larde

                absolutely not. i’ve made it clear that i believe over the course of a long season i think using your closer for the most part in the 9th inning and developing a bullpen to cover the load before that is the best way to go.

                it’s not an argument without merit. apparently every baseball manager and general manager agree with me.

  • http://mystiqueandaura.com Steve H

    As long as Joe Maddon is managing there is hope for someone to go against the grain and consider leverage as opposed to “the way things are done.”

    • gc

      Until he goes against the grain in a big game and it costs the Rays in a big way. When it comes time for heads to roll, as is commonly the case when such losses happen, it sure makes it easier for those who sign the checks to cut that manager loose. Especially when fan and media outrage come to a boil, which is also commonly the case as well.

  • Randy V Larde

    And as long as Joe Torre is managing we’ll see a guy who will leverage his best relievers in day after day after day only to wind up in September without a bullpen.

    • Yankeegirl49

      IETC

      • Ran DV Larde

        IETC means???

  • http://www.soxandpinstripes.net JGS

    Whenever I think about this, I always come back to this Posnanski piece. The key sentence is this:

    “It’s so important for them to be considered professional,” he [Bill James] says, “that they are unwilling to try anything that might make people think they’re amateurish.”

    Using your closer to close games is the professional thing to do–that’s why they call him the closer. What you need is a manager who doesn’t have to worry about what other people think of him anymore–a Tony La Russa, or even better, a Bobby Cox, especially in his final season (Joe Torre could do it too, but I wouldn’t trust his innovating with a bullpen)–to start using relievers smartly, the save stat be damned.

    The other problem is that saves are where the money is for relievers, and until that changes, nothing else will.

    • http://mystiqueandaura.com Steve H

      What you need is a manager who doesn’t have to worry about what other people think of him anymore–a Tony La Russa

      I agree, and the funny thing about this of course is that LaRussa is the one who “invented” the way bullpens are currently used. The Eck was truly the first strictly one inning closer, and that one inning was always the 9th. I’d love to see one of these guys just think about going against the grain a bit, but I’m not holding out much hope. Even if you just cut it to the 8th and 9th innings, if you had 3-4-5 coming up in the 8th with a 1 run lead, doesn’t it make sense to go to your best reliever against their best hitters?

      • http://www.soxandpinstripes.net JGS

        Here is the trouble–if a couple of them get on, then the 7 hitter makes the last out, and you get 8-9-1 in the ninth and you have used your best reliever. If your second best reliever then comes in and doesn’t shut the door immediately, you will have the best hitters coming up again, with runners on base this time, and your best reliever is out of the game.

        • http://mystiqueandaura.com Steve H

          Like K.B.D. and Tommie said below you can’t really tell when the highest leverage spot is until afterwards, but if you’re putting guys in the game, you’re assuming to get outs, so your best case scenario is to have your best reliever go against their best hitters, if that’s how it currently lines up. While getting those outs is no guarantee, the odds of getting the 3-4-5 hitters out are better if Mo is on the mound as opposed to anyone else. It’s certainly a slippery slope, and your argument is the reason that it likely won’t ever happen, that they manager would be too worried about getting skewered for making that decision, and then having it backfire.

    • Brien Jackson

      No, what you’d need is an innovative front office who understood the concept and were willing to go along with it. If the guy who signs your paycheck will back you, it doesn’t matter what the fans and media say.

  • Pasqua

    To an extent (and maybe I’m missing something here) isn’t it impossible for a manager to identify the most “critical” moment in a game until the game is over? In other words, while a tense / problematic situation might arise in the 7th inning, who’s to say that an even more “critical” moment won’t arise in the 8th? I love the concept of the leverage argument, like Ethan above, I’m not sure of the practicality.

    I suppose the the role of the 9th inning closer has developed, in part, due to the fact that the 9th is the 9th. There will be no more “critical” innings in the game, so long as the closer does his job. Therefore, it’ smuch easier for the manager to justify using his best pitcher there.

    My head hurts a little bit.

    • http://www.nomaas.org K.B.D.

      It’s impossible to identify the most critical moment until the game is over. That doesn’t mean you can’t identify a high-leverage situation. They’re pretty easy to spot.

      • http://twitter.com/tsjc68 tommiesmithjohncarlos a/k/a Ridiculous Upside the Elder

        That. There’s no guarantee that there won’t be an eventual higher leverage situation later in the game, but there is a guarantee that the situation you’re in is a really damn high leverage situation and if you get out of it, you’ve dodged a big bullet and your offense has an opportunity to score runs and make the next potential bullet smaller (and likely against weaker batters, to boot).

    • pete

      It’s a definite risk, and it’s why such use would have to be intelligent. If you cough up a lead in the 6th inning, you might have the top of your lineup hit twice more during the game. You might have a decent chance, in other words, of regaining the lead, making your loss of said lead insignificant.

      On the other hand, if you bring in your top reliever here and then proceed to score several more runs, not having that reliever available in the late innings would be equally insignificant. However, by bringing in the top reliever in this situation, you eliminate him for future use while the opponent gets several more chances to score runs.

      I believe, and Rebecca intimated this in the article, that the risk of leverage-based use of relievers prior to the 7th inning outweighs its potential benefits, because it gives the opponent a full 3 innings in which to re-rally. In 3 innings and against relievers other than the best reliever on the team (and likely other than the 2nd best until either a rally has begun or it is the 9th inning), most offenses would stand a good chance of generating more high-leverage situations (higher actually, because in their proximity to the end of the game they render a higher correlation to its outcome) in innings 7-9, during which “fireman #1″ would be unavailable.

      What’s more, the leverage at the beginning of an inning is never particularly high, so it’d probably be silly to have your best reliever start an inning other than the 9th unless the expected batters in that inning are like Bonds-Pujols-Ruth or something like that. Thus the earliest the fireman could come in would be the 7th inning, and in many or most of those situations, there would already be an out or two when he came in.

      If, for instance, you brought in Mo with 2 outs in the 7th and he quickly retired the last hitter, he’d likely be able to pitch the whole 8th inning, meaning you’d only need to get one inning out of a lower-caliber pitcher.

      I also think it’s worth noting what type of pitcher you’re talking about. In an extreme example, let’s say we have 2007-reliever-Joba, and standard Mariano. I would, in this instance, nominate Joba as fireman #1, since his ridiculous strikeout and contact rates would give him a higher likelihood of escaping jams via the K, rather than Mariano, who is semi-prone to bloop singles.

      In other words, the leverage theory could never be properly implemented by any manager who wasn’t intelligent and thorough in his thought processes. But there shouldn’t be managers who aren’t intelligent and thorough in their thought processes, even if there are.

      • Pasqua

        Really well-articulated. I appreciate it.

  • Joe

    Understand argument but it will never happen in the MLB.

  • http://theyankeeu.com Matt Imbrogno

    Predictably, I love this thinking and wish it was used more often in Major League Baseball. However, I have my doubts about its implementation. It does, also predictably, have to do with just that–predictability. Managers love routines and the implementation of this strategy requires the breaking of a routine. The other road block is the current situation of reliever usage. This strategy would likely require at least two guys in every bullpen who could go at least two innings, and not a lot of teams have that. If there is a reversion to the bullpen strategy of about 30 years ago, this could work wonderfully. In that sort of situation, there is a multi-pronged effect:

    1. It uses the best relievers in the best situations.
    2. It could cut down on the total number of relievers used, giving teams more leeway in teetering with the bench.
    3. It could bridge the “B-Jobber Gap” and give young prospect pitchers away to a) face Major League hitters and b) build up innings. I’m not saying these guys would be the A1 firemen, but they could at least pick up multiple innings more easily than if they were one inning/set up relievers.

    • Chris

      Managers love routines and the implementation of this strategy requires the breaking of a routine.

      It’s not just managers – it’s players as well. For example, Mo benefits from the fact that he knows his routine for basically every game – he stretches in the 6th, light tosses in the 7th, and warms up in the 8th. Based on how superstitious many baseball players are, a change in routine could have a pretty big impact.

      • http://theyankeeu.com Matt Imbrogno

        These dudes are really good athletes, right? I’m sure that they could physically handle the change in routine. As for the mental part, they’re obviously mentally tough if they’ve made it this far as ballplayers. There might be an adjustment period, but it’d be worth it.

        • Chris

          There might be an adjustment period, but it’d be worth it.

          Really? What if it take two years of struggles to get reliever accustomed to their new roles? Is that worth it? And then you would have to go through a similar transition every time you tried to bring in a new reliever from another organization.

          All of these headaches for 1-2 extra wins in a season. I don’t think it’s clear cut that it would be worth it.

          • http://mystiqueandaura.com Steve H

            If it takes two years of struggles to have success in a different inning than a pitcher is accustomed to, the pitcher sucks anyway.

            • http://theyankeeu.com Matt Imbrogno

              That.

            • Chris

              No. Just because someone struggles when you change their routine doesn’t mean they suck. And I’m not necessarily referring to one reliever – I’m talking about the whole bullpen (including manager, bullpen coach, etc who have to manage them). It’s not easy to completely remodel the way a bullpen is handled, and there are going to be growing pains. Is it worth enduring those pains for the next couple seasons just so down the road you can (possibly) have a better strategy?

              I’m a huge fan of sabermetrics, but they often dismiss the fact that these are people. And being people they can be influenced by (possibly irrational) mental factors. Just look at guys like Saltalamachia and Ankiel. They’re extreme cases, but if there are extreme cases like thay, then why isn’t it possible that there are more modest cases – like a guy struggling because his role in the bullpen was changed?

          • Jim S

            Seriously? You wouldn’t take headaches for 1-2 extra wins in a division like the AL East?

            • Chris

              Would I trade headaches and fewer wins for the next 1-2 seasons so that after getting through it we would have 1-2 more wins a season three years down the line?

              No.

  • Chris

    The real point, here, however, is what happens when your team is on the road in extra innings. Most managers (still) won’t use their closer on the road in extras, saving them for save opportunities that may never actually materialize.

    The problem with this argument is that it only makes sense if you’re willing to use your closer for 2 full innings. There’s no benefit to bringing in you closer to pitch the 9th if you’re then going to make a change for the 10th. In most cases, it’s probably reasonable to expect your closer to work two innings (particularly the playoffs), but depending on their prior work load, upcoming off days, etc, it just may not be practical.

    • http://theyankeeu.com Matt Imbrogno

      There’s no benefit to bringing in you closer to pitch the 9th if you’re then going to make a change for the 10th

      If the other team’s 3-4-5 hitters are coming up, I want my closer in there.

      • Chris

        If it’s a game in June, then I’m ok with letting the lesser relievers try to handle it. Among other things, using them in tight situations can help build up their confidence and give the manager a better idea of how they respond to pressure situations.

    • ROBTEN

      Not necessarily, as this still assumes that the most important three outs are the last. What Rebecca is pointing out is how often teams lose simply because they are saving their closer for a moment that never arrives.

  • pete

    I don’t get the whole “it will never happen” thing – if it’s the right plan, it’ll happen eventually.

    • gc

      Don’t count on it. If and until the fans and media get on board, and right now the overwhelming lot of them could care less about any of this, it will be generationS (plural) before we see any significant change.

  • Mike HC

    Nice, fun little article.

    I agree that your best reliever should be used in the most important situations. As you laid out, if its the 7th or 8th, and the teams best three hitters are coming up, or there are a couple of men on base, I really don’t see why you don’t bring in your best guy. Then use your second best guy to close out the game. I see absolutely no reason why that couldn’t work just as well, if not better than doing it the other way around.

    You would think that your best reliever would be able to withstand the pressure of being called on at any moment to handle any type of situation, where lesser relievers could be better knowing exactly when they will be needed. I know there is added pressure in the ninth, but you should be able to find at least one other reliever who can handle the that pressure.

  • pete

    The only part of this to which I think there are valid objections is pitcher warmup routines. I do think all pitchers pitch most effectively when they get ample time to warm up, which they can’t be given under these circumstances.

    • http://mystiqueandaura.com Steve H

      If you were to limit the certain leverage situations simply to beginning of inning scenarios you would be able to manage the warm ups, but you’d also then be ignoring other potential high (or higher) leverage situations where guys got on base in the 7th or 8th.

    • Mike HC

      I don’t really get this. The argument is that instead of bringing in a lesser reliever just because it is the 7th or 8th, you bring in your best guy. Meaning, someone had to warm up anyway. Its not changing warmup routines in any way, it just changes who enters the game.

  • http://mystiqueandaura.com Steve H

    The funny thing about this being on a Yankee site is that Yankee fans have a different perspective as we’ve enjoyed Mo for almost 2 decades. There is such a big difference between Mo, year in and year out, and every other closer. So while using your best pitcher in higher leverage situations in the 7th and 8th makes sense, it makes even more sense if the dropoff to your 2nd and 3rd best relievers isn’t vast. In the Yankees case it is, so I think we’d have a different take on it for the most part.

  • the209

    Sorry, stupid question here: is there an example of an actual college / semi-pro / professional team EVER using this system?

    • Mike HC

      Some teams have kinda done it. When Zumaya was setting up for Todd Jones, Zumaya was used as the set up/get out of trouble guy and was far better than Jones.

      Even the White Sox this year are kinda doing it. Jenks is not their best reliever and they use him the ninth and their best guys in the earlier innings.

      • Mike HC

        The Yanks did it in 1996 with Mo pitching as the leverage guy and Wetteland, the far inferior pitcher, as the ninth guy.

        It does happen in some form.

        • Chris

          I don’ think ’96 really counts. Mo was slightly better than Wetteland that year (2.82 ERA vs 2.09, FIP was more divergent because Wetteland was homer prone), but he was coming off a season where he had a 4.24 ERA as a reliever – and that included a .095 BABIP. After Mo proved himself, he was moved to the closer role in 1997.

          • Mike HC

            Right, but just saying it was done, for whatever reason.

      • http://www.nomaas.org K.B.D.

        Jenks is not their best reliever and they use him the ninth and their best guys in the earlier innings.

        His name is Matt Thornton. He is awesome.

      • The Ghost of Phil Rizzuto

        Boy you guys either have short memories or you’re all 20 years old. The fireman concept was how ALL relief pitching was handled pre-Tony LaRussa (boy do I hate that guy). Ever heard of Sparky Lyle? How about Rich Gossage? Those guys would often come in before the ninth, often with men on base and often pitch more than one inning.

        The chief motivation for the current 7th-inning guy, 8th-inning guy, closer is the save rule – one of the dumbest, most useless statistics in baseball. All of the mythology about how the 9th inning is different and all the other ridiculous bullshit you hear about relief pitching has been invented by idiot sportswriters and managers and GMs who are too chickenshit to buck the trend.

        If you have Mariano Rivera the strategy works (most of the time) provided the “setup guys” are semi-reliable. Most other teams would benefit from using the fireman strategy. Building your relief pitching strategy around the assumption that you have the equivalent of Mo in your bullpen is the equivalent of building your lineup strategy around the assumption that your lead off hitter is as good as Rickey Henderson and your cleanup guy is as good as Mickey Mantle.

  • Ran DV Larde

    Like I said above the bigger point to realize here is that baseball is not just a game it’s a season. A manager’s challenge is to use his closer both as much AND as little as possible.

    Mariano wouldn’t be notching saves in his forties if he was being used like Scott Proctor and Tanyon Sturtze in his thirties.

    When you think about it, it’s actually the use of the closer in the 9th inning that’s counter intuitive. It’s much easier to understand the red meat idea to bring a guy in whenever and not have to think through all the consequences that could occur innings, days, months or even years later.

    • Mike HC

      I don’t see how usage would have to change. You can manage his innings just the same as using him in the most important situations and using him when their is a “save” chance. If anything, you could probably control his usage even better as the leverage guy.

    • http://theyankeeu.com Matt Imbrogno

      The only thing in this scenario that would equate Rivera and those other guys is that he could potentially be used during the same innings as those guys. That doesn’t mean he’d be used every day or asked to throw multiple innings every time, just that he’d be used earlier.

  • http://theyankeeu.com Matt Imbrogno

    I’m playing FIFA ’10 and the announcer just said that C. Tevez of Argentina has “bulldog” attitude. Maybe we should sign him for the ‘pen.

  • http://www.lennysyankees.com lenNY’s Yankees

    GREAT article, Rebecca.

  • Chip

    I agree with the overall idea of using your best relievers in the best spot but this is what would inevitably happen. Say CC is pitching into the 7th of a 2-0 game against the Red Sox, they get two men on for Big Pop-up. Now Girardi notices that CC is almost done and suddenly it’s a high leverage situation. Alright, well time to get Mo up because it’s Youk, Victor and Beltre coming up. But Mo needs some time to warm up so you go to the mound, blah blah blah, throw to first a few times and then Ortiz pops up on the first pitch. Well then Youk comes up and grounds into a double play. You’ve just dry humped Rivera without actually using him in the game. Now he might come in the 8th, maybe let one guy on. But then in the ninth, Joba lets a runner on and suddenly you’re facing Ellsbury, Pedroia and Ortiz.

    Now that might not be the best example but my point is that this is a great idea if you can insert a great reliever the second a huge leverage situation comes up but you don’t have time to get your closer up and ready before that situation goes away.

    The reason I bring this up is both that relievers need time to get ready and guys in the bullpen usually like to know about when they’ll be coming in. I think it was Robertson last year or the year before who said as much. I do have one question, what inning has the highest occurrence of high-leverage situations? If that answer is the ninth (which it probably is) then you line your best pitcher up to pitch those innings

    • Brien Jackson

      I would think you’d have a pretty good idea you might need to get him ready if your starter is getting deep into the game with the heart of the order looming.

  • the209

    Thanks…This is a nice breakdown of Bill James’ work for people like me.

    While everything he/you say makes sense theoretically, I think it’s nothing more than that: a nice theory.

    It makes sense that the critical moments can occur in the 7th, 8th, wherever — BUT … we get that the critical moment isn’t the 9th — the relievers are getting paid to make sure it doesn’t become critical.

    And to say that “we have millions of games of data” (paraphrasing other commenters here) to back up the switch is to compare apples with oranges. You have “millions of games of data USING THE CURRENT SYSTEM” — you have no idea how 1, 5 or 50 years of data of using this leverage system will look like… you might find data that supports a different system entirely.

    • The Ghost of Phil Rizzuto

      you have no idea how 1, 5 or 50 years of data of using this leverage system will look like…

      Actually we do – all relief pitching prior to Tony LaRussa.

      History didn’t begin when you started paying attention pal.

      • Evan3457

        The historical record is mixed at best. For the most part, even before LaRussa, closers still closed.

        It’s just that many teams didn’t have a closer as we think of it, but they just closed from the 7th inning on, instead of just the 9th inning. Alexander came in in the 7th to get Lazzeri. Hornsby didn’t go to Reinhart or Bell for the 8th; Alexander finished it out.

        Sparky got the Yankees out of the 4th inning in game 4 in 1977 against the Royals with a 5-4 lead. Martin had already used Tidrow, he didn’t go to Clay, or Holtzman, or Hunter. Now, Billy did take Torrez out after he had gotten out of the 3rd for Guidry, and thrown four more innings, but 1) Torrez had just started and gone 5 2/3 two days earlier, and 2) Sparky was waiting to give him the 8th and 9th. (By the way, Lyle went 2 1/3 in game 3, 5 1/3 the next day in game 4, and the last 2 in game 5 the day after that. The Yanks used only 6 pitchers in that entire ALCS, Hunter, Holtzman, Clay…none of those 3 were used at all in the series.)

        Casey used everyone all the time, whoever he needed at the moment. Even so, of Allie Reynolds 86 relief appearances under Casey, he finished the game 70 times. Raschi finished 5 of his 10. Sain finished 73 of his 85.

        This isn’t exhaustive of the historical record, of course, but just 3 different examples from Yankees’ history.

        • The Ghost of Phil Rizzuto

          I’m unclear on what point you’re making. “The historical record is mixed at best” Huh? On the basis of a couple of cherry-picked examples that don’t even support the argument I think you’re trying to make?

          Alexander came in with the bases loaded and his team up by a run in the 7th. This IS the high-leverage, fireman approach. What part of that don’t you get? The fact that he finished the game is not relevant. Big Pete wasn’t even a reliever. In the regular season he started 23 games and saved only 2. How can you consider him a “closer”?

          Sparky was clearly used as fireman in the 1977 ALCS. In Game 4 he was brought in the 5th to preserve a 1-run lead! He did finish the game but get real – he pitched 5 1/3 innings!!! Prime relievers are not used that way anymore. Did Mo ever enter a game in the 5th inning or throw more than 3 innings post 1996?

          In Game 5 Lyle entered the game in the 8th with 2 on, 2 out and the Yankees trailing by a run! Please cite an example of Rivera (or any modern closer) being used in such a fashion in two consecutive games. Joe Torre (slave to the save) wouldn’t even use Rivera in a tie game on the road in the 2003 WS (Game 4)!

          • Evan3457

            I think we’re missing connections here…or maybe I’m misinterpreting Leverage Theory.

            I thought the point here was to bring in Mariano in the 7th or 8th, and let lesser relievers close out the last inning or two, because the “critcal point” of the game leverage-wise, might come in the 7th or 8th.

            That’s not how the great closers of the 70’s were used. They came in in the 7th/8th, got out of the critical situation, and, most of the time, went on to close the game out.

            Now, if you’re talking about pre-1960’s, it’s still not true. As I showed above with the Stengel Yankees, most of the time, when a team brought in a starter to close, he got the GF as well (90% of the time).

            What you’re advocating here is raising Mariano’s workload by 50-100%. (Gossage pitched 133 innings, all in relief, in 1977, 134 in 1978. In 1979, he got hurt fighting in the clubhouse with Cliff Johnson, in 1980, he threw 99 innings. In 1981, a strike ripped out 1/3 of the season and Ron Davis came along to “set up”. In 1982-3-4, he threw 93-87-102 innings.

            =================================

            By the way the examples in that post are not “cherry picked” they were the first three I thought of, and they DO support one of the points I wanted to make, which was, if Leverage Theory means use you best reliever in the critical spot and THEN TAKE HIM OUT, and let lesser relievers “close”, then 1) that’s NOT the way they did things long ago, and 2) it almost certainly won’t work with the modern players.

      • The209

        oh, so why didn’t Bill James (and Rebecca here) just advocate returning to pre-LaRussa strategies…? (would’ve saved a lotta time & keystrokes).

        Probably because LaRussa isn’t exactly what they are advocating here.

        • The209

          sorry “pre-LaRussa”

        • The Ghost of Phil Rizzuto

          Probably because LaRussa isn’t exactly what they are advocating here.

          Uh, no – the person providing the answers in this Q&A (Rebecca?) is advocating a strategy more like what used to be the case before the save rule came into vogue and LaRussa “invented” (yes, I know that’s an exaggeration) the modern closer. The data from that period is clearly relevant counter to your claim that there’s no data to evaluate the strategy on.

          Let me turn the argument around on you – before the save rule became the dictator of how relief pitchers were used was there any data to judge how effective or wise a strategy that would be? I didn’t think so. But that didn’t stop it from being adopted.

          • The209

            I think we are on the ‘same side’ of the argument here…

            As you said, they are “advoicating a strategy more like what used to be … ” BUT not exactly.

            The SABR-heads say — 9th inning isn’t necessarily the most important; bring in your best reliever at the most critical time, etc.

            But unlike all your examples above (and yes, I’m old enough to remember some of those times…) in their argument above, you’ll have Mo in (for example) to get out of an emergency in say the 6th — then go to Joba for the 8th, and whoever your 3rd-best guy is for the 9thetc.

            That’s a big difference than in earlier times when you had relief pitchers that could actually pitch 2, 3, 4, etc. innings.

            So I agree with you that it’d be great to go back to those times — but that’s not what the author here wants (unless I completely misread it)…

            sorry for any confusion.

            dHowever, what they are saying is

            • Evan3457

              Bingo; that’s what I’ve been trying to say both above and below.

  • Evan3457

    In theory, the Leverage Theory is sound. In practice, I don’t think it would be sound at all. In Strat-o-matic, or APBA baseball, it’s definitely the way to go. In table top baseball board games, the players are cardboard, mathematical constructions. In real baseball, they’re real people.

    I see two problems with it in real baseball. The lesser one:

    Like it or not, Jerome Holtzman invented the save. It’s not how relievers should be judged, but it is in fact how they are judged by everyone except the analysts that work in front offices and baseball blogs. Closers get paid more than set-up guys; not all of them, not all the time, but overwhelmingly, once they reach arbitration/free agency. The best paid closer gets paid twice as much, and more, than the best paid set up guy. Changing roles is going to drive relievers and agents nuts. Arbitrators may not be able to follow the argument for the Leverage Theory.

    Simply commanding, or even expecting, everyone to comprehend and put into practice Leverage Theory is not realistic. It doesn’t happen in real life outside of baseball.

    The second reason is more important: the players themselves will reject it. One of sure fast ways to get a team to quit on itself is to have relievers blow leads in the 9th inning, especially on the road, where they is no recovering from the blown lead. We have all seen teams quit down the stretch because their closer, succumbing to fatigue or the pressure, simply cannot close out “won” games.

    Because there are more chances teams can more readily accept a weaker reliever blowing the lead earlier in the game. It’s still a big letdown, but they can handle it better, because they still have chances. It’s built into the ethic of a ballplayer; never quit, the game isn’t over until the final out.

    It will not matter to most players that Mariano should be used in the 7th, when the LI is 2.91. They see the scoreboard, and it says 5-3 in the 7th, 2nd and 3rd, 1 outs. They’re not going to want Mariano to get out of it in the 7th, only to watch Park or Robertson blow it up in the 8th, or Joba implode in the 9th. They’re going to say, and rightfully so from their point of view, “Where the bleep was Mo in the 9th?”

    If a team tries using Leverage Theory, they increase the chances of blowing games in 9th innings. It may, theoretically, reduce the chances of blowing games as a whole, but the psychological damage to a team blowing games in the last inning is greater, and Leverage Theory can’t measure this. It’ll only take a few games like that for the team to revolt against the strategy.

    Finally, although it shouldn’t be this way, there are quite a few pitchers who are quite comfortable, even dominant, in the 7th and 8th innings, who have been tried as closers on several occasions, only to prove repeatedly they cannot get it done. (Ryan Madson, for example) Human psychology is not factored into Leverage Theory.

    Leverage Theory; great thinking, creative thinking, perfectly logical. Never happen, because the players won’t ever be convinced that the 7th inning is more important than the 9th. We can be convinced, because we can analyze rationally. They know better, from experience: the game isn’t over until the last man is out.

    • The Ghost of Phil Rizzuto

      But it has happened! It used to be almost exactly how relievers were used prior to the advent of the save rule.

      • Evan3457

        Not really. Managers didn’t put their best relievers in for the 7th or 8th inning, and then take him out and let a lesser pitcher close, and it doesn’t matter if the “closer” was a starter pitching off-rotation or a dedicated top reliever.

        Another example; the 1936-39 Yankees

        1936: #1 “closer” was Pat Malone, 9 (retroactive) saves. 26 relief appearances; games finished, 25.
        1937: Johnny Murphy, 10 saves. 35 relief appearances, 30 games finished, 110 innings (4 starts included).
        1938: Murphy, 11 saves, 30 relief appearances, 26 games finished, 91 innings (2 starts).
        1939: 38 relief appearances, 34 games closed, 19 saves, 0 starts, 61 innings pitched. Looks like a modern closer, with just 60% of the usual number of appearances, and few more 4 and 5 out saves.