What would it take for the Rockies to re-create their bullpen success?

There’s a scene in Moneyball where Billy Beane agonizes over how to replace Jason Giambi and Johnny Damon after losing them to free agency after the 2001 season. Beane’s insight is to attempt to replace the contributions of the players rather than replacing the players themselves. As explained by his (fictionalized) assistant:

The goal shouldn't be to buy players, what you want to buy is wins. To buy wins, you buy runs. You're trying to replace Johnny Damon. The Red Sox look at Johnny Damon and they see a star worth seven point five million a year. When I look at Johnny Damon, I see an imperfect understanding of where runs come from.

While they weren’t around in 2002, we now have readily available statistics that approximate how many wins each player contributes to his team, and statistics that capture a player’s contributions to those wins in context. By both metrics, the Rockies bullpen was one of the best in the major leagues last year. The Rockies are in the midst of constructing their bullpen for 2018, and all signs indicate that they’re not done. So let’s ask the same question Billy Beane asked in 2001—what do they need to replace or improve, and how much will it help them win?

In a bullpen, to buy wins, you buy innings. But you also need to buy the right kind of innings. Specifically, you need to buy great high leverage innings, good medium leverage innings, and a healthy number of low leverage innings, too. Here’s how many innings Rockies relievers logged in 2017, by their respective leverage:

As an initial matter, it’s worth noting that the Rockies were exactly league average in total number of relief innings pitched last year. Our percentage of those innings that were high leverage was slightly higher than usual—the major league average is 15.8%—but not by much.

If we exclude starters in a relief role and Shane Carle cameos, here’s what kind of innings the Rockies got from our relievers last year, and each reliever’s FIP in those innings:

A few things stand out here. First, Bud Black was incredibly effective at managing his relievers. Particularly impressive is the fact that Oberg, Rusin, Estévez and Dunn were all more effective in high leverage situations than they were at other times. Some credit goes to the pitchers themselves, but much of it goes to Black’s ability to sense which relievers fit in specific situations, especially when it mattered the most. That said, Rockies relievers actually weren’t all that much better in high leverage situations as a whole—but this is almost entirely attributable to Greg Holland, who pitched the most high leverage innings and struggled in July and August. Aside from those months, Holland’s FIP in high leverage situations was a tidy 2.51.

Second, internal improvements and the addition of Bryan Shaw may already make up for the loss of Greg Holland. The departure (as of now) of Holland and Pat Neshek leaves the Rockies in need of 29.2 high leverage, 29 medium leverage, and 20.2 low leverage innings. Enter Bryan Shaw, a workhorse who pitched the following innings last year:

Shaw would already displace all but 12 high leverage innings, almost all the lost medium leverage innings, and add 12 low leverage innings that weren’t there previously. Those excess low leverage innings have to go somewhere, so we can assume that some of the low leverage innings pitched by Oberg, Dunn, Estévez, or (a hopefully rejuvenated) Otto will become high leverage innings next year.

This becomes important when we start thinking about the effects of signing another relief pitcher. If the Rockies sign Holland, Davis, Reed, or another reliever to compete for closer, they’ll effectively create a surplus of pitchers who can pitch high and medium leverage innings. Good arguments remain for signing another reliever—injuries might occur, someone might regress, and we might prefer Holland or Reed to our other pitchers anyway. But someone in our bullpen will get displaced, and if that someone is a good reliever, the Rockies will be employing a valuable asset in a less valuable spot.

And that brings us to the third point, which is that low leverage innings are really unimportant. To illustrate, take the case of Jordan Lyles. Lyles managed to pitch 46.2 innings for the Rockies last year, almost all of which were (thankfully) in low leverage innings. But what’s remarkable is that for as awful as Lyles was, he really didn’t negatively affect the team that much:

Remember watching Jordan Lyles publicly suffer, throwing batting practice to the other team when the game didn’t matter? Well, in those situations—which accounted for 82% of the innings he pitched—he had a negligent-to-slightly-positive effect on the team. In the mercifully few medium and high leverage innings that he did pitch, his WPA was negative and much more significant—a testament to how unimportant low leverage innings really are. Of course, this makes some intuitive sense: we ask the worst pitcher in the bullpen to pitch low leverage innings for a reason.

But what matters here is that signing another reliever means that we will be asking Rusin, Oberg, Dunn, and Carlos Estévez to take on a greater share of our low leverage innings next year. And asking Chris Rusin to take on more low leverage innings is a waste of a valuable resource, because low leverage innings don’t make much of a difference. It’s the equivalent of having a Jake Lamb on the bench, backing up Nolan Arenado—it doesn’t hurt the team, but it’s not a good use of resources, and it won’t do much to help the team win.