How well Walt Weiss has managed the Colorado Rockies' bullpen is up for debate

The most important thing in baseball is, of course, having good players. Without that, nothing else matters. Take whoever you think is the best manager of all time, give him a bad roster, and you'll have a bad team.

On top of that, it's extremely difficult to quantify a lot of the things a manager does. There's no statistic that shows how well a manager builds relationships with players or how hard he gets those players to play. Even things that are arguably more important like managing rest and keeping players fresh are difficult, if even possible at all, to quantify.

All of this understandably leads to a general assumption that good teams have good managers and bad teams have bad managers. While likely an oversimplification of things, this line of thinking makes sense given the general lack of other information we have to work with.

One thing that can be quantified, however, is bullpen management. On Wednesday, Ryan Schoppe wrote about Walt Weiss' early struggles managing the bullpen. It's a fair criticism and a great look at bullpen mismanagement on a small scale, but how has Weiss done managing his bullpen as a whole? I dug into the numbers to find out.

To find this out, the first thing I needed to do was determine the most important aspects of properly managing a bullpen. I landed on two things.

1) How well did the manager do at using his best relievers in the most important situations?

2) How often did the manager put his relievers into situations when they had a platoon advantage?

Before I did anything, I created a list of the top 15 relief pitchers by innings pitched since Weiss took over as manager in 2013 (these numbers do not include the games played Wednesday and Thursday). To provide a point of reference, I did the same thing with the remaining four NL West teams (the Padres ended up with 16 due to a tie).

With that list compiled, let's try to answer the first question. From each team's list of relievers, I took the top five in gmLI -- the average leverage index each pitcher faced when he entered the game -- and found where they ranked among that same group of 15 in Context Neutral Wins (WPA/LI), which tells us how valuable a pitcher was regardless of leverage, and xFIP, a predictive ERA stat that works best in the relatively small sample sizes we see from relief pitchers. Theoretically, the top five in gmLI should line up pretty closely with the top five in WPA/LI and xFIP.

This method won't catch everything and may miss some of the smaller aspects of bullpen management that the other Ryan talked about earlier this week, but it should give us a good big picture view of how these teams have used their bullpens. Let's see how everybody did.

The Giants clearly come out on top here, but after that there's actually a strong argument to be made that the Rockies are the second best team, as they come in second in average WPA/LI and third in average xFIP.

Of that group of Rockies, Boone Logan sticks out like a sore thumb. He has been the worst Rockies reliever in terms of context neutral wins during the Weiss era, yet he has been used in the second most important situations. That isn't good. They aren't alone, though! With Heath Bell and Jeremy Affeldt, the Diamondbacks and Giants have each used one of their worst relievers repeatedly in high leverage situations as well.

On the whole, at least through this lens, it looks like Walt Weiss' Rockies haven't been egregiously worse than their division rivals at getting their top relievers into the most important parts of the game. That isn't everything, though. Perhaps Weiss hasn't platooned his relievers as well as others. Maybe he's been better! We'll have to take a look at the numbers to find out.

From that same list of 15 relievers, I took those with a weighted on-base average (wOBA) against at least 49 points better against left-handed hitters than it was against right-handed hitters, then calculated how much more often they were used against left-handed hitters than the team's overall average amount of left-handed hitters faced (any pitchers who faced more than ten percent of their total batters faced as a starting pitcher weren't included here so as not to skew the numbers). In theory, these are the pitchers managers should have been using more as specialists given how much better they were against left-handed batters, which in turn means the more often they were used against lefties, the better. Let's see how everyone did.

There are a few ways to interpret this information. The simplest is to say that the Rockies have done the best job at using specialists, as their 9.4 percent above average for the trio of Josh Outman, Wilton Lopez, and Boone Logan is more than two percentage points better than any team. That isn't the only thing worth looking at here, though.

One thing that immediately jumps out is, across the board, right-handed pitchers were not used as specialists against left-handed hitters, even if they had significantly better numbers against them. Digging a bit deeper into these pitchers, Lopez, Hudson, Guerrier, Wright, and Brach have relatively even platoon splits for their careers, while Reed, Harris, Withrow, Dunning, and Maurer have always been better against left-handed hitters.

That's a pretty even distribution of names, but it seems the more experienced pitchers have seen their splits even out as they've thrown more innings. When you consider that along with the fact that, league-wide, pitchers have always done better against same-handed hitters, it makes perfect sense that managers wouldn't intentionally be bringing in right-handed pitchers to face left-handed batters. Let's look at the same chart again, but this time without any of the right-handed pitchers.

This obviously gives us less data to work with, but the Rockies come out looking much less rosy here. The Dodgers and Giants have each used their lefty specialists more often against left-handed hitters, while the D-Backs and Padres only give us one pitcher to look at between the two of them -- hardly a representative sample size. This seems to be the area where Weiss has struggled the most, and it's backed up by the fact that Logan has been used far too often in high leverage situations and not often enough in situations when he had the platoon advantage.

So, what does all of this mean? It's hard to say. Weiss appears to be just as good as, if not slightly better than, his divisional peers at getting his top relievers into the most important situations, but he has also struggled to get his lefty specialists into situations that play to their strengths often enough.

It's possible—likely even—that poor bullpen management has cost the Rockies games since Weiss took over. It is also almost impossible to take a wide-ranging look at how well Weiss has done at things like managing rest and keeping all of his bullpen members as fresh as possible (though I suspect he hasn't been great in this area).

Walt Weiss doesn't appear to be the best bullpen manager out there, but he doesn't appear to be the worst, either. It's difficult to fairly evaluate a manager, but one thing we can say almost for certain is that a roster with a lot of good players would probably make all of them look quite a bit smarter.