Does Moneyball explain baseball?

The movie Moneyball, starring Brad Pitt, is now in cinemas. I read and thoroughly enjoyed Michael Lewis's book, but haven't had a chance to see the movie yet. But friends have asked what I think about the Moneyball premise - that is, the use of statistical analysis to improve the performance of sports teams - and here's what I tell them.

I would agree that the Oakland A's did play better than what you might expect from their relatively low payroll. Making the playoffs in a number of years during the heydey of Billy Beane's Moneyball era was a significant achievement. But  the A's never made it to the final round (the World Series), and so I think it is hard to call them an unqualified success story.

I do think there is a germ of truth in the idea: you can use statistics to better understand the game of baseball. As Duleep Allirajah notes in his spiked article, baseball, with set-piece match-ups between pitchers and batters, lends itself more to statistical analysis than a more fluid sport like English football or basketball.  Baseball scouts can become enamored with players who "look the part" of great athletes, even though there are well-chiseled types who fail and there are heavy-set types who look more like they should be sipping beer at a softball game who excel (just look at David Wells). Mining data can be a corrective to poor, subjective scouting reports.    

But even baseball cannot be reduced to the kind of statistical analysis cited in Moneyball. You can't capture everything about performance in team sports by breaking it down to statistics, because there are multiple individuals working together at the same time. At first glance it might seem that baseball can be boiled down to a pitcher facing a batter, but that pitcher is getting signals from his catcher regarding what pitch to throw.  Behind the pitcher are seven players in the field who will have a huge influence on the likelihood the pitcher will get the batter out. For his part, the batter is also receiving signals from coaches. Whether there are other runners on base will also be important as to how the pitcher will throw to him. There are many, many variables at play.

Furthermore, an important point that gets neglected or downplayed is that, at one point in the Moneyball book, Billy Beane mentions that, if he was the General Manager of the Yankees or another team, he wouldn't have adopted the approach he did. In other words, he felt forced to adopt a new approach given his situation, and while it might have been appropriate for the A's at that time, but it was not necessarily generally applicable.

I think we'll look back at the Moneyball ideas for baseball as something of a fad. Baseball has gone through them over time - for example, the move from having the starting pitcher pitch the entire game, to now where they almost never do, and instead you have defined roles for relief pitchers (the closer, the set-up man). This is not to say that nothing lasting will come of the Moneyball approach - for example, on-base percentage will be around for awhile (but then again, it was around in a different form before Moneyball - even in Little League coaches drilled into kids "a walk is as good as a hit").

Assessing performance is complex, especially in a team setting. Take the issue of evaluating the performance of an employee. The best kind of performance assessment is discretionary, not formulaic, because it can take into account a variety of factors - most people wouldn't want their performance to ride on one or two statistics. But discretionary, non-formulaic assessment can also be the worst kind too, as it can allow a boss to express favoritism or personal enmity.  For such reasons, some people prefer hardwired formulas, so there is no room at all for subjective assessment. But no data analysis can tell the whole story, and is unlikely to lead to a fair assessment. Ultimately, it comes down to the person or people making the assessment - the quality of their skills in assessment, based on a combination of data and other factors, some based even on intuition (which is really making conscious what is learned through experience, but not fully codified). 

Statistics can probably tell you more about a sport team member's performance than the performance of an employee on the job - and in some respects, that's one reason we like sports, because it is so much more clear-cut, with winners and losers, scorers and non-scorers, compared with the messiness of other aspects of life.

But even in sports, stats are not the whole story. This is especially true with team sports, where role players (as opposed to star players) are often critical. For instance, the US basketball team failed to win the gold medal in the 2004 Olympics, even though they clearly had the best collection of players. Statistics would have told you it was almost impossible for them to lose. But just watching them on the court it was obvious why they weren't playing well: they weren't doing the little things - setting picks, moving off the ball, making the extra pass, boxing out, etc - that are necessary. After that failure, the national team took that lesson to heart, and they now make a point of selecting role players and practicing more in advance, so they can learn to work together as a team.

Moneyball is great fun. But as much as we want to understand baseball, it remains unpredictable. Just this past year, consider the spectacular end-of-season collapse of the Red Sox, and the less-spectacular but still surprising fall of the Phillies, the team with the best regular-season record (and best pitching staff) in baseball. When upsets like that happen, you are reminded: that's why they play the game on the field, not on a computer program. And you are reminded why baseball is such a great game.