A common topic on r.s.bb and r.s.bb.a, as well as in baseball circles in general, is the impact of batting order on run production. For individual players, people discuss whether they are in the correct spot in the order. Managers are criticized (or lauded) based on their perceived weakness (or strength) in setting the batting order. One major problem, though, is that there isn't much data available on the impact of batting order on team batting performance. There is enough random variability in player performance that even fairly large possible effects on run scoring due to batting order effects can be obscured by random effects over times as long as individual careers or longer.
This large random element makes a simulation the only practical means of studying batting order effects. With a simulation, it is practical to get data over thousands of "seasons" with the same batting order, allowing the systematic batting order effects to overcome the randomness.
The core of my study was a simple baseball simulator which I wrote; I will gladly share the code with anyone interested. For player data, I used some late season stats from the 1996 L.A. Dodgers. I chose the Dodgers for several reasons.
For one thing, I am quite familiar with them; I live in the L.A. area, and see the Dodgers more often than any other team. Secondly, the Dodgers have an unusually large spread of talent, since their best hitter (Piazza) is clearly one of the best hitters in the league, while their worst (DeShields) was this year's Sultan of Squat. A large spread in talent should tend to enhance the effects of batting order on team scoring and winning. Thirdly, Piazza is the Dodgers' best hitter for both power and on base average, and two of the most popular (on r.s.bb at least) theories of batting order construction (conventional or OBA based) have different ideas about where in the order to place such a hitter. Finally, I felt that using genuine player statistics from a real team would help to ensure that the study was more realistic.
My simulation was rather simple. Each team was given the same players, but a different batting order. The teams batted in alternating half innings exactly as they would in a real baseball game, and played nine innings, going into extra innings if necessary. Each player was assigned probabilities of a hit, walk, or out based on their actual statistics. If a player got a hit, it was decided randomly if it would be for extra bases, and then if it would be for a double or a homer. To simplify the simulation, there were no triples, and the probabilities of a double or homerun were based only on the players' BA and SLG (in a way that accurately regenerated their real world SLG, but not necessarily their distribution of 2B's and HR's) rather than their personal patterns. The chance of a player advancing an extra base (three bases on a double, two on a single, or one on an out) were set in a rather ad hoc fashion (if anyone has real world data on these things, I would love to have it), and there were no errors or pitching effects.
The standard lineup used was one that Bill Russel often used late in the Dodgers' season (Hollandsworth, Kirby, Piazza, Karros, Mondesi, Wallach, DeShields, Gagne, Pitcher in that order.) I didn't have the actual pitchers' batting data, so I used a made up set which was intended to come close (.151/.180/.180). This standard order was played against three alternate teams, each with the same players but a different batting order: descending OBA order, ascending OBA order, and a random order. The teams played a large number of 162 game seasons, with the random batting order team assigned a new order each season. The program kept track of both wins and losses and winning and losing seasons. An 81-81 season was counted as half a winning season and half a losing season.
Over 5000 seasons (810,000 games), the team with the standard batting order recorded the following records:
|Opponent||Wins||Win%||W Seasons||Season W%|
Using a 50% chance of winning each game or season as a null hypothesis, the expected standard deviation for wins is 450, and 35 for winning seasons. That gives means that (using three sigma significance) both a random order and ascending OBA order are statistically significantly worse than the conventional batting order that Russel normally used, both in terms of wins and winning seasons. The descending OBA order was better, but it was only better by about 2 sigma in wins and one sigma in winning seaons, so its significance is questionable.
Because of the lack of statistical significance for the descending OBA trial, I tried a new one, over 20,000 seasons or 3.63 million games. The results were substantially similar (1615863 wins/49.872%, 9717.5 winning seasons/48.59%). The larger number of games, though, gave STD's of 900 games/71 winning seasons, meaning that the results were more than 3 sigma from expectations if the chance of a win/winning season were exactly 50%.
Batting order clearly has significant impact on a team's chances. A team which used the worst batting order tested (ascending OBA order) would have an average record of 77.8-84.2 (over six games below .500) against a team with the same talent but a convetional order. Six games is easily enough to make the difference between playing in the post-season and watching it from home, so it obviously behooves a team to use a reasonable batting order.
The difference between a convetional batting order and the best tested order (descending OBA), though, is comparatively small- about .4 games a season, or two games every five seasons. It is also important to notice that the two lineups are not terribly similar (Hworth, Kirby, Piazza, Karros, Mondesi, Wallach, Deshields, Gagne, Pitcher v.s. Piazza, Hworth, Gagne, Mondesi, Kirby, Wallach, Karros, DeShields, Pitcher) but still produce quite similar results. In practical terms, that probably means that lineup decisions based on tactical considerations (not putting all your lefties in a row, keeping a player at leadoff because he feels comfortable there, etc.) may outweigh the minor benefits of using a theoretically optimal order. Four tenths of a win is about four runs over the course of a season, and a player doesn't have to hit a whole lot better (or steal a whole lot more bases, etc.) to be worth four extra runs.
Equally importantly, though, is that the conventional wisdom WRT batting orders seems to be wiser than many r.s.bb regulars give it credit for. The conventional order was close enough to optimal that it took far more games than have been played in the history of MLB to show that the descending OBA order was statistically significantly better. Although lineups had considerably more variablity in the early days of MLB, it seems likely that the CW was derived more from intellectual consideration than from experimentation, since experimental results show such variablity.
I also tried an evolutionary algorithm in an attempt to "evolve" an optimal batting order. The program would start with a random batting order, then play it against an similar order in which two players' positions had been swapped. Whichever order did better would "survive" and play against a "mutated" version of itself, be tested, mutate, etc. This method turned out not to work very well for essentially the same reason that it was difficult to tell whether a conventional or OBA based order was best. Small changes in the order simply do not have a large impact on on the field results, and there is a significant chance that the _worse_ order will do better over trials which take significant amouts of computer time. (5000 season trials took about 10 hours.) When I started from a known, good order, the order frequently evolved _backward_ to give a worse order just by random chance.
Note:This piece was originally posted to the USENET group rec.sport.baseball.analysis, recovered using Dejanews, and modified into a HTML document. My basic program is available in text form.
back to writing index
back to Roger's home pageBatting Order / Roger Moore / firstname.lastname@example.org / June 1997