John F. Jarvis
The motivation that lead me to the pitcher evaluation method to be
described is simple: I wanted a single performance number that could
be used to rank all pitchers regardless of when and how long they
appear in a game. The increasing specialization of pitching (starter,
long relief, middle relief, closer, mop up) and a burgeoning of
statistics used to describe each specialty, makes comparing
individual pitchers increasingly difficult. Another problem with
specialized categories is that many pitchers don't fit into such neat
categories. However, to make comparisons between traditional pitching
evaluation categories and the new measures proposed, I will do
The method I will present assigns a fraction of the game win or loss to each pitcher that appears in it. The fraction is computed from the earned runs allowed and the number of innings pitched by each pitcher. This apportioning of the game win and loss gives the names for the measures: apportioned wins (APW) and apportioned losses (APL).
If the method is to prove useful, I believe it should satisfy a number of constraints. First, it should have an interpretation that is close to existing, well known measures. Pitching wins and losses are probably the best known and most widely used of the current performance measures. Apportioned wins and apportioned losses assigns one pitching win for each team win and one pitching loss for each team loss. At any time during the season, the sum of all pitching apportioned wins and losses will equal the team win and loss totals as the do the traditional pitching win and loss measures.
A second constraint on the new measure is that all pitchers appearing in a game should share in the win or loss. The present pitcher win-loss assignment system creates considerable inequities: A starter gives up 3 runs in 5 innings but his team is leading when he is relieved. The subsequent relievers shut out the opposition. The win goes to the starter under the present system. Another example: the starter provides 7 scoreless innings. One reliever allows the lead to vanish. A closer surrenders one more run but his team wins the game in the bottom of the ninth, 4-3. The closer gets the win. The relatively small number of wins credited to each pitcher limits minimizes the beneficial effects of averaging in reducing these types of inequities over the course of a season.
The examples given in the proceeding paragraph illustrate another problem with the traditional method. Credit given a pitcher depends on when his team provides the winning runs. My third constraint is to base credit for winning or losing solely on pitching performance and not on when the team's offense provides the runs.
There are two additional constraints I feel the measure should incorporate. There should be no credit for a loss if no earned runs were surrendered. The pitcher win or loss should not be held hostage to fielding performance. Similarly, I don't believe a pitcher should receive credit for a win unless he records at least one out.
I have chosen innings pitched and earned runs allowed as the basic quantities that will be combined to calculate the apportioned wins, APW, and apportioned losses, APL, performance measures. Each pitcher's contribution to a win is calculated from:
where IP is innings pitched and ER are earned runs allowed. This
ratio has the desired dependence on IP (increasing) and ER
(decreasing). The subscript k labels a particular pitcher.
In general, summing the contributions of each pitcher will not equal the required 1 for the game win. The first step in calculating APW is to sum the contribution of each pitcher (N appeared in the game):
Dividing individual pitcher contributions by T gives the desired apportioned wins for each pitcher in the game:
As before, the subscript k labels which pitcher.
This calculation is done for each game. Analogous to traditional pitching wins, the APW for all the games a pitcher appears in are added together to get his season total.
Some properties of this formula are obvious. By construction, exactly one win is assigned for each game and the sum of all N APWk is exactly 1. If there is only one pitcher, he gets the entire win. In a game with no earned runs, the win is distributed proportional to the innings pitched. If a pitcher does not record an out, his APW for the game is 0.
Also obvious is the close relation between the definition of the earned run average and the quantity used in the APW calculation. The primary difference is the addition of 1 to the ER terms in the denominator. This is a technical requirement since division by zero would render the results meaningless. Essentially, the term used in the APW calculation is the reciprocal of the ERA. A smaller ERA for the game leads to a larger fraction of the win.
Apportioned losses (APL) are defined in a manner analogous to APW. If a loss is to be credited, each pitchers contribution is:
As with APW, the calculation is made using outs rather than IP. Again, to avoid division by 0, 1 is added to each denominator. Here too, the sum of the individual contributions is not likely to be 1 so they are summed to get the normalizing quantity:
making each pitchers fraction of the loss:
It is entirely possible to have no earned runs in a loss. It
happens many times a season. The procedure I have followed to avoid
dividing by a 0 normalizing value is to apportion the loss based
solely on innings pitched if the losing staff did not surrender an
The properties of this formula parallel the APW formula. There is exactly 1 loss apportioned among the staff for each loss. If there is only one pitcher, he receives the entire loss. By design, if a pitcher does not give up an earned run, he does not get a portion of the loss. (Excepting the case where the losing staff surrendered no earned runs.)
With the exception of the 1 in the denominator, each losing fraction is proportional to the pitcher ERA for the game. A larger ERA for the game leads to a larger fraction of the loss.
Two examples will help clarify this. Pitcher A completes 7 innings giving up 2 earned runs. Reliever B pitches a scoreless inning followed by C who gives 1 run while recording the final outs. Assuming the winning run was scored before he left the game, the official scoring would give a win to A and a save to C and no credit to B.
The T(win) and APW lines indicate how the computation works.
Pitcher A's APW is computed 7/11.5 = .609 . The larger part of the
win goes to the starter. The run allowed by the closer significantly
has reduced his portion of the credit. If the same scoring occurs but
this time the pitching line describes a loss, the T(loss) and APL
lines indicate how the loss apportioning calculation is done. Here
the 1 run by the closer in a single inning earns a proportionally
larger share of the loss.
In the second example, the starter D has a rocky outing, is followed by E and F who pitch shut out ball in relief. The following table shows the APW and APL calculation assuming the pitcher data corresponds to either a win or a loss.
If the pitching corresponds to a win, the middle reliever get the
largest share. The starter receives some credit in spite of his poor
showing. If the pitching data corresponds to a loss the starter
receives all of it since the relievers gave up no runs.
These tables illustrate the basic calculations. More interesting is the application of the technique to actual data. To do this, I have obtained full season play by play accounts of the 1996 season from the Baseball Workshop and the 1983 season from Retrosheet, Inc. Analysis programs are also available but I have chosen to write my own. The ability to apply a measure of my own design at the game or even individual at bat level for an entire season is a significant reward for the effort invested in writing such a program.
The team name abbreviations I use are the BBWS and Retrosheet, Inc. 3 letter codes. When you understand that a final "n" stands for the National League and that a final "a" for the American, these codes are obvious.
In the data listings that follow there are two additional quantities I calculate from the game data. The first of these is ENT, the point in the game a pitcher enters. The value of this quantity, which is 0 for starters, is the average number of innings completed at time of game entry for the pitcher.
The second quantity I have defined to help understand the role of pitchers in the game data is INA, innings per appearance, the total number of innings pitched divided by the number of appearances for a pitcher. This quantity is listed in the tables that follow.
To select groups of pitchers from the record I have used the following selection criteria. First only pitchers with more than 50 innings credited have been classified. Then using the average time of entry into the game, ENT, four classes of pitchers are defined:
Starters: ENT < 1 Long relievers: 1 <= ENT < 5 Middle relievers: 5 <= ENT < 7 Closers: 7 <= ENT
These definitions are admittedly arbitrary but are useful in the
context of this discussion. They are used to classify pitchers in the
data presentations that follow indicated using the underlined
Figure 1 shows a plot of APW vs. Wins for 1996 starters and also the regression line relating APW and wins. The slope, 0.77, of the regression line combined with an intercept near zero, 0.19, indicates that for starters about 3/4 of an APW is credited for each official win. Using the ANOVA technique the regression line explains 91% of the variation. Apparently for starters, the fraction of wins lost by leaving a game early is more important than official wins lost to the relievers. Figure 4 shows APL as a function of losses for the same group of pitchers. The slope of the regression line is 0.71 and the ANOVA calculation indicates the regression line explains 78% of the variance.
Similarly for starting pitchers, a regression analysis between APL and losses also indicates a strong correlation. The slope of the regression line is 0.71, the intercept is 0.43 and the ANOVA calculation indicates the regression explains 78% of the variance between the two quantities. As with APW, the number of APL credited is a little smaller than the official loss numbers. The new measures are very analogous to the traditional wins and losses for starters.
Figure 2 displays a comparison of APW vs. wins for the 1996 middle relievers using the definitions given above. The poor correlation between APW and Wins as shown by the slope of the curve, 0.41, a high intercept of 3.6, and a small value for the amount of variation explained by the variance, 0.15, from the regression line all indicate that the traditional win measure does not adequately account for the importance of the middle reliever.
Figure 3 shows the correlation between APW and the traditional closer measure of excellence, the save. There were 49 pitchers in 1996 that met my closer definition. The regression line slope is 0.12, its intercept is 6.0 and the ANOVA calculation indicates the regression line represents 61% of the variance. The high intercept reflects the earning of wins by the closers. I feel this shows good correlation between saves and APW.
Tables 1 and 2 show the entire pitching staffs of the 1996 New York Yankees and Atlanta Braves ranked by APW. Column P is the pitching classification (S,L,M,C) defined above. The importance of the middle reliever, Rivera, and closer, Wetteland, to the Yankees is clearly shown in these rankings. The Atlanta pitching is dominated by the 3 starters, Smoltz, Glavine and Maddux with the closer Wohlers being fourth.
The following links are to tables containing a listing of the top 150 pitchers ranked by APW. Relievers, both middle and closers, appear in both these lists suggesting that the principle intent, finding a single measure that will rank all pitchers, has been achieved. Indeed, using the APW measure, a closer and middle reliever lead the 1983 pitcher standings. The traditional measure of pitching excellence, the 20 game winning season, can be achieved using APW but will not be quite so common.
Following are links to tables containing the top 150 pitchers when ranked by the APW statistic.
I acknowledge the earlier work of Stephen Grant, "A New Pitching Evaluation Tool" that was given at the 1990 SABR Convention. My work is similar in spirit to his but differs greatly in detail.
Table 1: 1996 New York Yankee Pitchers Ranked by APW
P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB M Rivera, M 61 6.1 1.2 107.2 2.090 14.5 3.1 8 3 5 130 34 S Pettitte, A 35 0.1 6.0 221.0 3.869 13.6 7.2 21 8 0 162 72 C Wetteland, J 62 7.2 1.0 63.2 2.827 10.2 2.1 2 3 43 69 21 S Rogers, K 30 0.0 5.2 179.0 4.676 9.8 6.0 12 8 0 92 83 S Key, J 30 0.0 5.1 169.1 4.677 8.5 7.5 12 11 0 116 58 S Gooden, D 29 0.0 5.2 170.2 5.010 7.6 5.2 11 7 0 126 88 M Wickman, B 70 5.2 1.1 95.2 4.422 6.0 6.7 7 1 0 75 44 M Nelson, J 73 6.2 1.0 74.1 4.359 5.6 5.3 4 4 2 91 36 S Cone, D 11 0.0 6.1 72.0 2.875 4.6 1.8 7 2 0 71 34 S Mendoza, R 12 0.0 4.1 53.0 6.792 2.2 5.2 4 5 0 34 10 Boehringer, B 15 3.1 3.0 46.1 5.439 2.1 2.5 2 4 0 37 21 Polley, D 32 7.0 0.2 21.2 7.892 1.4 4.9 1 3 0 14 11 Howe, S 25 6.2 0.2 17.0 6.353 1.2 2.3 0 1 1 5 6 Mecir, J 26 6.0 1.1 40.1 5.132 1.2 1.8 1 1 0 38 23 Kamieniecki, S 7 1.0 3.0 22.2 11.12 1.2 1.4 1 2 0 15 19 Pavlas, D 16 6.1 1.1 23.0 2.348 1.0 0.3 0 0 1 18 7 Whitehurst, W 2 0.0 4.0 8.0 6.750 0.6 0.2 1 1 0 1 2 Gibson, P 4 5.1 1.0 4.1 6.231 0.3 0.1 0 0 0 3 0 Brewer, B 4 5.1 1.1 5.2 9.529 0.0 0.7 1 0 0 8 8 Aldrete, M 1 7.0 1.0 1.0 0.000 0.0 0.0 0 0 0 0 0
Table 2: 1996 Atlanta Braves Pitchers Ranked by APW P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB S Smoltz, J 35 0.0 7.0 253.2 2.945 19.2 6.5 24 8 0 276 54 S Glavine, T 36 0.0 6.1 235.1 2.983 13.1 8.9 15 10 0 181 84 S Maddux, G 35 0.0 7.0 245.0 2.718 12.3 9.0 15 11 0 172 28 C Wohlers, M 77 7.2 1.0 77.1 3.026 11.5 3.4 2 4 39 100 21 M Mcmichael, G 73 6.2 1.0 86.2 3.219 8.8 5.5 5 3 2 78 27 M Clontz, B 81 6.1 0.2 80.2 5.690 6.8 7.9 6 3 1 49 33 S Avery, S 24 0.0 5.1 131.0 4.466 5.2 6.1 7 10 0 86 40 M Bielecki, M 40 5.1 1.2 75.1 2.628 4.3 1.6 4 3 2 71 33 M Wade, T 44 5.2 1.1 69.2 2.971 3.9 2.4 5 0 1 79 47 S Schmidt, J 19 0.1 5.0 96.1 5.699 3.3 4.4 5 6 0 74 53 Borbon, P 43 7.1 0.2 36.0 2.750 3.1 1.8 3 0 1 31 7 Borowski, J 22 7.0 1.0 26.0 4.846 1.1 3.7 2 4 0 15 13 Woodall, B 8 3.2 2.1 19.2 7.322 1.0 1.3 2 2 0 20 4 Lomon, K 6 5.1 1.0 7.1 4.909 0.3 1.2 0 0 0 1 3 Schutz, C 3 6.1 1.0 3.1 2.700 0.0 0.1 0 0 0 5 2 Thobe, T 4 8.0 1.1 6.0 1.500 0.0 0.3 0 1 0 1 0
Revisions: Jan 1, 2009 Complete 1954 - 2008
Back to the J. F. Jarvis baseball page.
Copyright 1997-2009, John F. Jarvis