The motivation that lead me to the pitcher evaluation method to be
described is simple: I wanted a single performance number that could
be used to rank all pitchers regardless of when and how long they
appear in a game. The increasing specialization of pitching (starter,
long relief, middle relief, closer, mop up) and a burgeoning of
statistics used to describe each specialty, makes comparing
individual pitchers increasingly difficult. Another problem with
specialized categories is that many pitchers don't fit into such neat
categories. However, to make comparisons between traditional pitching
evaluation categories and the new measures proposed, I will do
exactly this.
The method I will present assigns a fraction of the game win or loss
to each pitcher that appears in it. The fraction is computed from the
earned runs allowed and the number of innings pitched by each
pitcher. This apportioning of the game win and loss gives the names
for the measures: apportioned wins (APW) and apportioned losses
(APL).
If the method is to prove useful, I believe it should satisfy a
number of constraints. First, it should have an interpretation that
is close to existing, well known measures. Pitching wins and losses
are probably the best known and most widely used of the current
performance measures. Apportioned wins and apportioned losses assigns
one pitching win for each team win and one pitching loss for each
team loss. At any time during the season, the sum of all pitching
apportioned wins and losses will equal the team win and loss totals
as the do the traditional pitching win and loss measures.
A second constraint on the new measure is that all pitchers appearing
in a game should share in the win or loss. The present pitcher
win-loss assignment system creates considerable inequities: A starter
gives up 3 runs in 5 innings but his team is leading when he is
relieved. The subsequent relievers shut out the opposition. The win
goes to the starter under the present system. Another example: the
starter provides 7 scoreless innings. One reliever allows the lead to
vanish. A closer surrenders one more run but his team wins the game
in the bottom of the ninth, 4-3. The closer gets the win. The
relatively small number of wins credited to each pitcher limits
minimizes the beneficial effects of averaging in reducing these types
of inequities over the course of a season.
The examples given in the proceeding paragraph illustrate another
problem with the traditional method. Credit given a pitcher depends
on when his team provides the winning runs. My third constraint is to
base credit for winning or losing solely on pitching performance and
not on when the team's offense provides the runs.
There are two additional constraints I feel the measure should
incorporate. There should be no credit for a loss if no earned runs
were surrendered. The pitcher win or loss should not be held hostage
to fielding performance. Similarly, I don't believe a pitcher should
receive credit for a win unless he records at least one out.
I have chosen innings pitched and earned runs allowed as the basic
quantities that will be combined to calculate the apportioned wins,
APW, and apportioned losses, APL, performance measures. Each
pitcher's contribution to a win is calculated from:
where IP is innings pitched and ER are earned runs allowed. This
ratio has the desired dependence on IP (increasing) and ER
(decreasing). The subscript k labels a particular pitcher.
In general, summing the contributions of each pitcher will not equal
the required 1 for the game win. The first step in calculating APW is
to sum the contribution of each pitcher (N appeared in the game):
Dividing individual pitcher contributions by T gives the desired apportioned wins for each pitcher in the game:
As before, the subscript k labels which pitcher.
This calculation is done for each game. Analogous to traditional
pitching wins, the APW for all the games a pitcher appears in are
added together to get his season total.
Some properties of this formula are obvious. By construction, exactly
one win is assigned for each game and the sum of all N APWk is
exactly 1. If there is only one pitcher, he gets the entire win. In a
game with no earned runs, the win is distributed proportional to the
innings pitched. If a pitcher does not record an out, his APW for the
game is 0.
Also obvious is the close relation between the definition of the
earned run average and the quantity used in the APW calculation. The
primary difference is the addition of 1 to the ER terms in the
denominator. This is a technical requirement since division by zero
would render the results meaningless. Essentially, the term used in
the APW calculation is the reciprocal of the ERA. A smaller ERA for
the game leads to a larger fraction of the win.
Apportioned losses (APL) are defined in a manner analogous to APW. If
a loss is to be credited, each pitchers contribution is:
As with APW, the calculation is made using outs rather than IP. Again, to avoid division by 0, 1 is added to each denominator. Here too, the sum of the individual contributions is not likely to be 1 so they are summed to get the normalizing quantity:
making each pitchers fraction of the loss:
It is entirely possible to have no earned runs in a loss. It
happens many times a season. The procedure I have followed to avoid
dividing by a 0 normalizing value is to apportion the loss based
solely on innings pitched if the losing staff did not surrender an
earned run.
The properties of this formula parallel the APW formula. There is
exactly 1 loss apportioned among the staff for each loss. If there is
only one pitcher, he receives the entire loss. By design, if a
pitcher does not give up an earned run, he does not get a portion of
the loss. (Excepting the case where the losing staff surrendered no
earned runs.)
With the exception of the 1 in the denominator, each losing fraction
is proportional to the pitcher ERA for the game. A larger ERA for the
game leads to a larger fraction of the loss.
Two examples will help clarify this. Pitcher A completes 7 innings
giving up 2 earned runs. Reliever B pitches a scoreless inning
followed by C who gives 1 run while recording the final outs.
Assuming the winning run was scored before he left the game, the
official scoring would give a win to A and a save to C and no credit
to B.
The T(win) and APW lines indicate how the computation works.
Pitcher A's APW is computed 7/11.5 = .609 . The larger part of the
win goes to the starter. The run allowed by the closer significantly
has reduced his portion of the credit. If the same scoring occurs but
this time the pitching line describes a loss, the T(loss) and APL
lines indicate how the loss apportioning calculation is done. Here
the 1 run by the closer in a single inning earns a proportionally
larger share of the loss.
In the second example, the starter D has a rocky outing, is followed
by E and F who pitch shut out ball in relief. The following table
shows the APW and APL calculation assuming the pitcher data
corresponds to either a win or a loss.
If the pitching corresponds to a win, the middle reliever get the
largest share. The starter receives some credit in spite of his poor
showing. If the pitching data corresponds to a loss the starter
receives all of it since the relievers gave up no runs.
These tables illustrate the basic calculations. More interesting is
the application of the technique to actual data. To do this, I have
obtained full season play by play accounts of the 1996 season from
the Baseball Workshop and the 1983 season from Retrosheet,
Inc. Analysis programs are also available but I have chosen to
write my own. The ability to apply a measure of my own design at the
game or even individual at bat level for an entire season is a
significant reward for the effort invested in writing such a
program.
The team name abbreviations I use are the BBWS and Retrosheet,
Inc. 3 letter codes. When you understand that a final "n" stands
for the National League and that a final "a" for the American, these
codes are obvious.
In the data listings that follow there are two additional quantities
I calculate from the game data. The first of these is ENT, the point
in the game a pitcher enters. The value of this quantity, which is 0
for starters, is the average number of innings completed at time of
game entry for the pitcher.
The second quantity I have defined to help understand the role of
pitchers in the game data is INA, innings per appearance, the total
number of innings pitched divided by the number of appearances for a
pitcher. This quantity is listed in the tables that follow.
To select groups of pitchers from the record I have used the
following selection criteria. First only pitchers with more than 50
innings credited have been classified. Then using the average time of
entry into the game, ENT, four classes of pitchers are defined:
Starters: ENT < 1
Long relievers: 1 <= ENT < 5
Middle relievers: 5 <= ENT < 7
Closers: 7 <= ENT
These definitions are admittedly arbitrary but are useful in the
context of this discussion. They are used to classify pitchers in the
data presentations that follow indicated using the underlined
letters.
Figure 1 shows a plot of APW vs. Wins for 1996 starters and also the
regression line relating APW and wins. The slope, 0.77, of the
regression line combined with an intercept near zero, 0.19, indicates
that for starters about 3/4 of an APW is credited for each official
win. Using the ANOVA technique the regression line explains 91% of
the variation. Apparently for starters, the fraction of wins lost by
leaving a game early is more important than official wins lost to the
relievers. Figure 4 shows APL as a function of losses for the same
group of pitchers. The slope of the regression line is 0.71 and the
ANOVA calculation indicates the regression line explains 78% of the
variance.
Similarly for starting pitchers, a regression analysis between APL
and losses also indicates a strong correlation. The slope of the
regression line is 0.71, the intercept is 0.43 and the ANOVA
calculation indicates the regression explains 78% of the variance
between the two quantities. As with APW, the number of APL credited
is a little smaller than the official loss numbers. The new measures
are very analogous to the traditional wins and losses for
starters.
Figure 2 displays a comparison of APW vs. wins for the 1996 middle
relievers using the definitions given above. The poor correlation
between APW and Wins as shown by the slope of the curve, 0.41, a high
intercept of 3.6, and a small value for the amount of variation
explained by the variance, 0.15, from the regression line all
indicate that the traditional win measure does not adequately account
for the importance of the middle reliever.
Figure 3 shows the correlation between APW and the traditional closer
measure of excellence, the save. There were 49 pitchers in 1996 that
met my closer definition. The regression line slope is 0.12, its
intercept is 6.0 and the ANOVA calculation indicates the regression
line represents 61% of the variance. The high intercept reflects the
earning of wins by the closers. I feel this shows good correlation
between saves and APW.
Tables 1 and 2 show the entire pitching staffs of the 1996 New York
Yankees and Atlanta Braves ranked by APW. Column P is the pitching
classification (S,L,M,C) defined above. The importance of the middle
reliever, Rivera, and closer, Wetteland, to the Yankees is clearly
shown in these rankings. The Atlanta pitching is dominated by the 3
starters, Smoltz, Glavine and Maddux with the closer Wohlers being
fourth.
The following links are to tables containing a listing of the top 150
pitchers ranked by APW. Relievers, both middle and closers, appear in
both these lists suggesting that the principle intent, finding a
single measure that will rank all pitchers, has been achieved.
Indeed, using the APW measure, a closer and middle reliever lead the
1983 pitcher standings. The traditional measure of pitching
excellence, the 20 game winning season, can be achieved using APW but
will not be quite so common.
Following are links to tables containing the top 150 pitchers when
ranked by the APW statistic.
I acknowledge the earlier work of Stephen Grant, "A New Pitching Evaluation Tool" that was given at the 1990 SABR Convention. My work is similar in spirit to his but differs greatly in detail.
Table 1:1996 New York Yankee Pitchers Ranked by APW
P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB
M Rivera, M 61 6.1 1.2 107.2 2.090 14.5 3.1 8 3 5 130 34
S Pettitte, A 35 0.1 6.0 221.0 3.869 13.6 7.2 21 8 0 162 72
C Wetteland, J 62 7.2 1.0 63.2 2.827 10.2 2.1 2 3 43 69 21
S Rogers, K 30 0.0 5.2 179.0 4.676 9.8 6.0 12 8 0 92 83
S Key, J 30 0.0 5.1 169.1 4.677 8.5 7.5 12 11 0 116 58
S Gooden, D 29 0.0 5.2 170.2 5.010 7.6 5.2 11 7 0 126 88
M Wickman, B 70 5.2 1.1 95.2 4.422 6.0 6.7 7 1 0 75 44
M Nelson, J 73 6.2 1.0 74.1 4.359 5.6 5.3 4 4 2 91 36
S Cone, D 11 0.0 6.1 72.0 2.875 4.6 1.8 7 2 0 71 34
S Mendoza, R 12 0.0 4.1 53.0 6.792 2.2 5.2 4 5 0 34 10
Boehringer, B 15 3.1 3.0 46.1 5.439 2.1 2.5 2 4 0 37 21
Polley, D 32 7.0 0.2 21.2 7.892 1.4 4.9 1 3 0 14 11
Howe, S 25 6.2 0.2 17.0 6.353 1.2 2.3 0 1 1 5 6
Mecir, J 26 6.0 1.1 40.1 5.132 1.2 1.8 1 1 0 38 23
Kamieniecki, S 7 1.0 3.0 22.2 11.12 1.2 1.4 1 2 0 15 19
Pavlas, D 16 6.1 1.1 23.0 2.348 1.0 0.3 0 0 1 18 7
Whitehurst, W 2 0.0 4.0 8.0 6.750 0.6 0.2 1 1 0 1 2
Gibson, P 4 5.1 1.0 4.1 6.231 0.3 0.1 0 0 0 3 0
Brewer, B 4 5.1 1.1 5.2 9.529 0.0 0.7 1 0 0 8 8
Aldrete, M 1 7.0 1.0 1.0 0.000 0.0 0.0 0 0 0 0 0
Table 2: 1996 Atlanta Braves Pitchers Ranked by APW
P PLAYER GM ENT INA IP ERA APW APL WN LO SV K BB
S Smoltz, J 35 0.0 7.0 253.2 2.945 19.2 6.5 24 8 0 276 54
S Glavine, T 36 0.0 6.1 235.1 2.983 13.1 8.9 15 10 0 181 84
S Maddux, G 35 0.0 7.0 245.0 2.718 12.3 9.0 15 11 0 172 28
C Wohlers, M 77 7.2 1.0 77.1 3.026 11.5 3.4 2 4 39 100 21
M Mcmichael, G 73 6.2 1.0 86.2 3.219 8.8 5.5 5 3 2 78 27
M Clontz, B 81 6.1 0.2 80.2 5.690 6.8 7.9 6 3 1 49 33
S Avery, S 24 0.0 5.1 131.0 4.466 5.2 6.1 7 10 0 86 40
M Bielecki, M 40 5.1 1.2 75.1 2.628 4.3 1.6 4 3 2 71 33
M Wade, T 44 5.2 1.1 69.2 2.971 3.9 2.4 5 0 1 79 47
S Schmidt, J 19 0.1 5.0 96.1 5.699 3.3 4.4 5 6 0 74 53
Borbon, P 43 7.1 0.2 36.0 2.750 3.1 1.8 3 0 1 31 7
Borowski, J 22 7.0 1.0 26.0 4.846 1.1 3.7 2 4 0 15 13
Woodall, B 8 3.2 2.1 19.2 7.322 1.0 1.3 2 2 0 20 4
Lomon, K 6 5.1 1.0 7.1 4.909 0.3 1.2 0 0 0 1 3
Schutz, C 3 6.1 1.0 3.1 2.700 0.0 0.1 0 0 0 5 2
Thobe, T 4 8.0 1.1 6.0 1.500 0.0 0.3 0 1 0 1 0
Back to the J. F. Jarvis baseball page.