Technical Appendix

Part A

As mentioned in the main article, DRA is to defense what Pete Palmer's Linear Weights has been over the last 30 years or so to offense. More specifically, DRA

1. takes whatever objective, publicly-available, team-level, seasonal defensive statistics exist in any given period of major league baseball history,
2. determines the statistically and practically significant relationships between and among such statistics and team runs allowed each season,
3. produces objective Linear Weights-type formulas for defense based on those statistical relationships, and
4. estimates the number of runs 'saved' or 'allowed' by each pitcher and fielder on a team relative to the league-average player at his position based on such formulas.

The estimates of individual pitcher and fielder runs "saved" or "allowed" add up to an estimate of the number of runs the team allowed above or below the league-average that season. The team-level estimate is usually about as accurate (standard error of about 25 runs) as Linear Weights is in estimating team offensive runs above or below the league rate.

DRA is the first and only system for predicting team runs allowed per season throughout major league history based on the team defensive statistics that season. As mentioned before, DRA is also the only open-source model for estimating the impact of each pitcher and fielder on team runs allowed throughout major league history.

What follows is an overview of the DRA method for estimating singles allowed at shortstop using objective, public data generally available since 1989 (this method is updated from the method in Wizardry).

Clear away duplicative or uninformative data

DRA starts by ignoring statistics that don't reflect fielder skill or involve events more properly credited or charged to another fielder or the pitcher.

Infielder putouts (with one exception we'll address shortly) are a classic example of an officially recorded statistic that should be ignored when evaluating infielders. Credit for putouts recorded on force outs or for tagging runners belongs to the assisting fielder. Putouts recorded by shortstops fielding pop-ups and high fly balls are almost always automatic outs that could be fielded by any major league shortstop or by one or two neighboring fielders (e.g., the third baseman, the left fielder, the center fielder or the second baseman, depending on where the pop fly was hit).

Yes, there are very occasionally outstanding fielding plays by shortstops on line-drives and Texas Leaguers, but there is no evidence supporting the notion that any shortstop has, year-by-year, consistently prevented a significant number of hits that way. And of course putouts recorded by receiving a throw and tagging a base or a runner should also be ignored; credit for the out belongs to the assisting fielder.

The one exception involves cases in which an infielder fields a ground ball and runs to a base to record a putout. For first baseman, this is a very common event and needs to be dealt with. For infielders it is comparatively rare, but courtesy of retrosheet.org we have the data, so we will use it to count all ground balls fielded by a shortstop in which the lead runner or the batter is out, whether because the shortstop ran to second to record a putout there or threw to second or first or wherever to record the out (ground outs fielded at short, or "GO6").

DRA also ignores errors (though the most recent version tracks pure throwing errors by catchers and outfielders trying to throw base-runners out). Errors fielding batted balls (whether one drops the ball or fields a ground ball but makes a poor throw) are already 'recorded' as plays not made. As noted in Wizardry, "extensive studies have shown that an error is, on average, essentially no more harmful than simply allowing a clean hit."

"Center" each defensive outcome and variable that impacts such defensive outcome

The defensive outcome we are trying to predict is ground out plays at shortstop. We initial "center" this defensive outcome based on innings played. Thus, we would calculate for each team GO6 above or below the league average rate, given innings played (GO6|ip).

The thin vertical bar between the upper-case outcome and the lower-case explanatory variable means "given." A thin vertical bar is commonly used in other contexts when you want to say, "expected value of this, given the value of that." Here it means that season's net team difference above or below that season's league-average value of 'this' (GO6), given the net team difference above or below the average league value of 'that' (ip).

GO6|ip is simply team GO6 minus the product of team ip and the quotient of league GO6 over league ip.

Next, we try to think of every publicly available statistic reflecting a factor beyond the control of shortstops that would increase or decrease GO6|ip (each an "explanatory variable"), such as ball in play (BIP), the number of ground balls (GB), the number of ground balls hit by left-handed batters (LGB), etc.

Next, we collect large samples of seasonal team totals and center each explanatory variable in such a way that the variables are at least arithmetically independent of each other, so we don't factor them in more than once.

Thus we might calculate BIP allowed by the team pitchers above or below the league average rate, given ip (BIP|ip).

Then GB allowed by the team's pitchers, above or below the league average rate, given, not innings pitched, but BIP (so we don't double-count the effect of BIP) (GB|bip).

Note that negative GB|bip means proportionately more balls hit in the air.

Then we calculate team ground balls allowed to left-handed batters (LGB), above or below the league average rate, given total ... GB (LGB|gb) (so a negative number means proportionately more ground balls allowed to right-handed batters).

Then we calculate team ground balls allowed by left-handed pitchers to left-handed batters, given total LGB (LLGB|lgb) (so a negative number means proportionately more right-handed pitchers allowed ground balls to left-handed batters, given the total ground balls hit by left-handed batters).

Then we calculate team ground balls allowed by left-handed pitcher to right-handed batters, given total ground balls hit by right-handed batters (LRGB|rgb) (so a negative number means more right-handed pitchers gave up ground balls to right-handed batters, given the number of total ground balls hit by right-handed batters).

Regress centered outcomes onto these centered explanatory variables

Dizzy yet? When you "regress" GO6|ip "onto" BIP|ip, GB|bip, LGB|gb, LLGB|lgb and LRGB|rgb, the result reveals the amount by which each variable on the right (again, an "explanatory variable") increases or decreases the outcome we are trying to predict, GO6|ip.

Turns out that only the first three matter, so you can simply take the number of ground balls fielded by a shortstop hit by left- and right-handed batters above or below the league-average rate, given the number of ground balls hit by left- and right-handed batters while that shortstop is in the field, or the sum of LGO6|lgb and RGO6|rgb. Pitcher-handedness, after you have taken into account batter-handedness (which is obviously influenced by pitcher-handedness) doesn't matter.

Regress again

Then we regress team LGO6|lgb onto the number of times above or below the league average the team was fielding with a runner on first base with less than two outs and a ground ball is hit by a left-handed batter, and then do the same for RGO6|rgb with right-handed batters. No statistically significant effect.

Finally, and most painfully, using my new MySQL database built from Retrosheet.org data, I calculated the career out-distributions for batters batting left and batters batting right and summed up for each team the relative number of such 'expected' LGO6|lgb and RGO6|rgb per batter. Again, no effect at shortstop or any other position, except a borderline important effect for ground balls to third base by right-handed batters. When I say that the effect is not important, we're talking about an effect of about 2 or at most 3 runs a season, which does not persist for fielders from year-to-year and accordingly is more or less random. Put another way, including this factor did not reduce the average annual variance in team fielding outcomes at any position by more than 1%.

On the basis of the above regressions, I concluded in my article in The Hardball Times Baseball Annual 2012 that you could rate shortstops by the sum of their LGO6|lgb and RGO6|rgb, that is, their net ground balls fielded against left-handed batters, given the number of total ground ball hit by left-handed batters when the shortstop being rated was on the field, and the same for ground ball hit by right-handed batters.

When preparing this article, I thought I'd include pitcher fielding to be scrupulously fair to Jeter. I regressed LGO6|lgb onto the similar fielding statistic for pitchers, LGO1|lgb, and did the same for balls hit by right-handed batters. It turns out that for every ground ball hit by a left-handed batter and fielded by the pitcher, 0.1 fewer LGO6|lgb were recorded, and for every ground ball hit by a right-handed batter and fielded by a pitcher, 0.3 fewer RGO6|lgb were recorded. So I backed-out this factor for Jeter as shown above.

Part B

The Yankee Stadium park factor for shortstops

To address the Yankee Stadium park factor identified by Sean Smith, I calculated, for all major league games when a shortstop other than Jeter was playing the field

1. the number of ground balls hit by left-handed batters (LGB) and corresponding ground outs by shortstops (LGO6) and pitchers (LGO1) at Yankee Stadium and not at Yankee Stadium, and
2. the same for right-handed batter ground balls (RGB) and outs at short (RGO6) and pitcher (RGO1).

This calculation was done for all seasons Jeter played full-time for which the Retrosheet ground ball data was available (1996-99; 2003-12).

I then calculated the net plays at Yankee Stadium relative to what they were at all other stadiums. These calculations are just like the calculations of team net plays at a position relative to the league, except what is being compared is plays at Yankee Stadium (YS) by non-Jeters versus plays by non-Jeters at Other Stadiums (OS). The calculation for net YS_LGO6, given YS_LGB, is:

YS_LGO6|ys_lgb = YS_LGO6 – YS_LGB*(OS_LGO6 ∕ OS_LGB).

Doing the same calculations for LGO1, RGO6 and RGO1, we obtain the following net plays at Yankee Stadium relative to other stadiums for all ground balls hit when Jeter was not playing. Again, none of these calculations involves Derek Jeter at all.

Recall that regression analysis showed that pitcher fielding impacted expected plays at short.

When we reduce (increase) YS_LGO6|ys_lgb by each positive (negative) 0.1 YS_LGO1|ys_lgb and reduce (increase) YS_RGO6|ys_rgb by each positive (negative) 0.3 YS_RGO1|ys_rgb, we obtain regression-adjusted LGO6 and RGO6 at Yankee Stadium (r_YS_LGO6 and r_YS_RGO6).

When we divide those regression-adjusted numbers by the number of LGB and RGB at Yankee Stadium, we obtain the following estimates of the percentage of shortstop plays "taken away" from non-Jeter shortstops when they were playing at Yankee Stadium, calculated separately by batter-hand:

As can be seen, Yankee Stadium seemed to help shortstops other than Derek Jeter from 1996-1999. If you look just post-2003, about 2% lgb were 'lost' by shortstops and 1% of rgb. Like all park factors, it varied a bit year-by-year, so I simply took the average and applied it to the number of lgb and rgb when Jeter was playing at Yankee Stadium, not only for 2003 onward, but also 2000-02, just in case.

Conclusion

Ignoring Derek Jeter, we looked at shortstop and pitcher ground outs against left- and right-handed batter ground ball at Yankee Stadium and all other stadiums. We then backed out the effect of pitcher ground outs by batter hand, to get net plays at short at Yankee Stadium relative to the rate at other stadiums, separately by batter hand. We divided those net amounts by the total number of ground balls for the non-Jeters at Yankee Stadium by batter hand.

That yielded estimates—completely independent of Jeter—of the percentage of left- and right-handed batter ground ball outs at short 'taken away' by Yankee Stadium after taking into account pitcher fielding. The average impact was only negative for the post-2003 seasons. We took that average negative percentage impact (2% of all LGB and 1% of all RGB) and applied it to Derek based on the number of LGB and RGB 'faced' by Derek at Yankee Stadium each season, not only from 2003-12, but also, just to be nice, for 2000-02 and also for 2014.

Part C

Baseball-reference.com and Fangraphs.com are the two best-known sabermetric on-line baseball encyclopedias. Both sites report Sean Smith's "TotalZone" estimated defensive runs for 1995-2002 and Baseball Info Solutions (BIS) Defensive Runs Saved (DRS) for 2003-13. The combined TotalZone and BIS totals show Jeter has allowed −246 runs as a shortstop. Each single allowed is equal to the value of the hit allowed (about .47 runs) and the value of the out that was missed (about .27 runs), or about .74 runs.

So the consensus estimate is the equivalent of −332 singles allowed, which is about 72% of the latest DRA estimate above.

The short answer why the DRA estimate is very likely to be much closer to the truth is that the TotalZone estimates for 2000-02 use math that systematically 'shrinks' estimates of fielding runs allowed or saved at all positions (makes bad fielders look not so bad and good fielders not so good), while the TotalZone and BIS estimates for 1995-99 and 2003-14 used data that likely systematically shrank fielder ratings, except for very recent seasons.

Biased math

The TotalZone estimates for 2000-02 are based on publicly available data from Retrosheet.org and the description for estimating hits allowed for middle infielders is as follows: "For example, if 30% of a batter's [career] outs are hit to shortstop, then every time that batter gets a [clean] hit the shortstop is charged 0.3 hits." (Emphasis added.) Under this method, the quality of the Yankee fielders at all positions impact the rating of Derek. As discussed above, there is some evidence that pitchers can take a few plays 'out of the hands' of the shortstop. But the other positions can't. It makes no sense that every time a first baseman or center fielder 'saves' a hit, Derek gets .3 plays credit. And assuming for the sake of argument that the Yankee teams were, aside from Derek, exactly average, this approach only charges Derek 0.3 hits allowed for each single he allows.

So the TotalZone estimates for 2000-02 (and also for the 1950s through 1988, which use the same method), cause individual fielder ratings to be biased by team fielding performance, and even if a player plays on a team with otherwise totally average fielding quality, that fielder's performance will be drastically 'haircut'.

Biased data

TotalZone for 1989-99 and DRS for 2003-13 use something I and some others call "batted ball data." Batted ball data is data about the location and trajectory (or speed) of every batted ball hit into the field of play. In theory, batted ball data should give the most direct, precise and objective estimate of hits 'saved' or 'allowed'. For ground ball defense by shortstops, all you have to do is compare the average major league shortstop 'success' percentage for each location of the infield he covers to that of the shortstop you are trying to rate.

For example, let's say you divided the infield into 8 pie slices radiating out from home plate: the "5" slice inside the third base line, covered by third basemen, a "56" slice for the hole between third and short, a "6" slice covered by shortstops, a "6M" slice that is up the middle but on the shortstop side, a "4M" slice that is up the middle on the second baseman's side, a "4" slice covered by the second baseman, a "43" slice in the hole between second and first, and a "3" slice inside the first base line, covered by the first baseman.

With very rare exceptions, shortstop ground ball plays would only occur in the 56, 6 and 6M slices. You would count the total number of ground balls hit 'inside' each slice throughout the major leagues, the total number converted by all major league shortstops, and calculate the average success rates. Just making up some numbers, let's say major league shortstops converted 25% of 56 grounders into outs, 85% of 6 grounders into outs, and 25% of 6M grounders into outs. Then all you have to do to estimate Jeter's hits allowed is count the ground balls hit into each of those slices while he was playing, applying the relevant percentages to get expected ground ball outs for Derek, and subtract that number from the ground ball plays he made.

Unfortunately, batted ball data is

(1) not publicly available during Derek Jeter's career, except for his first four (three full-time) seasons, and

(2) subject to human scorer biases that tend to compress fielder ratings (making bad fielders look less bad and good fielders less good).

Batted ball data was collected by volunteers at an organization called Project Scoresheet from 1989 through 1999, which 'sliced up' the field in much the way describe above. That group disbanded and the collection of batted ball data went private, so the information was no longer open for public inspection. One of the new companies involved in collecting such data, Baseball Info Solutions, provided its data from 2002-05, which was much more 'granular' regarding locations than the 8-slice version described above, to Harvard and Stanford PhD statisticians Shane Jensen, Ken Shirley and Abraham Wyner, for purposes of writing an article in The Journal of Applied Statistics. (The authors were not free to release the data to the pubic.) The article, which is available here,

http://www-stat.wharton.upenn.edu/~stjensen/pape…,

included density plots showing where ground balls were coded as being hit (see the third page in the file, numbered 493 in the original academic publication). The surprising thing I noticed, and confirmed with the authors, is that the coded locations of ground balls (and also balls hit into the air) were 'clumped' around each fielding position rather than spread randomly throughout the infield.

But batted balls do not leave the bat seeking fielders.

Why this distortion in the coding of location of batted balls? Try imagining you had to code batted ball locations while watching the game on video, which the BIS coders do. Even if you were able to replay the video as often as you liked, the problem would be that the camera does not stay still, but follows the ball. Very often all frames of reference for distance and direction are lost other than the player fielding or attempting to field the ball. Sometimes all you see is the ball and the player against a solid field of green grass or, in the case of an infielder, green grass and the curve of the infield dirt, without seeing the nearby bases.

So if the ball is caught, the coder might be inclined to code the batted ball location nearer to the starting position of the player. If the ball is not caught, the coder might be inclined to code the ball as being farther away from the starting position of the player. Since 70% of all balls are caught, you'd see clumping of coded batted ball locations around each position.

If you are coding batted ball locations at the ballpark, where at least some, and maybe all Project Scoresheet batted ball data was collected, you'll have all sorts of "parallax" problems in estimating directions and distances. For example, if you're sitting in field-level seats behind first base, you might have a good sense of the distance but not the direction of line drives hit between center and right, and a good sense of direction of line drives hit between left and center, but not the distance. And even if you are high up in the bleachers directly behind home plate, you only get one look—no instant and repeatable replays.

Given these limitations, I would guess, though I haven't checked, that Project Scoresheet data from 1989 through 1999 is also subject to some bias toward coding caught batted balls closer to the original fielding position and clean hits as farther away.

How would this impact fielder evaluation? Well, if a poor fielder without range doesn't reach the ball, it will likely be coded farther away, in a zone of low out-conversion probability, so he won't be penalized as much for missing it. And if a great fielder with range reaches a ball farther away, it will tend to be coded as being closer and easier to field, so he won't be credited as much for grabbing it.

After alerting fans to this bias in BIS data in my article in The Hardball Times Baseball Annual 2012, BIS published The Fielding Bible III, which reported that BIS had gradually fixed this data bias problem over the next few seasons following the seasons studied in the academic article.

So the bottom line is that batted ball data, even if it has been improved in the last few years, very probably had a significant bias over most of Jeter's career that would result in 'compressing' fielder ratings toward the mean—overrating immobile fielders and underrating rangy ones.

Michael Humphreys lives in Palo Alto, California, and advises on international tax matters at Ernst & Young LLP. He is the author of Wizardry: Baseball's All-Time Greatest Fielders Revealed (Oxford University Press 2011).

Top photo via Getty