How Is Deshaun Watson Currently Second in ESPN’s QBR?

Last week I started posting the expected points added (EPA) for Week 1 by quarterback. The total figure aggregates the EPA on all the different types of play a quarterback was involved in: passes, interceptions, scrambles and runs. Sam Bradford led all Week 1 passers in total EPA and EPA per play, and his efficiency mark still stands at the top after missing Week 2.

One of the more well-known quarterback statistics that uses EPA in its calculation is ESPN’s Total Quarterback Rating (QBR). But QBR isn’t as simple as totaling up the various EPA figures, it adjusted credit/blame for the quarterback on various plays and also has a “clutch-weighting” on individual plays based on how close the score is at that time. I like QBR in concept, and I believe the analysis built into the calculation is likely sound. But, when there is a confusing or counterintuitive result in the final QBR numbers, we’re left guessing as to why QBR differs so much from fan impressions, or even the raw EPA figures.

Has Deshaun Watson been a top QB?

The biggest head-scratcher in the year-to-date QBR figures through Week 2 is Deshaun Watson coming in second, despite playing fairly poorly according to most who observed his game-and-a-half of play. Watson’s adjusted net yards per attempt – which adjusts for touchdowns, interceptions and lost sack yards – is currently 2.85 ANY/A, or roughly equal to Jared Goff in 2016 (2.82 ANY/A).

Watson’s QBR varies substantially from the figure I calculated through the raw EPA numbers. I have Watson’s total EPA at -12.0, and his per play EPA at -0.19, both fifth worst among 33 qualifying QBs. I’ll acknowledge that my passing and sack numbers are more a reflection of the entire passing offense than just the quarterback, but that can’t explain the whole difference.

PlayerTeamEPAPlaysEPA/PPassesPass EPAPass EPA/PIntsInt EPAInt EPA/PSacksSack EPASack EPA/PScrmblsScrmbl EPAScrmbl EPA/PRunsRun EPARun EPA/P
D.WatsonHOU-1263-0.1946-0.501-6.8-6.87-13-1.969.21.53-0.9-0.3

Why the discrepancy?

It’s understandable that there would be some differences between the raw EPA numbers and more refined QBR, but fifth-worst to second-best is an enormous jump.

One of the big figures driving Watson’s QBR is his rushing EPA, which tops all quarterbacks at 7.0. The obvious driver of that number was Watson’s hugely important 49-yard rushing touchdown in a close game against the Bengals last Thursday. But that run is also reflected in the raw EPA numbers I calculated, giving Watson 8.1 EPA from scrambles and runs, much more than second place Russell Wilson at 5.8 EPA.

The real discrepancy comes in the passing numbers, where QBR has Watson at -0.2 EPA, and my numbers see his combined value on passing plays1 at -20.3 EPA.

How could the numbers be so different? While ESPN has an explanatory primer what goes into QBR, it doesn’t give the type of details to allow us to that question. My best guess is that the clutch-weighting heavily discounted Watson’s difficult performance in Week 1, where he entered after halftime already down 19 points to the Jaguars. Not only does QBR seem to have discounted Week 1, it seems to have almost wiped it off of the calculation entirely.

I get that it’s much more important how a quarterback performs when the score is close, but should we be fully discounting performance in games with one team leading by multiple scores? While it may not have a material effect on the team’s ability to win the game, I find it hard to believe that how a quarterback plays in those situations says nothing about who that quarterback is. At the very least, we’re limiting our sample of meaningful data in the calculation.

Conclusion

While I continue to be a QBR defender, it’s going to be tough sledding to argue its accuracy and relevance when quarterbacks who the vast majority of fans would say struggled, like Deshaun Watson, find themselves near the top of the leaderboard. I don’t think having Watson near the top is necessarily wrong, but the black-box nature of the QBR calculation leaves us unable to make that determination  and wondering how it happened.

  1. Passes (including interceptions) and sacks

2 thoughts on “How Is Deshaun Watson Currently Second in ESPN’s QBR?

  1. Pingback: Weekly Sports Analytics News Roundup - September 26th, 2017 - StatSheetStuffer

  2. Anonymous

    He is now number 1 in qbr. Maybe qbr adjust not just for point differential but point differential when entering the game.

    Since the backup QB did not receive snaps early in the game when the score was close. Discounting the value of snaps when the team is down and in passing situation when new QB enters the game.

    Or.
    Every play in a 0-0. (tie) game has a higher potential value that an equal play in a 0-17 game.

    A perfect 100 qbr is only possible in a competitive game. I would discount the weight of the samples (snaps) taken when entering a game significantly behind. —

    think of it this way.

    Starting pitchers era in the sixth inning is the highest era inning in baseball. If you only look at a pitchers era in later innings it would skew your data. The defense can stack against the pass when a team is down by 17.
    If you only look at that players data on passing downs compared to other players you have a sample bias.

Leave a Reply