Last week I started posting the expected points added (EPA) for Week 1 by quarterback. The total figure aggregates the EPA on all the different types of play a quarterback was involved in: passes, interceptions, scrambles and runs. Sam Bradford led all Week 1 passers in total EPA and EPA per play, and his efficiency mark still stands at the top after missing Week 2.
One of the more well-known quarterback statistics that uses EPA in its calculation is ESPN’s Total Quarterback Rating (QBR). But QBR isn’t as simple as totaling up the various EPA figures, it adjusted credit/blame for the quarterback on various plays and also has a “clutch-weighting” on individual plays based on how close the score is at that time. I like QBR in concept, and I believe the analysis built into the calculation is likely sound. But, when there is a confusing or counterintuitive result in the final QBR numbers, we’re left guessing as to why QBR differs so much from fan impressions, or even the raw EPA figures.
Has Deshaun Watson been a top QB?
The biggest head-scratcher in the year-to-date QBR figures through Week 2 is Deshaun Watson coming in second, despite playing fairly poorly according to most who observed his game-and-a-half of play. Watson’s adjusted net yards per attempt – which adjusts for touchdowns, interceptions and lost sack yards – is currently 2.85 ANY/A, or roughly equal to Jared Goff in 2016 (2.82 ANY/A).
Watson’s QBR varies substantially from the figure I calculated through the raw EPA numbers. I have Watson’s total EPA at -12.0, and his per play EPA at -0.19, both fifth worst among 33 qualifying QBs. I’ll acknowledge that my passing and sack numbers are more a reflection of the entire passing offense than just the quarterback, but that can’t explain the whole difference.
[table id=41 /]
Why the discrepancy?
It’s understandable that there would be some differences between the raw EPA numbers and more refined QBR, but fifth-worst to second-best is an enormous jump.
One of the big figures driving Watson’s QBR is his rushing EPA, which tops all quarterbacks at 7.0. The obvious driver of that number was Watson’s hugely important 49-yard rushing touchdown in a close game against the Bengals last Thursday. But that run is also reflected in the raw EPA numbers I calculated, giving Watson 8.1 EPA from scrambles and runs, much more than second place Russell Wilson at 5.8 EPA.
The real discrepancy comes in the passing numbers, where QBR has Watson at -0.2 EPA, and my numbers see his combined value on passing plays[note]Passes (including interceptions) and sacks[/note] at -20.3 EPA.
How could the numbers be so different? While ESPN has an explanatory primer what goes into QBR, it doesn’t give the type of details to allow us to that question. My best guess is that the clutch-weighting heavily discounted Watson’s difficult performance in Week 1, where he entered after halftime already down 19 points to the Jaguars. Not only does QBR seem to have discounted Week 1, it seems to have almost wiped it off of the calculation entirely.
I get that it’s much more important how a quarterback performs when the score is close, but should we be fully discounting performance in games with one team leading by multiple scores? While it may not have a material effect on the team’s ability to win the game, I find it hard to believe that how a quarterback plays in those situations says nothing about who that quarterback is. At the very least, we’re limiting our sample of meaningful data in the calculation.
While I continue to be a QBR defender, it’s going to be tough sledding to argue its accuracy and relevance when quarterbacks who the vast majority of fans would say struggled, like Deshaun Watson, find themselves near the top of the leaderboard. I don’t think having Watson near the top is necessarily wrong, but the black-box nature of the QBR calculation leaves us unable to make that determination and wondering how it happened.