Introducing True Sack Rate (TSR): A film project to quantify sack production

From a team perspective, all sacks are positive for a defense, but they can mean very different things for an individual pass-rusher making the play. Depending on a variety of different factors, player evaluation based on raw sack statistics can often be misleading.

Players can add a ‘sack’ on the stat-sheet after the quarterback handles the ball for too long (either due to great coverage or the QB failing to recognize the opening in the defense), as a result of a protection breakdown, by tackling the QB at the line of scrimmage, when they’re being blocked, or by soundly beating the blocker in front of him due to superior athletic ability and/or technique.

By differentiating and qualifying these different pathways to notching a sack into four different categories — using film study, charting, and a simple grading scale — we can gain valuable insight into which players are earning their production vs. which players are beneficiaries of fortunate circumstances.

True Sack Rate (TSR) is a project aimed at studying and charting every sack of the NFL season recorded by an edge-rusher (DE/OLB), with the goal of identifying which edge-rushers utilized the highest level of skill against the most difficult set of circumstances to record a sack. Through this process, we will be able to more accurately evaluate sack and pass-rush production as a whole for the edge-rusher position.

The TSR will provide us with each rusher’s “sack score,” based on a point system that specifies four different types of sacks and forced fumbles, as outlined below:

1.25 points: Rare High Quality (RHQ) Sack - A 1-on-1 win over a very good (Ex: Duane Brown) or elite (Ex: Terron Armstead) blocker due to the rusher’s skill, move(s) and/or athletic ability.

Example: (2019) Cowboys DE DeMarcus Lawrence beating Lane Johnson 1v1 with his signature cross-chop move for a strip/sack.

1.0 points: High Quality (HQ) – A 1-on-1 (or 1-on-2) win over an above average (or below) blocker due to the rusher’s skill, move(s) and/or athletic ability.

Example: Chargers DE Joey Bosa stringing together multiple moves (side-scissors) to win 1v1 vs. RT Elijah Wilkinson.

.5 points: Low Quality (LQ) – A sack coming as a result of being unblocked or a scheme such as a twist or stunt, in which no special skill or move was required in order to record the sack.

Example: Broncos edge-rusher Bradley Chubb getting a free run at the QB due to a linebacker blitz occupying the left tackle.

.5 points: Coverage / Cleanup Sack – An effort sack coming as the result of excellent secondary work or a quarterback hanging onto the ball for too long.

Example: Former Vikings and current Ravens DE Yannick Ngakoue getting shut down by the left tackle before retracing and sacking the QB due to him holding onto the ball for too long.

Both a low-quality and coverage/cleanup sack are valued and should be viewed similarly, but the distinction between the two is another layer of context in the evaluation process.

.5 points: Per forced fumble (strip/sack) - Stripping the ball and creating an opportunity for a turnover is as valuable as getting a low quality or coverage/cleanup sack.

NOTE: Beating tight ends or running backs can still be considered a high quality sack if a special skill was used to defeat the blocker. A high quality sack can also still occur on a stunt/twist, so long as the pass rusher wins a 1-on-1 battle against a blocker.

Example of a high-quality sack on a stunt: Chargers OLB Melvin Ingram using excellent lateral quickness, speed, and bend to cross the LG’s face.

There are eight additional categories I charted for each player on every sack that bring more context into the production and evaluation of the individual pass-rusher. The move the rusher used, whether they forced a holding penalty, the opponent, down/distance, and if the sack ended the drive are all valuable data points to help paint a clearer picture on their impact.

Notes:

- Only edge-rushers with ≥100 snaps OR ≥2 sacks were charted

- All stats are via Pro Football Reference.

- Half sacks were counted as full sacks if it was determined that the rusher beat the blocker and was going to likely take down the quarterback without the help of a teammate.

- Recorded sacks when the quarterback got back to the line of scrimmage and didn’t lose any yardage were not counted, because of their relatively limited impact.. All sacks I counted required at least a loss of one yard or more.

- If the QB dropped 10 yards or more behind the line of scrimmage and an edge-rusher got the sack, the rush was more heavily scrutinized to determine if it was high or low-quality. The threshold that QBs are given on nearly every drop-back is between 9-9.5 yards. Anything past 9.5 yards, and the angle becomes increasingly difficult and unrealistic for tackles to match vs. rushers. If the QB drops beyond that depth, it’s then his responsibility to step up into the pocket or evade the rush on his own. This is admittedly a gray area, which sometimes makes  it difficult to assign a grade. In those rare situations, I reached out to at least one outside expert for an extra set of eyes to get their opinion for clarity.

- Over the last two seasons there have been around 75 sacks per week with about a 1/3 of those from edge-rushers. The weekly positional breakdown of sacks can obviously vary, but that’s about the average of what I’ve been seeing so far in 2020. That means that over the course of a season, there are 1,250-1,300 total sacks and about 425-510 (25-30 per week) from edge-rushers that I will be studying and charting.

Now for the fun stuff. Let’s get into some of the results and takeaways through the first 10 weeks of the 2020 season.

Listed below is every edge defender (4-3 defensive end or 3-4 outside linebacker) in the NFL with a ‘sack score’ of at least ‘3’. You’ll also see their snap count, snap percentage, average number of snaps per sack (Sack / Snap), and average number of snaps per HQ sack (HQ Sack / Snap).

If you’re looking for only HQ sacks, that list can be found below, featuring every edge defender with at least 3 HQ sacks on the season:

Here are the (somewhat) surprising results for each metric:

• The top 5 leaders in most sacks per snap (minimum of 200 snaps):
  1. Ryan Kerrigan - 1 sack every 42.2 snaps
  2. Trey Hendrickson - 1 sack every 44.3 snaps
  3. Myles Garrett - 1 sack every 49.2 snaps
  4. Za’Darius Smith - 1 sack every 56.1 snaps
  5. T.J. Watt - 1 sack every 56.3 snaps

• Top 5 leaders in most HQ sacks per snap (minimum of 200 snaps):
  1. Myles Garrett - 1 HQ sack every 55.7 snaps
  2. Ryan Kerrigan - 1 HQ sack every 105.5 snaps
  3. Joey Bosa - 1 HQ sack every 109 snaps
  4. Derek Barnett - 1 HQ sack every 109.3 snaps
  5. Za’Darius Smith - 1 HQ sack every 112.3 snaps

Some other interesting context to note: 

• Trey Hendrickson has 8 total sacks and the 2nd best rate of overall sacks per snap, but only has 1 HQ sack with the other 7 being LQ or of the coverage/cleanup variety. (Qualifying sack production and identifying nuggets like this is exactly why I wanted to build this metric).

• Ryan Kerrigan is only playing 37% of snaps for Washington but is still making a notable impact on the quarterback, ranking 1st in most sacks per snap and 2nd in HQ sacks per snap (5 total sacks, 2 HQ).

• Adrian Clayborn is only playing in 27% of snaps for the Browns but has a couple sacks, both of which came from his signature cross-chop move and both being HQ.

• Za’Darius Smith is quietly having a very productive year when it comes to sacking the QB and is cementing his status as a rare edge-rusher that is more effective rushing over the guard than tackle, with 6 of his 8 sacks coming as a 3-technique (2 of those 5 on stunts & all 6 using power moves).

• And maybe most important: Myles Garrett is an alien. Garrett is the best edge-rusher in the NFL through 10 weeks and is head and shoulders above his peers when it comes to sack score & HQ sacks. Garrett has 5 more HQ sacks than 2nd place Joey Bosa’s 5, and at least 6 more than every other edge-rusher in the league. Garrett also has more HQ sacks this season (10) than the 6 other edge-rushers drafted in the first-round with him in 2017 have combined (T.J. Watt, Takk McKinley, Charles Harris, & Derek Barnett have 7 total).

  

• The only rushers in the NFL this season to record an RHQ (again, defined as a 1-on-1 win over a very good or elite blocker due to the rusher’s skill, move(s) and/or athletic ability) are Joey Bosa (2), Myles Garrett (1), Khalil Mack (1), and Uchenna Nwosu (1). Nwosu is 13th in sacks per snap (1 sack per 65.5 snaps) with 4 total and his RHQ sack was on a bull-rush against Terron Armstead in Week 5.

• Notable edge-rushers without a single sack yet this season are Melvin Ingram, Jadeveon Clowney, Celin Ferrell and Arden Key.

• Montez Sweat has 5 sacks and all are LQ.

• There are just four rushers that have played 70% or more of their teams’ snaps and only recorded a single HQ sack: Ifeadi Odenigbo (71% of snaps), Kyler Fackrell (79% of snaps), Preston Smith (81% of snaps), and Tanoh Kpassagnon (81% of snaps).
  
  A few housekeeping notes before we wrap this first installment up:
  

• This will be updated weekly in-season, and you should expect a series of articles in the off-season breaking down different ways to interpret the data when I have more time to dissect it.

• In the spring I plan on going back over the entire 2019 season and charting this same information for edge-rushers so we have another year’s worth of data to study.

• I would like to expand this to defensive tackles in the off-season and at least get the 2020 season done so we can start understanding interior sack production better and uncovering more nuggets in the evaluation process.

  I want to end this first installment by saying thank you to Jon Ledyard and letting you know that this idea originated from his 2016 series called “Contextualizing Sack Production” on InsidethePylon.com. This was something Jon and I used to discuss back then in a lot of detail but he was the one that wound up starting the project and it was always something I thought brought tremendous value into the evaluation process for pass-rushers, so I am bringing it back while changing many of the details, altering the grading scale, charting different information, and of course studying every sack myself.