Note: Given recent events surrounding the Bears QB situation, I am putting the Russell Wilson analysis on hold unless something material occurs.
Draft season is the best time of year for teams who are not true contenders. Every fan base is full of hope, dreaming of finding that sixth-round QB who will go on to win seven Super Bowls in his 57-year career. Given that we are now just one month away from NFL Draft Day in Cleveland, I am sure many people are looking over Big Boards from Kiper, McShay, or PFF searching for their late-round gem or trying to figure out why nobody went to Bo Callahan’s birthday party. I’m sure Ryan Pace has Draft Day on repeat, taking notes on how Sonny Weaver Jr. was able to get all those picks AND David Putney.
But how often do we actually look back at Big Boards from previous years in order to get a feel for which sources are more or less reliable?
Over the past two drafts (2019 and 2020), I have been tracking Big Board data, and recently decided to try to find and answer just how good these analysts’ scouting abilities really are. I have tracked Big Board data for:
- Mel Kiper Jr. (ESPN) Scouts Inc. (ESPN)
- Todd McShay (ESPN)
- The Athletic – Consensus Big Board
- The Athletic – Dane Brugler
- Lance Zierlein (NFL.com)
- The Draft Network (TDN) – Consensus Big Board
- USA Today – The Draft Wire
- Chris Trapasso – CBS Sports
- Bleacher Report (B/R) – Matt Miller
- Note: Matt Miller recently left B/R and is now running his own site at TheDraftScout.com
ProFootballReference.com’s AV has been used heavily in this analysis. Pro Football Reference describes AV as “putting a single numerical value on any player’s season, at any position, from any year.”
AV is awarded to offensive players based on team points per drive relative to league average and divided up among each player. A similar strategy is used for defensive players using points against. Adding to that figure, points are awarded for being named an All-Pro, or Pro Bowl at certain positions. A more complete definition can be found here.
I broke down each NFL Draft class by each player’s AV big board rankings and draft slots. In the chart below, you will see each Big Board source (i.e. ESPN – Kiper), and a percentage under each grouping (i.e. Top 5, Top 10, etc.). This can be described as follows:
- Matrices for “ESPN – Kiper” & “Top 10” – Kiper’s top 10 players in his Big Board for 2019 totaled AV of 136.
- The 10 Best Players Available in the Draft Class (BPA) totaled AV of 196.
- Therefore, Kiper’s grade for the “Top 10” is calculated by taking his Big Board AV divided by the BPA AV: (136 Big Board AV / 196 BPA AV = 69%).
- Also in the table for reference is a row called “NFL Draft”.
- This row shows the Total AV of the first 10 players drafted, divided by the BPA AV: (149 AV Drafted in Top 10 / 196 BPA AV = 76%).
2019 NFL Draft:
In 2019, none of the above analysts “Beat the Draft”, so to say. The Average Grade is a straight average of each percent grade in the columns to the left. I had considered doing a weighted average here to highlight the importance of getting the earlier rankings correct, but I felt that it became too messy to be able to explain clearly. ESPN dominated in these rankings between Kiper, McShay, and Scouts Inc, with The Athletic Consensus just behind them.
I did notice that in 2019, Todd McShay’s big board was almost the exact same as that of Scouts Inc. Nobody beat the draft in the top 5, top 10, or top 25. But we did see Scouts Inc & McShay shoot par in the Top 10, and we also see that Kiper and The Athletic Consensus beat the draft for the top 50 prospects.
We see much bigger miss rates in the Top 5 & Top 10 than we do at Top 25 and beyond. This is because the margin for error is so much thinner in the Top 5 & Top 10. One miss can completely drop a grade here.
2020 NFL Draft:
In 2020, seven analysts beat the draft in the Top 5, with five analysts beating the draft in the Top 10, and four beating the draft in the Top 25. In the end, we see three analysts who beat the draft in average grade:
- Mel Kiper Jr.
- The Athletic Consensus
- TDN Consensus
- Honorable Mention to Dane Brugler shooting 0.2% off of Par. Impressive for a single individual doing all his own scouting as opposed to the consensus big boards.
The combined result of the two draft classes / big boards can be seen in the below chart:
As shown above, the combined results show that no analyst was able to beat the draft in average grade, with the closest being Kiper, falling 2.3% below actual drafted results (it’s got to be the hair). Here are a few final notes for the combined draft classes/big boards:
- Kiper beat the draft in the Top 10 (+0.6%), in the Top 25 (+0.5%), and in the Top 50 (+1.0%). But he fell off in the Top 100 (-0.9%), Top 150 (-1.8%), and Top 200 (-3.7%). The Athletic Consensus beat the draft in the Top 50 (+1.7%) but trailed the draft in every other window.
- Dane Brugler’s Top 200 is the most accurate Top 200 Big Board as far as listing the most players to find success in the NFL. His Top 200 achieved 96.8% of the BPA AV, which is a ridiculous 3.1% ahead of what NFL teams drafted. No other analyst came within 5% of his score for the Top 200.
- Brugler would likely be at the top of the list if it were not for his misses in the Top 5 and Top 10, where he was below average by a solid margin. If you have never heard of the Draft Guide he puts out before every draft, I would highly suggest taking a look if you are a subscriber to The Athletic. It is 250 pages of nerding out on draft prospects.
Next time, I will be looking at how each analyst has done in ranking position groups (QB, RB, WR, TE, OL, DL, EDGE, LB, DB). Below is a preview showing each analyst’s grades for their top 100 offensive players and top 100 defensive players.
Listen To Bears On Tap Podcast
BET WITH BETRIVERS ILLINOIS
More Bears News
/ 4 days ago
Identifying trends and outliers from the 2020 NFL season and examining how we can...
/ 1 week ago
There have been conflicting reports regarding the Bears' starting QB. Here is a breakdown...