Naturally, President of Football Operations/GM John Elway takes most of the heat for the Denver Broncos' post-Super Bowl 50 failures. The No. 1 gripe from those disgruntled fans and reporters is Elway's perceived lack of success in the draft.
However, perception is not always reality. Those who say Elway is bad at drafting can easily point to the players who obviously didn’t pan out. Those that argue against that viewpoint can point to the players who were a success.
Which side is more correct, though? Is Elway bad at drafting? Analytics will provide the answer.
In order to ascertain whether Elway is good or bad at the draft, he must be compared to something. That comparison starts with the other 31 NFL teams.
Every player drafted since 2011 (Elway’s first draft) has been analyzed so that we can see how well the other teams performed and use that performance as a comparison to Elway’s performance over his tenure.
Next, a comparable metric(s) must be defined. The importance of establishing comparable metrics is to remove ambiguity and bias. As an example, if one were to count the number of Pro Bowls a player has been invited to as a metric to understand value, that would introduce bias because the Pro Bowl voting is in itself inherently biased.
It's also ambiguous because there has never been a defined set of credentials that make a player 'Pro Bowl-caliber' for the voters to use in their selection process.
The metric used in this analysis is called Player Performance Value. This value is used to indicate how well drafted players performed on the field in comparison to other players. More specifically, compared to other players at their same position and/or compared to players who were drafted in the same round. This metric starts with Pro Football Reference’s Car AV value.
PFR uses the same methodology for each player to establish Car AV (they have provided details of their calculation, so that it's not ambiguous). The importance of this is consistency. By applying their computation equally to each player, Car AV can be used to compare players, especially when comparing players to others at the same position.
I then divided a player’s Car AV by the number of seasons a player has been in the NFL. This normalizes the value so that longevity is not the driving factor. Also, by doing this, missed seasons due to injury are part of the equation.
Next, a player’s Car AV per season (now known as Player Performance Value for this analysis) is compared to the average PPV for all drafted players at their position. That value is also compared to other players drafted in the same round regardless of position.
Each player was put into several groups depending on how much better or worse they are from the average Player Performance Value. When reviewing these groups, remember one thing; performance is different than potential. A player may have elite traits, but if injuries have kept them off the field, they are unlikely to be placed in an elite level group. The groups are:
Below Average Players: This is quite self-explanatory. It is any player whose PPV is below the average for all players at their position. These players are those who never saw the field, played very little, or their contribution to the teams was minimal when they did play. An example is Paxton Lynch. By most measures, he was a bust.
Above Average Players: This is any player whose PPV is equal to or greater than the average for all players at their position, but less than two times the average for all players at their position. These are role players and spot starters who can be easily replaced, but have contributed to the team. An example is Devontae Booker. He is a backup who has been a spot starter. He is a player who can be replaced without much pain to the team.
Starting-Caliber Players: This is a player whose PPV is equal to or greater than two times the average, but less than three times the average for all players at their position. A player like this is someone who has been a regular starter because they have been good enough to do so, not because they were the best option out of a few mediocre options. This level could also represent a young player who hasn’t realized his full potential, but might be on the upward swing to the next level. An example of this caliber of player is Julius Thomas. He was a solid starter and his skills were missed when he left via free agency.
Star Players: This is a player whose PPV is equal to or greater than three times the average, but less than four times the average for all players at their position. This is a player a team can’t live without and is very hard to replace. Courtland Sutton has already become an example of this level of player. If he were to suddenly leave the team, it would be painful and the team would have difficulty finding his replacement.
Elite Players: This is a player whose PPV is equal to or greater than four times the average. This is player who is the best of the best, a potential Hall of Fame career type. The example is Von Miller.
A note to remember, players land in these groups because they are compared to others at their position, not to all players. Andy Janovich is compared to other fullbacks, not to quarterbacks.
The last grouping is done so that players can be compared to others drafted in the same round regardless of position. This group is any player whose Player Performance Value is at least 1.5 times better than the average for all players drafted in the same round. In this scenario, Janovich is compared to fullbacks, quarterbacks, and any other position drafted in the same round he was selected (round 6).
For the analysis, the number of players drafted in each group is divided by the total number of players drafted by the team so that it can be compared across all teams who have selected a varying number of draft picks.
What happens next for the Broncos in free agency & the draft? Don't miss out on any news and analysis! Take a second and sign up for our free newsletter and get breaking Broncos news delivered to your inbox daily!
Elway has been very good at finding players who can contribute. He ranks fourth in the NFL at finding Above Average Players. 28.6% of the players he has drafted are Above Average and those players, although not spectacular, do contribute to the team.
Elway is also good at not whiffing often. He has selected Below Average Players only 57.1% of the time. Only eight other teams have been better than him at avoiding Below Average Players. This is also a good indication of how difficult it is drafting NFL players (see this article for more information).
Success with those two groups indicates Elway is one of the best at finding players who can at least contribute to the team. He has had some misses, but he is better than most at finding NFL talent. Where he has failed is in finding players in the next two groups.
Elway has had difficulty at finding Starting-Caliber Players or Star Players when compared to the 31 other teams. Only 10% of his draft picks have been true Starting-Caliber Players, ranking him just above the bottom-third of the league.
Worse yet, Elway has only hit on Star-Caliber players on 2.9% of his draft picks, ranking him again in the bottom-third. A team cannot live on just contributors alone and missing on these two groups can leave significant voids on a roster.
Elite Players are like finding a needle in a haystack. To his credit, Elway has not been blanked in this group like several other teams. However, his 1.4% success rate has him about middle-of-the-pack.
See the below graph for reference.
However, to have ultimate success in the NFL, getting it right at certain positions is critical. For example, a team could draft a star-caliber guard and fullback, but miss on the quarterback and edge rushing positions. Although the team is successful in getting good players, it’s not successful in the more important positions. Has Elway been good at selecting those more important positions?
By most accounts, the positions that have the most effect on the team’s success are QB, OT, CB, and Edge Rusher.
To understand how Elway has been at drafting positions, I used two groups for each position. The first group are players whose Player Performance Value is greater than or equal to the average.
The second group are players whose Player Performance Value is two times greater than equal to the average. The below graph indicates Elway’s percent of 'hits' at the position (blue bar) compared to the other NFL teams (black Gantt bar).
Elway nailed the fullback position when he selected Janovich. However, does that position have a tremendous effect on the team? I really like Janovich and he is a contributor, but he doesn’t command the offense.
When looking at the more important positions, Elway has been good at selecting at least above average players at Edge Rusher and OT. In fact, he is much better than the average than the rest of the league. However, he has missed on QB and CB. When looking at that next tier of players (at least 2x the average), edge rusher has been his only real win. He has whiffed on CB, OT and QB.
For the first group of players, Elway has been better than the average at selecting FB, Edge Rusher, OT, RB, S, iOL and DL. The second tier for players indicates he has been better at finding good players at FB, TE, iOL, ER, and S when compared to the other NFL teams. He finds solid players, he just needs to find them at the most important positions.
The next evaluation of Elway’s prowess in the draft comes from looking at the different rounds of the draft. Getting the first round right is of the utmost importance because most of the star to elite-level players do come from round one. However, it doesn’t matter which round a highly talented player comes from, a team just needs to find them.
Elway is in the middle of the pack when drafting first-round players at 12.5%. His overall drafting by round is also average. However, he has hit on players in every round. This is significant because several teams have missed completely throughout the draft.
This is an indication that Elway finds NFL level talent at all spots in the draft. See the graph below.
When looking at his drafts over time, Elway has had more draft wins than losses as the following graph indicates. Six of his drafts have been better than the average (2011, 2012, 2014, 2016, 2018, and 2019). Only three have been busts (2013, 2015, and 2017).
Elway is improving at drafting over the past four years. From 2016 to 2019, he is still highly ranked at finding Above Average Players and avoiding the Below Average Players. However, his ability to find Starting-Caliber Players has improved enough to move him from 21st to 18th.
The biggest leap has been in his ability to find Star Players. He now ranks in the top-third of the NFL. That is a significant improvement. See the below graph.
What it Means
Over his tenure, Elway has been very average at drafting. He is not terrible nor is he great. He has struggled to find QB, OT and CB, which hurts the team’s success on the field.
However, Elway has been able to select a nucleus of contributor-level to valuable players, which sets the Broncos up to be a very strong team, if he can get those three positions right. He also has found an elite-level edge rusher, which isn’t an easy task.
Elway has shown significant improvement in three of the last four drafts. As the players from the last two drafts gain more tenure, his drafting prowess will likely look much better, especially if Drew Lock is the real deal.
The answer to the question at hand is this; Elway hasn't been bad at the draft nor has he been good. However, the trajectory over the last four drafts indicates he is going to be one of the better GMs at finding those Starting-Caliber and Star Players.