Jacksonville Jaguars used scouting and stats in harmony during 2014 NFL draft
"The problem we're trying to solve is that there's rich teams, there's poor teams, then there's 50 feet of crap, and then, there's us. It's an unfair game. And now we've been gutted, we're organ donors for the rich ... and you guys are talking the same good body nonsense like we're selling jeans. Think differently. We are the last dog at the bowl. You see what happens to the runt of the litter? He dies."
The scene in the movie version of Moneyball, where Brad Pitt-as-Billy Beane sounds off on his scouts for their antiquated ways of thinking was an overdramatized version of the truth, but when professional sports franchises try to implement a serious use of metrics in order to change the odds in what can be an unfair game, there can be growing pains.
One team that did not experience those growing pains was the Jacksonville Jaguars, whose erosion of talent under former general manager Gene Smith was serious cause for concern to new owner Shad Khan when he bought the team from Wayne Weaver in November 2011. After a year of analysis and a 2-14 record that was the worst in franchise history, the new Jags made two important moves for the 2013 season -- replacing Smith with new general manager David Caldwell and bringing in former Seahawks defensive coordinator Gus Bradley to take over for head coach Mike Mularkey. Khan's son Tony had already been named to the position of senior vice president, football technology & analytics. And from the moment Caldwell and Bradley were hired in 2013, the younger Khan's passion for metrics as they applied to football performance became an overarching concept.
"It’s something we’re heavily involved in and dealing with right now," Caldwell told me at his first scouting combine as a general manager in 2013. "It’s just another part of the process -- it can help us confirm some of the things we already know and can raise some red flags on some of the things we don’t know. It pretty much confirms what we’re seeing."
When I brought up that scene in Moneyball to Tony Khan in a recent interview, he laughed and said that it hasn't really been repeated in Jacksonville's personnel meetings, because everyone understands the paradigm.
"Every once in a while, when you're trying to merge analytics and scouting, you're going to have those clashing moments. But for me, it's been great working with Dave and Gus, because they've been very open to ideas driven by analysis. It's something that Dave and I have talked about every day since he first came down to interview with my dad for the general manager position."
And it was USC receiver Marqise Lee, whom Jacksonville picked in the second round of the 2014 draft, who got the ball rolling in those conversations. Not the injured and sub-productive Marqise Lee of the 2013 season, but the first-year collegian of '11 who caught 73 passes for 1,143 yards and 11 touchdowns, and the guy who led the nation with 118 catches in '12. That was the jumping-off point for Khan and Caldwell -- the idea to use metrics in a way that would project and reflect sustained performance over time, and mitigate statistical bumps in the road.
"When Dave and I were in his office for the first time ever, we started talking about Marqise Lee. He was a player that ... I don't think either one of us thought, back when we first met in January 2013, was going to be on the board for us in the second round the next year. I brought statistics into the conversation then, because Marqise's numbers were so strong as a freshman and sophomore, it seemed like he was going to be a top pick.
"But I've brought statistics in through that process, as we've been discussing the draft and free agency, and Dave's asked some very good questions through the process. We've been able to introduce ideas that have helped us identify some pretty good players, and helped us find some red flags that some potential players might have had."
Khan didn't want to go too far into those red flags; he sees the work he's done as proprietary, and to a certain extent, it is. The Jaguars do their own charting in addition to the purchase of metrics from services like STATS, Inc., Football Outsiders and Pro Football Focus, so there were lines he preferred not to cross. But in a general sense, teams must marry stats to tape (and vice versa) in certain organic ways, or they're just paying lip service to the value of that marriage. One is a cross-check for the other.
"I've spent hundreds of hours with Dave over the last couple of years, while he's been discussing evaluations he's made from tape. We've done a lot of analysis with the raw data. So, I've got a good stack of numbers to compare to Dave's tape evaluations, and we match a lot of things up. I go in and we compare notes, to a certain extent. We watch a lot of these guys, and we talk through it, and I think Dave takes it into account.
"You're putting the two things together, and it might result in asking a question about a player you might not have graded so highly [based on tape]. Is there a reason to re-evaluate a guy based on these statistics?"
So, what happens when the scouts love a player's tape, but there is a red flag in the metrics?
"That's very much the case when looking for a player who might be the exception to the rule, but I think we've stuck to what the analysis told us on a number of our picks over the last couple of years. Dave hasn't been picking players with red flags, and he hasn't needed to look for a reason why this or that player might be the exception to a particular rule."
Khan wanted to make very clear who was in charge throughout draft weekend. Over and over, he insisted to me that the metrics are a part of what the Jaguars do, but that it's never to undermine the efforts of the scouting department.
"Dave has the authority to make the picks he wants to make, and he's made a lot of selections over the last two years of players with numbers that indicate success. It's fun when those two things match up, because they should, but if a player is making the best possible play a high percentage of the time, you want your scouting to confirm that."
When I asked Khan for an example of a player for whom the tape and stats really matched up, he brought Lee up again -- it was clear that Lee is a prime mover when it comes to the ultimate proof of the Jaguars' current dual view of the scouting process.
"If you look at his entire body of work, he's been an incredibly dynamic receiver, and he's been such a statistical marvel that after his freshman year, as my father was buying the Jaguars, we were regularly interacting with our West Coast scouts. We had been interested in him for many years, and Dave had been a highly regarded college scouting director, so he was interested as well. We wanted to get a good, young quarterback early in this draft, and then to get someone we regarded as an elite receiver with the 39th pick -- that was something that we were very pleased to be able to do."
Of course, the controversial pick was the Jags' first-rounder. Taking UCF quarterback Blake Bortles with the third overall selection was seen as a pretty big reach by just about everyone, though some will tell you that when you have a specific eye on your quarterback of the future, you don't worry about draft position -- you just go get him. And that's where we started when discussing the Jaguars' 2014 picks.
Blake Bortles, QB, UCF -- First round, third overall pick
"Well, Dave was extremely high on Bortles, and he was a player I tracked all year statistically," Khan said. "As the season went on, he continued to show up at the top of a lot of the lists I consider to be important statistically for projecting college players to the pros. After the season ended, and through the draft evaluation process, he continued to show up as a quarterback who did a lot of things very well. Without just looking at the draft class -- in looking at the entire NCAA, Blake was in the top 10 in the country -- the 96th percentile in the nation in yards per attempt, and he did extremely well under pressure. I've heard some television personalities and seen some writers saying that he doesn't perform well under pressure, when nothing could be further from the truth -- especially relative to other quarterbacks selected in this draft. He averaged 7.8 yards per attempt when under pressure, which was in the 95th percentile, and he had the best yards per attempt under pressure of any quarterback drafted in this class."
(By comparison, Teddy Bridgewater averaged 6.9 yards per attempt under pressure, Johnny Manziel averaged 6.9 as well, Zach Mettenberger averaged 6.6, Aaron Murray averaged 6.2, A.J. McCarron averaged 5.5, Tajh Boyd averaged 5.1, Tom Savage averaged 4.5, Logan Thomas averaged 4.4, and Derek Carr averaged 3.5.)
"Blake didn't have a giant sack total that was a major red flag -- he took some sacks and you certainly want to minimize that, but it wasn't a major problem for him. He scored 1.8 standard deviation above the national population of NCAA quarterbacks in overall yards per attempt, and he scored 1.6 standard deviation over the population mean in yards per attempt while under pressure. He was among the top 10 of all college quarterbacks in rushing touchdowns per rush attempt ... so, he did a lot of things really well. When Dave told me that he was a quarterback he was very interested in early in the process, we drove down [to UCF] from Jacksonville along with Gus and our offensive coordinator Jedd Fisch, in Jedd's SUV. It's not a long drive from Jacksonville to Orlando at all, and we filled out our March Madness brackets on the way back. In the end, he was someone who did a lot of things well, along with Dave's very strong evaluation of him."
Marqise Lee, WR, USC -- Second round, 39th overall pick
Then, it was tine to circle back and take the player who seemed to personify Jacksonville's need to wed these seemingly disparate (to some, at least) concepts of scouting and stats.
Again, many of Jacksonville's own metrics are proprietary and therefore confidential, but by more public measures, Lee's 2012 season was particularly amazing. His 118 catches led the nation, his 1,721 receiving yards ranked second, his 14 touchdown catches tied for third most and he finished third in the nation with 2,683 all-purpose yards. He averaged 16.7 yards every time he touched the ball, and the average length of his 2012 touchdowns was 40.8 yards.
He had 16 plays of 40-plus yards, 11 of 50-plus yards and seven of 70-plus yards, and more than half of his 2012 receptions (63 of 118, 53.4 percent) were for first downs. He accounted for 38.2 percent of USC's all-purpose yardage in 2012, more than any Heisman-winning wide receiver has ever enjoyed. Khan added that Lee averaged more than a first down per target through his career (10.27 yards) and that he averaged 13.28 yards per catch on third and fourth down.
So, you can see why the Jags were willing to look past a slightly disappointing 2013 season in ways other NFL teams were not.
"One of the most interesting things about Marqise in 2013 is that he had suffered a [knee] injury that doesn't show as recurring as a major red flag on any player in the future," Khan said. "It limited his production, and he wasn't able to play a full slate of games, and I think when you piece it together, his 2013 season wasn't reflective of the great player. And when he finally had a long break to recover from his injury, he went out in the final game of the season in the [Las Vegas Bowl] game against Fresno State and put up the kinds of numbers [seven catches for 118 yards and two touchdowns] you'd expect to see from him in 2012.
"For Dave, his evaluation as the general manager -- based on his scouting expertise, this was an elite receiver, and from a statistical standpoint, I would concur. His drop rate increased in 2013, but his career drop rate is very low, and I don't think the player you saw in that slate of games in 2013 is indicative of the player we drafted."
Allen Robinson, WR, Penn State -- Second round, 61st overall pick
As it was when the Seahawks took Colorado receiver Paul Richardson in the second round (and as Seattle's area scout confirmed when asked about Richardson's specific value), receivers who dominate in offenses not built for the passing game can often be projected to explode with productivity when they hit the NFL. Of course, there are schematic issues to overcome, and the coverages will be much tougher and more diverse, but Robinson's dominance in Penn State's offense -- in 2013, he caught 97 passes for 1,432 yards and six touchdowns on a team that completed 233 passes for 3,110 yards and 21 scores -- was a big indicator of his NFL potential. And in Jacksonville, the scouts and stats agreed completely on this.
"It's a very interesting question. And I'll point to a stat that's publicly available that I'd looked at before, and I thought was really interesting. Out scouts were incredibly high on Allen, and this was another case of the statistics saying something good about a player's ability. And it concurred with what they were thinking. Allen averaged 14.2 yards after the catch on screen passes, and Penn State threw a lot of screens. Allen was a very important part of Penn State's offense, and if you were going to play Penn State, one would think that [an opponent] would spend a lot of time scheming to try and limit the damage Allen Robinson does to you. I thought it was very impressive, the numbers he was able to put up. And again, that's a case of the tape matching up the stats."
Rashaad Reynolds, CB, Oregon State -- Undrafted free agent
My conversation with Khan about this draft class actually started with an undrafted free agent the Jaguars signed -- Oregon State cornerback Rashaad Reynolds. Khan reached out to me after I expressed on Twitter that I couldn't believe Reynolds wasn't picked, just based on his tape.
"The scouts go through their process, and Dave, as the GM, does an internal review of all the players in a draft class. And while Dave was going though this, I was in there with Dave and Gus and the scouting directors. As a group, we went through a number of players, and one player I pulled up from the stats and brought up that I'd like to have an extended discussion about was Rashaad Reynolds. We put on the tape, and everyone agreed that it was pretty strong. The reason I wanted to review him was that he had been such a productive cornerback at Oregon State. He had allowed just 33 catches on 77 targets into his coverage, a 42.9 completion percentage, for 481 yards, two touchdowns, six interceptions and had broken up 11 passes. In a number of my metrics, which also examine his ability to finish tackles and his overall tackling, he fared very well.
"So, we kept an eye on him through the draft, and we weren't able to address that need through the draft because Dave had so many needs that he wanted to fill, and I think he did a great job. I made it a priority to get this player, because I thought it was fortunate that he wasn't drafted -- I was watching the board very hopefully for the last 10 picks. As soon as the draft was over, I was on the phone with his agent, to explain why I thought he'd be a great fit here. And it was another great example where scouting, coaching and analytics all come together. DeWayne Walker, our defensive backs coach, actually recruited him in college. He called Rashaad and said, 'Hey -- it would be great if you came here.' We all worked together, and I thought it was really cool." It will be years before the Jaguars know whether this class of players panned out to their projections, but everyone in the room seems to be on board with an expanded model of player evaluation that includes as many advanced metrics as possible. They may have started out under fifty feet of crap after that 2-14 season two years ago, but the Jags appear to be moving in a positive -- and proactive -- direction.