Why the Wonderlic test is an outdated method of thinking about NFL intelligence
As the Director of the Center for Evaluation at Harvard Medical School for 14 years, it was my job to determine the best ways of assessing whether our students would make good doctors. When thinking about the combine and the NFL draft, I cannot help but see many parallels between the challenges faced by medical schools and pro sports teams in figuring out who will be a star and who will fall by the wayside. All of the students at Harvard have grades that go through the roof, just as the combine invitees have extraordinary athletic skills. But do they also have those other factors that will help them become the next great surgeon? Do they have that other something that makes them more likely to become the next Peyton Manning rather than the next Ryan Leaf?
What has struck me in observing both talent pools is that regardless of how well they did in school, no matter what their IQ is, some candidates are really smart in ways that are unrelated to grades and tests. These “smart” people become stars in their respective fields. Others just don’t have the sense that makes them successful practitioners of their art, whether on the football field or in the doctor’s office. It has been my job to understand what to measure, how to measure it, and how to provide a metric by which we can compare one person to another. In medicine, this is an art and science that has made significant strides, but is far from perfect. In football, the gap between data that is meaningful and useful and data that is readily available is deep and wide.
Franchise tag roundup: With designations set, how might each player's negotiation unfold?
This week in Indianapolis, 330 prospects are being poked, prodded, observed and measured in almost every way possible at the NFL scouting combine. NFL talent evaluators will ponder the implications of every slow 40-yard dash time. They will wonder about the ability of a smallish defensive back to outlift a couple of the big-name defensive linemen. And they may have to try to figure out what to make of an otherwise hot quarterback’s score of 17 on the Wonderlic Test.
The Wonderlic has become the NFL’s standardized IQ test, a measure that proposes to determine how smart a football player is. In the 1970s, legendary Cowboys coach Tom Landry began administering an aptitude test developed by the psychologist Eldon F. Wonderlic that had been used in other industries for decades to make hiring decisions about clerks and managers. Based on Landry’s recommendation, the test was adopted by the NFL and administered to every player at the combine. The Wonderlic contains 50 items to be completed in just 12 minutes, and offers test-takers the kinds of mental challenges high school kids hate facing on college entrance tests. The questions range from math problems (What’s the next number in the following sequence…) to vocabulary questions (Are the words “respectable” and “responsible” similar, different, or unrelated?).
The concept makes sense: If a team wants to place a premium on selecting “smart” players, then why not give everyone a test? If nothing else, the Wonderlic puts players under time pressures and requires them to have the ability to read. But will a large vocabulary help the next J.J. Watt get to the quarterback? Should the next Aaron Rodgers have a better understanding of calculus than his peers?
NFL combine viewer's guide: The race for QB1 and other 2017 draft storylines
The only perfect Wonderlic score of 50 ever achieved was by Bengals punter Pat McInally, who just happened to be a graduate of Harvard. The highest quarterback score, 48, was received by Ryan Fitzpatrick, a graduate of ... Harvard. At the other end of the Wonderlic spectrum, Dan Marino got a woefully low score of 15, but no one ever accused him of not being smart enough to find open receivers. Ray Lewis wasn’t much better, with a score of 16, but that didn’t seem to keep him from outthinking offensive coordinators and stuffing runners in the backfield.
In an attempt to look for qualities that are more closely associated with football, the NFL began to administer the Player Assessment Tool in 2013. This one-hour test, devised by an employment lawyer and a psychology professor, borrows closely from the principles used to select firefighters and attempts to assess motivation, learning styles, and other psychological attributes as well as various mental abilities. While clearly a step in the right direction, this new test in predicting football productivity hasn’t been around long enough to be deemed a success yet.
So what kind of a test does the NFL need? First, it should be free of the cultural and racial biases that the Wonderlic and many other cognitive and psychological tests have been accused of. Second, it should be a test that is hard to game. Players and their agents shouldn’t be able to figure out what evaluators want to hear so that they can offer up answers that will make them look good, regardless of what they truly think or believe. Third, the test should be predictive of on-field performance. Players’ scores should correlate with some index of NFL success—at least one study suggests that the Wonderlic holds virtually no correlation with the player’s NFL future. Most importantly, the tests should be at the cutting edge of how psychologists think about intelligence today and based on modern cognitive science.
The combine's most disastrous 40-yard dash ever? A year ago, Chris Jones took it in stride
Howard Gardner, a professor at the Harvard School of Education, has noted the need to broaden our definition of intelligence beyond the kinds of knowledge and skills tested in the Wonderlic. According to Gardner, intelligence includes “the potential to process information ... to solve problems and create products that are of value,” and encompasses interpersonal abilities and self-reflection as much as it does book learning. Gardner’s “smart” wide receiver might not know how to calculate square roots, but he would be able to discuss coverages with his quarterback, adjust his pattern at the line of scrimmage and reflect on his errors in order to correct them.
From this elasticized version of intelligence has come the concept of emotional intelligence (EI), which merges people’s thoughts with their feelings to lay the foundation for a person’s capacity for leadership and teamwork. Peter Salovey, a social psychologist who is the current president of Yale, described EI as a set of skills that allow people to use emotions rather than to let them get in the way, noting that a person’s ability to monitor his own and others’ emotions can guide thinking, improve decision-making and help with problem solving.
With Adrian Peterson, Jamaal Charles entering free agency, RB market on brink of a huge shift
Football merges head and heart: Like students in school, players are constantly involved in film study and have to learn the playbook. Once on the field, they have to be able to read coverages and make sense of formations. Football is also a game of emotion, but smart football players also learn to manage their emotions and use them productively.
Those demands simply make the Wonderlic—with all due respect to Landry—an outdated way of thinking about intelligence when it comes to predicting performance on the football field, and it’s still unclear how much the PAT adds to the conversation.
Perhaps someday NFL talent evaluators will devise a better way of assessing intellect and emotion together, developing a metric that can be a valuable guide to predicting who will succeed and who will fail. For now, however, measuring the elusive quality of football smarts remains the most imprecise of imprecise sciences.
