Can an intelligence test forecast which quarterback draft prospects will have NFL success?

Quarterback Tua Tagovailoa is slated^[1] to be selected^[2] with one of the first 10 picks^[3] in the NFL Draft.

Like all top prospects, Tagovailoa has been subjected to months of evaluation, with teams’ scouting departments measuring his athletic abilities, interviewing his college coaches and researching his personal life.

He’s also taken the Wonderlic Personnel Test, which, for about 50 years, teams have administered to prospects. This 12-minute intelligence test consists of 50 multiple choice questions measuring cognitive ability, with the score reflecting the number of correct answers. While all prospects take the test, the scores of quarterbacks – due to the belief that the position requires more brainpower – tend to generate the most media interest. The scores are nominally private, but every year they’re leaked and publicly reported on online databases^[4].

Tagovailoa scored a 19^[5] out of 50. Should that be a cause for concern? A debate rages among fans^[6], analysts^[7], players^[8] and pundits^[9] over the test’s usefulness as an evaluation tool.

But there’s very little actual research on its effectiveness. So my colleague, Brent Evans, and I recently conducted a study^[10] examining the relationship between a quarterback’s Wonderlic score and his NFL success.

The great debate

During World War II, the United States Navy famously used the Wonderlic test, which was developed in 1936 by psychologist Eldon F. Wonderlic^[11], to select fighter pilots. Scores were seen as a good indicator of how pilots would perform under pressure.

Like pilots, NFL quarterbacks must routinely make quick decisions under pressure. They also need to relay complex play calls, read opponents’ defenses and, in response, adjust offensive formations. That’s why quarterbacks are often referred to as “field generals^[12].”

For these reasons, Dallas Cowboys Hall of Fame coach Tom Landry was drawn to the Wonderlic. Widely credited^[13] with introducing the test as an evaluation tool for NFL prospects, Landry won two Super Bowls and appeared in three others^[14] in the 1970s. During that same period, the league, following Landry’s lead, began administering the test at the NFL Combine, which is a nine-day annual event^[15] in Indianapolis that gives teams the chance to scout over 300 potential draftees.

Yet even though the test continues to be given today, many argue that a quarterback’s score doesn’t reveal much about his likelihood of NFL success. Even Eldon F. Wonderlic’s daughter holds this view^[17].

Those who believe the Wonderlic test is a poor assessment tool for NFL quarterbacks often point to players like Dan Marino^[18], who only scored a 15 on the test but went on to become a Hall of Famer. Then there’s Ryan Fitzpatrick^[19], who scored a 48 on the test but has spent his career bouncing from team to team as a journeyman quarterback.

“We’re here to tell you what a growing number of NFL executives already know – the Wonderlic is totally worthless,” Joseph Stromberg wrote in Vox^[20]. In 2015 commentator NFL analyst Mike Florio described the Wonderlic test^[21] as “an outdated, irrelevant intelligence exam to which the league clings.”

Nonetheless, the test has its evangelists. Clay Travis, founder and lead writer of Outkick the Coverage, has maintained^[22] that Wonderlic scores matter a great deal. He points out that the New England Patriots^[23] – the NFL’s most successful team of the 21st century – consistently draft players that score highly on the Wonderlic test. Travis also notes that many star quarterbacks, from Tom Brady to Aaron Rodgers, received excellent scores.

Digging into the data

So which camp is correct?

Using a statistical tool known as regression analysis, we were able to control for a large number of variables that might influence a quarterback’s performance in the NFL, from his college football statistics, to whether his college coach had experience as an NFL coach, to whether he was a finalist for the Heisman Trophy, the annual award given to the most outstanding player in college football. We also controlled for a player’s Wonderlic score.

To quantify NFL success, we considered several measures, including – but not limited to – career passing yards, wins and games started in the NFL. Of all the variables included in our regression models, only two were significantly and consistently associated with a quarterback’s NFL success: whether he was a Heisman Trophy finalist and his Wonderlic score. This is overwhelming evidence that, all else equal, quarterbacks with better Wonderlic scores enjoy more successful careers in the NFL.

Interestingly, we found that a quarterback’s Wonderlic score doesn’t have a significant impact on his draft position. This indicates that – despite the fact that test scores are a good predictor of NFL success and receive a fair amount of media attention – teams, by and large, don’t give them a lot of weight when deciding whether to draft a quarterback. Rather, our research indicates that teams mostly focus on variables such as a quarterback’s completion percentage in college, and physical attributes such as his body mass index, height and speed.

This doesn’t mean that teams should automatically draft quarterbacks with higher Wonderlic scores ahead of quarterbacks with lower ones. The “all else equal” element of the analysis is key. In other words, if two quarterbacks are extremely similar in most aspects, but one has a higher Wonderlic score, our research does suggest that the quarterback with the higher score will enjoy more success in the NFL.

This might sound obvious. But with everything else being equal, other measures you would think might forecast NFL success, such as the quarterback’s college statistics and his university’s reputation for producing successful NFL quarterbacks, don’t have the same predictive abilities about his future NFL success. To further cement the importance of the Wonderlic, after holding other factors constant, a quarterback’s actual draft position is not significantly related to his NFL success. But his Wonderlic score is.

So teams looking for a slight edge on draft day should take their cues from Tom Landry, the Patriots and Clay Travis. A quarterback’s Wonderlic score is revealing something important, and the stakes are high: Over half of all Super Bowl MVPs have been quarterbacks^[24], and choosing correctly can set a team up for years of success. On the other hand, given quarterbacks’ astronomical salaries^[25], drafting a dud in the first round is a mistake most teams can’t afford to make.

[Like what you’ve read? Want more? Sign up for The Conversation’s daily newsletter^[26].]

References

1. ^{^} slated (bleacherreport.com)
2. ^{^} to be selected (www.nfl.com)
3. ^{^} one of the first 10 picks (www.cbssports.com)
4. ^{^} online databases (nflcombineresults.com)
5. ^{^} scored a 19 (247sports.com)
6. ^{^} fans (www.reddit.com)
7. ^{^} analysts (www.vox.com)
8. ^{^} players (247sports.com)
9. ^{^} pundits (ramblinfan.com)
10. ^{^} a study (doi.org)
11. ^{^} Eldon F. Wonderlic (wonderlic.com)
12. ^{^} field generals (www.google.com)
13. ^{^} Widely credited (www.si.com)
14. ^{^} won two Super Bowls and appeared in three others (www.pro-football-reference.com)
15. ^{^} nine-day annual event (www.nflcombine.net)
16. ^{^} AP Photo (www.apimages.com)
17. ^{^} holds this view (www.cbssports.com)
18. ^{^} Dan Marino (www.pro-football-reference.com)
19. ^{^} Ryan Fitzpatrick (www.pro-football-reference.com)
20. ^{^} wrote in Vox (www.vox.com)
21. ^{^} described the Wonderlic test (profootballtalk.nbcsports.com)
22. ^{^} has maintained (www.outkickthecoverage.com)
23. ^{^} the New England Patriots (www.pro-football-reference.com)
24. ^{^} have been quarterbacks (athlonsports.com)
25. ^{^} given quarterbacks’ astronomical salaries (www.spotrac.com)
26. ^{^} Sign up for The Conversation’s daily newsletter (theconversation.com)