Researchers at the University of Texas at Arlington are partnering with the U.S. Naval Research Laboratory to improve aptitude tests in the U.S. military.
Over three years, the $700,000 collaborative research agreement will test different measures, including those that focus on body function, to home in on specific abilities that make someone more likely to succeed in training programs like flight school and other military occupations.
Lead investigator Matthew Robison, UTA assistant professor of psychology, and U.S. Naval researchers hope to find measures that better predict who will succeed in training and also reduce unintended bias in the tests.
“It’s been very interesting for me to get a glimpse at how cognitive psychology can have implications for a meaningful societal problem,” said Robison. “Sometimes it can feel like you’re just working on a problem that no one cares about. And then all of a sudden, you meet people who also care about that problem and have a reason to care about it.”
Certain occupations in the military — like those in aviation — are cognitively demanding and require specific skill sets.
Before initial efforts to standardize tests in the early 1940s, attrition rates for pilots were as high as 63%, said Lt. Nicholas Armendariz, an aerospace experimental psychologist in the U.S. Navy. Today, he said, the attrition rate is about 6%.
And while this is a leap forward, there is still room to improve these tests.
People have naturally occurring differences in their abilities, and that includes our cognitive abilities, said Robison. For example, one person might be particularly good at maintaining attention, while another has excellent emotional intelligence.
These differences in cognitive ability arise from a complex combination of a person’s biology, environment throughout life and, as researchers like Robison are beginning to understand, day-to-day conditions like sleep, nutrition and levels of stress.
Researchers will explore several individual differences including cognitive abilities like attention control, working memory capacity and fluid intelligence — or someone’s ability to reason with abstract information, develop rules and create solutions quickly, said Robison.
(Attention control is someone’s ability to avoid distractions and maintain attention for a period. And working memory capacity is the ability to handle multiple ideas, tasks or goals at the same time.)
The researchers will use eye-tracking to measure these abilities. “There’s been decades of research showing that when you’re doing something cognitively demanding — you’re trying to remember a list or you’re trying to solve a complicated math problem — your pupil dilates and you show this physiological response to the output of cognitive effort,” said Robison.
The researchers will determine whether eye-tracking measures can help predict individual differences in ability and also provide additional information about an individual’s cognitive effort and attention control while completing a task, said Ciara Sibley, a senior scientist at the U.S. Naval Research Laboratory.
Depending on researchers’ results, these measurements may be used to continue to improve two aptitude tests that are used to screen trainees in uniform for different occupations. These are the Aviation Selection Test Battery and the Armed Services Vocational Aptitude Battery.
Robison will gather findings from his lab at UTA, while researchers with the U.S. Naval Research Laboratory collect similar data from trainees in uniform at their lab in Pensacola, Fla.
Every year, military aptitude tests are evaluated for their ability to predict success, and adjustments are made to certain aspects of scoring and administration as needed, said Sabrina Drollinger, a U.S. Navy personnel psychologist.
Every two to three years, tests are also reviewed for adherence to the latest military policies. Bigger, more holistic changes — like major changes to test items or sections — require a longer process of research and review.
In some instances, prior knowledge or experience before the exam, like knowledge of planes or ships, can be predictive of success. But in some cases, they are not, said Robison.
This is also an area where unintended gender bias can come into play.
Take, for example, stick-and-throttle tests — one of the tests used in the screening and selection process for trainees, especially those who want to be aviators. An applicant’s experience with certain video games can improve scores in this portion of the test, said Robison, and since women play video games at lower rates than men, this may create an unintended bias.
But this bump in someone’s score may not actually predict how well they do in training, said Robison. This is one of the areas where researchers hope to be able to improve the tests.
“By using eye-tracking, and also focusing in on some more pure measures of, for example, attention control, we hope to be able to reduce some of the influences of experience that lead to score differences across genders and race and ethnicity,” said Sibley.
Jessica Rodriguez reports on science for The Dallas Morning News as part of a fellowship with the American Association for the Advancement of Science.
