By Drew Harwell • The Washington Post

An artificial-intelligence hiring system has become a powerful gatekeeper for some of America’s most prominent employers, reshaping how companies assess their workforce — and how prospective employees prove their worth.

Designed by the recruiting-technology firm HireVue, the system uses candidates’ computer or cellphone cameras to analyze their facial movements, word choice and speaking voice before ranking them against other applicants based on an automatically generated “employability” score.

HireVue’s “AI-driven assessments” have become so pervasive in some industries, including hospitality and finance, that universities make special efforts to train students on how to look and speak for best results. More than 100 employers now use the system, including Hilton, Unilever and Goldman Sachs, and more than a million job-seekers have been analyzed.

But some AI researchers argue the system is digital snake oil — an unfounded blend of superficial measurements and arbitrary number-crunching, unrooted in scientific fact. Analyzing a human being like this, they argue, could end up penalizing nonnative speakers, visibly nervous interviewees or anyone else who doesn’t fit the model for look and speech.

The system, they argue, will assume a critical role in helping decide a person’s career. But they doubt it even knows what it’s looking for: Just what does the perfect employee look and sound like, anyway?

“It’s a profoundly disturbing development that we have proprietary technology that claims to differentiate between a productive worker and a worker who isn’t fit, based on their facial movements, their tone of voice, their mannerisms,” said Meredith Whittaker, a co-founder of the AI Now Institute, a research center in New York.

“It’s pseudoscience. It’s a license to discriminate,” she added. “And the people whose lives and opportunities are literally being shaped by these systems don’t have any chance to weigh in.”

Loren Larsen, HireVue’s chief technology officer, argues that such criticism is uninformed and that “most AI researchers have a limited understanding” of the psychology behind how workers think and behave.

Larsen compared algorithms’ ability to boost hiring outcomes with medicine’s improvement of health outcomes and said that the science backed him up. The system, he argued, is still more objective than the flawed metrics used by human recruiters, whose thinking he called the “ultimate black box.”

“People are rejected all the time based on how they look, their shoes, how they tucked in their shirts, and how ‘hot’ they are,” he told The Washington Post. “Algorithms eliminate most of that in a way that hasn’t been possible before.”

The AI, he said, doesn’t explain its decisions or give candidates their assessment scores, which he called “not relevant.” But it is “not logical,” he said, to assume that some people might be unfairly eliminated by the automated judge.

“When 1,000 people apply for one job,” he said, “999 people are going to get rejected, whether a company uses AI or not.”

The inscrutable algorithms have forced job-seekers to confront a new kind of interview anxiety. Nicolette Vartuli, a University of Connecticut senior studying math and economics with a 3.5 GPA, said she researched HireVue and did her best to dazzle the job-interview machine. She answered confidently and in the time allotted. She used positive keywords. She smiled, often and wide.

But when she didn’t get the investment-banking job, she couldn’t see how the computer had rated her or ask how she could improve, and she agonized over what she’d missed. Had she not looked friendly enough? Did she talk too loudly? What did the AI hiring system believe she’d gotten wrong?

“I feel like that’s maybe one of the reasons I didn’t get it: I spoke a little too naturally,” Vartuli said. “Maybe I didn’t use enough big, fancy words. I used ‘conglomerate’ one time.”

HireVue says its system dissects the tiniest details of candidates’ responses — their facial expressions, their eye contact and perceived “enthusiasm” — and compiles reports companies can use in deciding who to hire or disregard.

Job candidates aren’t told their scores or what little things they got wrong, and they can’t ask the machine what they could do better. Human hiring managers can use other factors, beyond the HireVue score, to decide which candidates pass the first-round test.

The system, HireVue says, employs superhuman precision and impartiality to zero in on an ideal employee, picking up on telltale clues a recruiter might miss.

Major employers with lots of high-volume, entry-level openings are increasingly turning to such automated systems to help find candidates, assess resumes and streamline hiring. The Silicon Valley startup AllyO, for instance, advertises a “recruiting automation bot” that can text-message a candidate, “Are you willing to relocate?” And a HireVue competitor, the “digital recruiter” VCV, offers a similar system for use in phone interviews, during which a candidate’s voice and answers are analyzed by an “automated screening” machine.

But HireVue’s prospects have cemented it as the leading player in the brave new world of semi-automated corporate recruiting. It says it can save employers a fortune on in-person interviews and quickly cull applicants deemed subpar. HireVue says it also allows companies to see candidates from an expanded hiring pool: Anyone with a phone and internet connection can apply.

Nathan Mondragon, HireVue’s chief industrial-organizational psychologist, told The Post the standard 30-minute HireVue assessment includes half a dozen questions but can yield up to 500,000 data points, all of which become ingredients in the person’s calculated score.

The employer decides the written questions, which HireVue’s system then shows the candidate while recording and analyzing their response. The AI assesses how a person’s face moves to determine, for instance, how excited someone seems about a certain work task, or how they’d behave around angry customers. Those “Facial Action Units,” Mondragon said, can make up 29% of a person’s score; the words they say and the “audio features” of their voice, like their tone, make up the rest.

HireVue’s growth is already running into some regulatory snags. In August, Illinois Gov. J.B. Pritzker signed a first-in-the-nation law that will force employers to tell job applicants how their AI-hiring system works and get their consent before running them through the test. The measure, which HireVue said it supports, will take effect Jan. 1.

State Rep. Jaime Andrade Jr., who co-sponsored the bill, said he pushed the transparency law after learning how many job applicants were rejected at the AI stage of a job interview. He worried that spoken accents or cultural differences could end up improperly warping the results, and that people who declined to sit for the assessment could be unfairly punished by not being considered for the job.

“What is the model employee? Is it a white guy? A white woman? Someone who smiles a lot?” he said. “What are the data points being used? There has to be some explanation, and there has to be consent.”

Emma Rasiel, an economics professor at Duke University who regularly advises students seeking jobs on Wall Street, said she has seen a growing number of students excessively unsettled about their upcoming HireVue test.

“It’s such a new and untried way of communicating who they are that it adds to their anxiety,” she said. “We’ve got an anxious generation, and now we’re asking them to talk to a computer screen, answering questions to a camera … with no real guidelines on how to make themselves look better or worse.”