CHROs around the world are rolling out AI within their own teams and workforces—with varying success. But there’s an important pitfall to keep in mind: A set it and forget it approach to the new tech just won’t cut it.

That’s according to Karyn Twaronite, EY’s global vice chair of diversity, equity and inclusiveness, who shared the insight at Fortune CHRO virtual series this month. Her company was using AI to screen job applicants when the system automatically sent a rejection letter to a stellar candidate with all the skills necessary for the job. The problem? The candidate’s resume and cover letter didn’t hit all of the key words the AI was trained to search for.

“AI does really require human augmentation and people to be empowered to raise their hand to say, ‘Mistake, mistake,’’’ she said. “We can learn from mistakes along the way and tech systems get it right much of the time. But great recruiters, DEI and HR professionals can help take these systems from good to great.”

Twaronite’s team caught the oversight and quickly reached out to the candidate to move forward in the job process, but not all businesses will be as lucky.

Roughly 25% of organizations are currently using AI to support HR-related activities, according to an April report from SHRM. But the current knowledge gap is sizable: 43% of HR leaders recently reported that they possess either limited or no theoretical knowledge of AI, and just 54% have an intermediate-level understanding of the AI basics. 

Kelley Steven-Waiss, chief transformation officer at ServiceNow, stressed that leaders must ensure there is accountability for the ethical use of AI across a business, spanning people, process and technology. The tech doesn’t know the difference between right and wrong, but most humans do. 

“The role that the HR executive plays in AI strategy is serving as the conscience of the organization,” she says.

Kristin Stoller
kristin.stoller@fortune.com

Today’s edition was curated by Emma Burleigh.

Around the Table

A round-up of the most important HR headlines.

Air Canada pilots have voted in favor of striking starting mid-September if the airline does not reach an agreement with workers over a new pay deal. WSJ

An agricultural conglomerate and the United Farm Workers union are battling it out after some employees allege they were tricked into joining the union, but the labor group says otherwise. New York Times 

A tsunami of work apps and products are overwhelming staffers as trimmed HR budgets have stretched managers thin, forcing them to rely on technology to fill gaps in workforce needs. Business Insider

Watercooler

Everything you need to know from Fortune.

Spying comfortability. A new report finds that a fair amount of workers are fine with their employers monitoring their work-related messages and communication—more than what HR leaders expected. —Marco Quiroz-Gutierrez

Soccer strides. The National Women’s Soccer League has agreed on new terms with its players’ union, removing the draft and earnings caps while giving newcomers the choice to pick the league they play for. —Paolo Confino 

Attracting big egos. While companies commonly used words like “ambitious” and “results-oriented” in job postings to describe ideal candidates, a study finds that narcissists are drawn to this language. —Lindsey Leake