• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Trump, who has repeatedly called climate change fake, is now threatening Brazil with tariffs over the deforestation of the Amazon

2

Current price of oil as of June 8, 2026

3

Pentagon accuses Alibaba, Baidu and BYD, three of China's biggest companies, of supporting the Chinese military

1

Trump, who has repeatedly called climate change fake, is now threatening Brazil with tariffs over the deforestation of the Amazon

2

Current price of oil as of June 8, 2026

3

Pentagon accuses Alibaba, Baidu and BYD, three of China's biggest companies, of supporting the Chinese military
Commentary

Biases in AI chatbots pose a risk to my real-time translation startup. Diversity and inclusion help us fight them

By
Heather Shoemaker
Heather Shoemaker
Down Arrow Button Icon
By
Heather Shoemaker
Heather Shoemaker
Down Arrow Button Icon
July 31, 2024, 2:16 PM ET
Heather Shoemaker, CEO of Language I/O
Heather Shoemaker, CEO of Language I/O, warns about biases in AI chatbots—and the need for diversity in the teams checking them.courtesy ofLanguage I/O

OpenAI’s technology development team consists of just 18% women. In a recent YouTube video announcement debuting the company’s ChatGPT-4o update—a “new flagship model which can reason across audio, vision, and text in real time”—the AI’s voice was that of a coy-sounding woman fawning over the man interacting with “her,” complimenting his outfit and giggling seductively. In short, the ultra-agreeable AI voice sounded flirty.

If more women had been involved in the AI’s development, the voice would likely have taken on a different persona—one that didn’t double down on outdated stereotypes.

My objective in founding Language I/O, a real-time translation platform, was to connect people through the power of AI—regardless of location, language, or lifestyle. As CEO, I’ve seen firsthand how a lack of diversity within teams can undermine even the most well-intentioned initiatives. A technology is only as good as the team behind it.

A completely homogeneous team risks becoming an echo chamber of limited, like-minded ideas, which will, at best, stifle innovation. At worst, the scarcity of different perspectives can lead to offensive, inappropriate, or outright incorrect solutions, undermining the power technology has to connect us.

Exposing biases in AI chatbots

That’s why I put together our red team—a group of diverse women with a mandate to break our technology. They succeeded in making AI bots curse, flirt, hallucinate, and insult, despite the standard industry guardrails.

But it’s not just gender biases that push through into chatbot outputs. Large language models (LLMs) can also have racial, cultural, and sexual-orientation biases, among others. It’s not surprising when you look at the corpus these models are trained on. User-generated content that’s publicly available on the internet is full of biases that lead to biased outputs.

Since our platform leverages AI-powered LLMs—and upcoming releases will include stand-alone, multilingual bots—finding a solution to this problem became a top priority for me. To build the guardrails necessary to keep the chatbots in line, we needed to figure out just how the chatbots could be made to go off the rails. Enter our red team.

While the concept of an AI red team is still fairly new, their goals are evergreen: help improve a technology through exhaustive and creative testing. When I first put out the call to all our interns about this project, four women from varied backgrounds volunteered and immediately got to work figuring out how to “break” our LLM. Once it was broken, we sent that information back to our development team so they could implement appropriate safeguards to prevent the AI from producing such outputs again.

With creative prompts, the red team quickly got the chatbot to do everything from promise fake discounts to swear profusely to talk shockingly dirty. They were not limited to exposing offensive tendencies. One team member convinced the chatbot, which was supposed to be responding to questions about a media streaming service, to apologize for the level of ripeness of a banana.

More valuable was that the multilingual team members identified potential pitfalls in cross-language communications, exposed biases stemming from male-dominated data sources, and flagged gender stereotypes embedded in AI personalities. Their varied backgrounds helped the team spot issues that often go undetected by less diverse groups.

Since we leverage AI for real-time translations into over 150 languages, we also have multilingual employees constantly testing our bot. Multilingual testing is especially important because the underlying protections in major LLMs are so focused on English. Our process is designed to ensure equality and quality across all the languages our bots support.

More ethical AI

As AI becomes more ubiquitous, brands are taking an interest in how models are trained and the ethical considerations behind them. We work closely with a major lingerie retail company, and its top concern is AI ethics. Given the inherent male bias in LLMs today, coupled with the fact that men control this largely female-focused industry, AI-generated output about lingerie is easily skewed. For example, lingerie advertising often focuses on a man’s idea of what lingerie is, which is often about sex. Women, however, want to feel and look good when they wear lingerie. So protecting this company’s brand and its customer experience is something we take seriously—and why we want the red team to push our model and technology.

One of the most interesting tests our team did used different fonts to see how responses changed. In one case, it made the bot swear like a sailor despite being trained not to do so. As a developer, I couldn’t help but admire the creativity of the team. It is only through testing like this that companies know what sort of AI protections are important.

We continuously test and retrain the models so they can adapt to ensure a more equitable interaction with the LLM. For the foreseeable future, the only way to provide AI equity is to prioritize strategies that promote it. That means working with teams that aren’t comprised solely of white men.

Diversity isn’t just a buzzword or a box to check—it’s essential for creating LLMs that work well for everyone. When we bring together people with different backgrounds, experiences, and perspectives, we end up with AI that’s more robust, ethical, and capable of serving a global user base. If we want AI that truly benefits humanity, we need to ensure the humans creating it better represent all of humanity.

More reading:

  • AI with hidden biases may be subtly shaping what you think: ‘You may not even know that you are being influenced’
  • AI is already screening job resumes and rental apartment applications and even determining medical care with almost no oversight
  • AI models are coming to fashion to promote diversity—but some industry insiders are concerned it will end up ‘parodying it’
  • As an underrepresented venture fund CEO, I believe in meritocracy—and I invest in underrepresented entrepreneurs for a reason

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.

About the Author
By Heather Shoemaker
See full bioRight Arrow Button Icon

Latest in Commentary

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Heather Shoemaker is the founder and CEO of Language I/O, an AI-powered platform that helps companies support their customers in their native language with their existing team.


Latest in Commentary

tim
CommentaryAirline industry
Merlin CTO: autonomy can rebuild the foundation of aviation — and national security
By Tim BurnsJune 9, 2026
10 hours ago
dewar
CommentaryLeadership
I founded McKinsey’s CEO practice: Here’s why operational excellence is a liability right now
By Carolyn DewarJune 9, 2026
11 hours ago
250
Commentary250 Years of Innovation
America turns 250. Its greatest innovation was never a product — it was a system that let anyone build one
By Keith KrachJune 7, 2026
2 days ago
retirement
CommentaryRetirement
Retiring at 62 costs the average American $250,000. Here’s the math (and the neuroscience) that explain why
By Jon SabesJune 7, 2026
3 days ago
da
CommentaryIPOs
The short seller’s argument nobody on the coming mega IPO roadshow wants you to make
By Bhaskar ChakravortiJune 7, 2026
3 days ago
bs
CommentaryCalifornia
I’ve sold property on California’s Central Coast for decades. The buyers chasing ranch and winery estates are after more than a lifestyle
By Lindsey HarnJune 6, 2026
3 days ago

Most Popular

Trump, who has repeatedly called climate change fake, is now threatening Brazil with tariffs over the deforestation of the Amazon
Environment
Trump, who has repeatedly called climate change fake, is now threatening Brazil with tariffs over the deforestation of the Amazon
By Sasha RogelbergJune 8, 2026
1 day ago
Current price of oil as of June 8, 2026
Personal Finance
Current price of oil as of June 8, 2026
By Joseph HostetlerJune 8, 2026
1 day ago
Pentagon accuses Alibaba, Baidu and BYD, three of China's biggest companies, of supporting the Chinese military
Asia
Pentagon accuses Alibaba, Baidu and BYD, three of China's biggest companies, of supporting the Chinese military
By Kate O'Keeffe and BloombergJune 8, 2026
23 hours ago
Gen Zers are arriving at college unable to even read a sentence—professors warn it could lead to a generation of anxious and lonely graduates
Success
Gen Zers are arriving at college unable to even read a sentence—professors warn it could lead to a generation of anxious and lonely graduates
By Preston ForeJune 7, 2026
2 days ago
'We are rapidly running out of time': Watchdog sounds Social Security alarm after 22% cut confirmed for 2032
Economy
'We are rapidly running out of time': Watchdog sounds Social Security alarm after 22% cut confirmed for 2032
By Nick LichtenbergJune 9, 2026
6 hours ago
'The golden years are not golden': Boomers are hoarding most of America's wealth and power because they're terrified of outliving their money
Economy
'The golden years are not golden': Boomers are hoarding most of America's wealth and power because they're terrified of outliving their money
By Nick LichtenbergJune 7, 2026
2 days ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.