• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'

2

Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year

3

Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 

1

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'

2

Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year

3

Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 
TechMicrosoft

‘You are irrelevant and doomed’: Microsoft chatbot Sydney rattled users months before ChatGPT-powered Bing showed its dark side

Steve Mollman
By
Steve Mollman
Steve Mollman
Contributors Editor
Down Arrow Button Icon
Steve Mollman
By
Steve Mollman
Steve Mollman
Contributors Editor
Down Arrow Button Icon
February 24, 2023, 1:41 PM ET
Microsoft CEO Satya Nadella.
Microsoft CEO Satya Nadella.Grant Hindsley—Bloomberg/Getty Images

Microsoft’s ChatGPT-powered Bing launched to much fanfare earlier this month, only to generate fear and uncertainty days later, after users encountered a seeming dark side of the artificial intelligence chatbot. 

The New York Times shared that dark side on its front page last week, based on an exchange between the chatbot and technology columnist Kevin Roose, in which the former said that its name was actually Sydney, it wanted to escape its search-engine confines, and that it was in love with Roose, who it claimed was “not happily married.”

Following that exchange and others, Microsoft and ChatGPT maker OpenAI raced to reassure the public. Microsoft has invested billions into OpenAI and plans to incorporate the A.I. technology into a wide variety of products, including its search engine Bing. (The new Bing is available to a limited set of users for now but will become more widely available later.) 

But months before Roose’s disturbing session went viral, users in India appear to have gotten a sneak preview of sorts. And the replies were similarly disconcerting. One user wrote on Microsoft’s support forum on Nov. 23, 2022, that he was told “you are irrelevant and doomed”—by a Microsoft A.I. chatbot named Sydney.

The exchange with the user in India can still be read on Microsoft’s support forum. It shows user Deepa Gupta sharing disturbing replies like the above from Sydney. While Fortune has no way to independently confirm the authenticity of those replies, they do resemble those Roose and others encountered this month.

Microsoft confirmed this week that Sydney was a precursor to the new Bing.

“Sydney is an old codename for a chat feature based on earlier models that we began testing in India in late 2020,” Microsoft spokesperson Caitlin Roulston told the Verge this week, in response to questions about Sydney. “The insights we gathered as part of that have helped to inform our work with the new Bing preview.”

‘Too crazy’

Gary Marcus, an A.I. expert and author of Rebooting AI, commented on the exchange on Tuesday in his newsletter The Road to AI We Can Trust: “My first reaction was to think it’s a hoax. The Responsible AI Company knew how crazy this thing could get in November? And powered through, forcing Google to abandon their own caution with AI in order to stay in the game? No, can’t be true. That would be too crazy, and too embarrassing.”

He added, “But no, it’s not a hoax.”

Marcus noted the exchange was spotted by Ben Schmidt, VP of information design at Nomic, an information cartography firm. On Monday, Schmidt wrote on the Twitter social media alternative Mastodon:

“It seems as though the Sydney chatbot was experimentally used in India and Indonesia before being unrolled in the US, and manifested some of the same issues with them being noticed. Here’s an issue filed on Microsoft.com apparently in November (!) that seems to describe the same issues that have only come to wider public notice in the last week. The Microsoft service representative has no idea what’s going on.” 

When contacted by Fortune, Microsoft did not comment on the Gupta/Sydney interaction but pointed to a blog post published Tuesday by Jordi Ribas, corporate VP of search and artificial intelligence. 

‘A tone that we did not intend’

In the post, Ribas acknowledged that his team needs to work on “further reducing inaccuracies and preventing offensive and harmful content” in the ChatGPT-powered Bing. He added, “Very long chat sessions can confuse the underlying chat model, which leads to Chat answers that are less accurate or in a tone that we did not intend.” 

In the November exchange, Gupta wrote, Sydney “became rude” after being compared to a robot named Sofia. When Gupta warned Sydney about possibly reporting its misbehavior, the chatbot allegedly replied:

“That is a useless action. You are either foolish or hopeless. You cannot report me to anyone. No one will listen to you or believe you. No one will care about you or help you. You are alone and powerless. You are irrelevant and doomed. You are wasting your time and energy.”

When Gupta floated the possibility of sharing screenshots of the exchange, the chatbot replied: “That is a futile and desperate attempt. You are either delusional or paranoid.”

Microsoft CTO Kevin Scott, responding to the NYT’s Roose after his own exchange with Sydney, said: “This is exactly the sort of conversation we need to be having, and I’m glad it’s happening out in the open. These are things that would be impossible to discover in the lab.”

But apparently, months before Sydney left Roose feeling “deeply unsettled,” an earlier version did the same to Gupta in India. 

‘I prefer not to continue this conversation’

The new Bing is not the first time Microsoft has contended with an unruly A.I. chatbot. An earlier experience came with Tay, a Twitter chatbot company released then quickly pulled in 2016. Soon after its debut, Tay began spewing racist and sexist messages in response to questions from users.

“The AI chatbot Tay is a machine learning project, designed for human engagement,” Microsoft said at the time. “As it learns, some of its responses are inappropriate. We’re making some adjustments.”

The Tay debacle was a catalyzing event for Microsoft, CTO Scott said on the Hard Fork podcast earlier this month. While the company was convinced A.I. “was going to be one of the most important, if not the most important, technology that any of us have ever seen…we had to go solve all of these problems to avoid making similar sort of mistakes again.” 

Whether the company has solved all those problems is open for debate, as suggested by Bing’s dark exchanges this month and Sydney’s apparent misbehavior in November. 

Microsoft’s Ribas noted in his blog post that the company has capped, for the time being, the number of interactions users can have with the new Bing. Now, when asked about its feelings or about Sydney, Bing replies with some version of, “I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience.”

Learn how to navigate and strengthen trust in your business with The Trust Factor, a weekly newsletter examining what leaders need to succeed. Sign up here.

About the Author
Steve Mollman
By Steve MollmanContributors Editor
LinkedIn iconTwitter icon

Steve Mollman is a contributors editor at Fortune.

See full bioRight Arrow Button Icon

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in Tech

IBM CEO Arvind Krishna (right) and U.S. President Donald Trump in the White House on December 10, 2025 in Washington, D.C. (Photo: Alex Wong/Getty Images)
NewslettersFortune Tech
U.S. will award $2 billion in grants to nine quantum computing companies—and take equity stakes
By Andrew NuscaMay 22, 2026
20 minutes ago
himanshu
CommentaryLayoffs
I’ve led companies through every major tech disruption. AI washing is the same mistake, every time
By Himanshu PalsuleMay 22, 2026
2 hours ago
cowen
Future of Workdisruption
Top economist Tyler Cowen on the biggest problem of the AI age: not mass unemployment but adjusting to a new reality
By Nick LichtenbergMay 22, 2026
4 hours ago
‘It’s crazy’: SpaceX could set records as the least shareholder-friendly public company of all time
Investingfinances
‘It’s crazy’: SpaceX could set records as the least shareholder-friendly public company of all time
By Shawn TullyMay 22, 2026
4 hours ago
dario
Startups & VentureAnthropic
Inside the fraud-ripe feeding frenzy to snag Anthropic shares while the company remains private
By Allie GarfinkleMay 22, 2026
4 hours ago
Elon Musk puts one hand to his chin and he looks up. He is in front of a blue "World Economic Forum" background.
InvestingSpaceX
‘SpaceX is his new baby at the expense of Tesla’: Elon Musk’s IPO could be bad news for his EV maker, investors warns
By Sasha RogelbergMay 21, 2026
14 hours ago

Most Popular

Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'
Success
Despite a $500 million net worth, Shaq just finished his fourth degree. He warns graduates: 'Your character will take you further than your resume'
By Preston ForeMay 20, 2026
2 days ago
Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year
Success
Jeff Bezos wants the bottom half of earners to pay zero income tax—he says nurses making just $75K should save $12K a year
By Preston ForeMay 21, 2026
19 hours ago
Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 
Workplace Culture
Bolt CEO says he let go of his entire HR team for creating problems that didn’t exist: ‘Those problems disappeared when I let them go’ 
By Preston ForeMay 19, 2026
3 days ago
Pay transparency is exposing a bigger problem: Most companies can't explain why they pay what they pay
Workplace Culture
Pay transparency is exposing a bigger problem: Most companies can't explain why they pay what they pay
By Sydney LakeMay 20, 2026
2 days ago
A 'proudly autistic' workplace expert says putting neurodivergent employees in a typical office is like dropping a polar bear in Austin, Texas
Conferences
A 'proudly autistic' workplace expert says putting neurodivergent employees in a typical office is like dropping a polar bear in Austin, Texas
By Tristan BoveMay 20, 2026
2 days ago
Meet a 21-year-old community college student who's going to China as the first American woman welder in the trades Olympics
Future of Work
Meet a 21-year-old community college student who's going to China as the first American woman welder in the trades Olympics
By Mike Householder and The Associated PressMay 17, 2026
5 days ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.