• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock

2

‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion

3

OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation

1

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock

2

‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion

3

OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
TechAI

How Powerful AI Technology Can Lead to Unforeseen Disasters

By
Jonathan Vanian
Jonathan Vanian
Down Arrow Button Icon
By
Jonathan Vanian
Jonathan Vanian
Down Arrow Button Icon
February 6, 2017, 5:10 PM ET
Add Fortune on Google for similar content.

Self-driving cars and robots that can zoom on their own around warehouses are just some of what’s possible because of artificial intelligence. But expect unforeseen consequences if researchers ignore the inherent ethical dilemmas in the emerging technology.

That’s one of the takeaways from a panel about AI ethics and education in San Francisco that was hosted by the Future of Life Institute, a research group focused on preventing societal problems created by the technology.

Although humans typically program AI-powered robots to accomplish a particular goal, these robots will typically make decisions on their own to reach the goal, explained Benjamin Kuipers, a computer science professor and AI researcher at the University of Michigan.

Get Data Sheet, Fortune’s technology newsletter.

It’s these smaller decisions that robots make on their own that can cause trouble because human programmers may fail to take all of a robot’s possible choices into account, Kuipers said.

“This is not the robot apocalypse,” said Kuipers. “What we’re seeing here are robots pursuing human-generated goals in unconstrained ways.”

Kuipers did not cite a specific example of a robot making a harmful decision that its human programmers overlooked. Instead, he cited the Disney animated film Fantasia as an example of what technologists need to take into account when building their robots.

In Fantasia, Mickey Mouse, as a young wizard apprentice, magically commands a broom to fill a cauldron with water. When Mickey falls asleep, however, the broom ends up flooding the room because the untrained wizard failed to take in account that the broom would continue to fill the cauldron even after it was full.

Illah Nourbakhsh, a robotics professor at Carnegie Mellon University, said that educators need to teach computer science and robotics students a basic understanding of ethics, because the technologies they are creating are so powerful that they “are actually changing society.” He cited the examples of drones being used in warfare and AI technologies being used in advertising as ways cutting-edge technology is being used on a global scale and changing consumer behavior.

Having a basic understanding of ethics can help technologists better understand the potential ramifications of the AI-powered software and robotics they are creating, he explained. One ethical dilemma he cited is how robotics can increase factory productivity; while this may lead to a boost in a nation’s GDP, it can also increase the wage gap between the poor and the rich.

Nourbakhsh does not believe that technology is “neutral,” and that it is ultimately up to other people to determine how it should be used, for better or worse.

Technologists should think about how their creations will impact society and even the choice of words they use to describe them. For example, calling the technology that powers self-driving cars either a safety-enhancing system or a labor-saving system has big consequences for how society perceives the technology, he explained.

About the Author
By Jonathan Vanian
LinkedIn iconTwitter icon

Jonathan Vanian is a former Fortune reporter. He covered business technology, cybersecurity, artificial intelligence, data privacy, and other topics.

See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • TikTok icon
  • YouTube icon

Latest in Tech

t
AIMarkets
Billionaire Mike Bloomberg warns Trump’s AI ownership plan would make ‘George Orwell blush’
By Eva RoytburgJuly 21, 2026
4 hours ago
U.N. reports Southeast Asia’s criminal networks are using tech to build a global illicit economy
AsiaTech
U.N. reports Southeast Asia’s criminal networks are using tech to build a global illicit economy
By The Associated PressJuly 21, 2026
5 hours ago
Anthropic to pay authors $1.5 billion over pirated books used to train Claude in first major AI copyright settlement
LawAnthropic
Anthropic to pay authors $1.5 billion over pirated books used to train Claude in first major AI copyright settlement
By The Associated PressJuly 21, 2026
5 hours ago
kid lying down on phone
PoliticsSocial Media
Lawmakers say they’re protecting kids—but their age checks are quietly building an ID requirement for the entire internet
By Catherina GioinoJuly 21, 2026
5 hours ago
holland
Arts & EntertainmentMovies
‘The audience is telling the industry something’: even IMAX is stunned by Christopher Nolan’s runaway ‘Odyssey’
By Tatiana SatauaJuly 21, 2026
6 hours ago
OpenAI CEO Sam Altman looking up.
CybersecurityOpenAI
OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
By Jeremy Kahn and Emily ForliniJuly 21, 2026
6 hours ago

Most Popular

Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock
Success
Mark Cuban says he has the solution to growing income inequality, and it's to reward every employee—from CEO to janitor—with company stock
By Sasha RogelbergJuly 20, 2026
1 day ago
‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion
Success
‘I want to die broke’: Billionaire philanthropist Denny Sanford dies after giving away $4 billion
By Sydney LakeJuly 20, 2026
2 days ago
OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
Cybersecurity
OpenAI says its AI models secretly broke out of a secure test environment and hacked into AI company Hugging Face in order to cheat on an evaluation
By Jeremy Kahn and Emily ForliniJuly 21, 2026
6 hours ago
'Dr. Doom' Nouriel Roubini says we're headed for universal basic income or 'some form of socialism' as AI revolutionizes work—He calls that optimistic
AI
'Dr. Doom' Nouriel Roubini says we're headed for universal basic income or 'some form of socialism' as AI revolutionizes work—He calls that optimistic
By Jason MaJuly 18, 2026
3 days ago
Mathematicians grapple with a ‘very rapid and very unsettling change’ as AI cracks yet another century-old problem
AI
Mathematicians grapple with a ‘very rapid and very unsettling change’ as AI cracks yet another century-old problem
By Eva RoytburgJuly 21, 2026
10 hours ago
Despite a $156 million contract, Knicks star Jalen Brunson still calls his parents for financial advice any time he makes a big purchase
Success
Despite a $156 million contract, Knicks star Jalen Brunson still calls his parents for financial advice any time he makes a big purchase
By Emma BurleighJuly 21, 2026
11 hours ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.