• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

As Big Tech showers employees with perks to win the talent war, Nvidia built a nearly $5 trillion company by making people pay for their own lunch

2

MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year

3

Current price of oil as of July 1, 2026

1

As Big Tech showers employees with perks to win the talent war, Nvidia built a nearly $5 trillion company by making people pay for their own lunch

2

MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year

3

Current price of oil as of July 1, 2026
TechPointCloud

Amazon explains database glitch that impacted big customers

Barb Darrow
By
Barb Darrow
Barb Darrow
Down Arrow Button Icon
Barb Darrow
By
Barb Darrow
Barb Darrow
Down Arrow Button Icon
September 23, 2015, 8:24 AM ET
Photograph by David Ryder — Getty Images
Add Fortune on Google for similar content.

Amazon Web Services has posted an explanation of what happened at its big U.S. East data center that caused customers like Netflix to experience issues on Sunday.

According to the post-mortem of the “service event” (you have to love that term) a brief network disruption at 2:19 a.m. PDT affected a subset of the servers running Amazon’s (AMZN) DynamoDB database service which stores and maintains data tables for customers. Each table is divvied up into partitions, containing a portion of the table data and those partitions, in turn, are parceled out to many servers to provide fast access and to allow data replication.

Per the post, which apparently went up Tuesday night:

The specific assignment of a group of partitions to a given server is called a “membership.” The membership of a set of table/partitions within a server is managed by DynamoDB’s internal metadata service. The metadata service is internally replicated and runs across multiple data centers. Storage servers hold the actual table data within a partition and need to periodically confirm that they have the correct membership. They do this by checking in with the metadata service and asking for their current membership assignment. In response, the metadata service retrieves the list of partitions and all related information from its own store, bundles this up into a message, and transmits back to the requesting storage server.

Emphasis is mine. Read the post for the full blow-by-blow, but in essence, Amazon said the issue Sunday was that because so many customers are using a new DynamoDB feature called Global Secondary Indexes, the affected DynamoDB servers could not query the metadata service within the allotted time and took themselves offline.

As of 5:06 a.m. PDT Amazon thus decided to pause requests to the metadata service to relieve the load. By cutting down all those server retries, it was able to bring up additional capacity and restart the service.

Basically this all boils down to the fact that this Amazon service was operating at near full-capacity but AWS internal monitoring apparently did not pick that up in time to avert an outage, said David Mytton, CEO of Server Density, a London-based company which keeps an eye on web server performance across providers for customers.

“A normal network issue caused enough extra load to push the system over capacity which caused the issue,” he said via email.

Amazon will likely fix the monitoring situation and adjust its processes to improve analysis so this does not happen again, he noted. Amazon is nothing if not reactive.

Events like this one reignite the debate about whether businesses should entrust so much of their critical workload to shared public cloud infrastructure that they themselves do not own or control.

Some even brought up the notion that AWS, the self-proclaimed master of distributed and redundant resources, has become the de facto single point of failure IT people dread.

This morning’s #AWS outage reminds us that we all have a single point of failure now. In some ways, we used to be more resilient than that.

— Michael Jackson (@mjackson) September 20, 2015

Fortune reached out to Amazon for comment, but typically the cloud giant and big customers— including Netflix (NFLX)—note that if applications are designed correctly to take advantage of public cloud resources the benefits outweigh the risks.

The Amazon post ended in an apology and a promise to do better, explaining “… we will do everything we can to learn from the event and to avoid a recurrence in the future.”

Expect to hear more about this issue at AWS Re:invent in Las Vegas next month.

For more on Amazon Web Services, see the video.

Subscribe to Data Sheet, Fortune’s daily newsletter on the business of technology.

About the Author
Barb Darrow
By Barb Darrow
See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest in Tech

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in Tech

Why build AI data centers in a cornfield when Saudi Arabia has cheap oil and cheaper power?
Big TechMeta
Why build AI data centers in a cornfield when Saudi Arabia has cheap oil and cheaper power?
By Catherina GioinoJuly 2, 2026
60 minutes ago
Scott Bessent, US treasury secretary, during an Economic Club of New York (ECNY) event in New York, US, on Tuesday, June 23, 2026.
Economynational debt
Elon Musk says AI is the only way to fix the $40 trillion U.S. debt crisis—but a new study says even the most optimistic scenario won’t fill the hole
By Eleanor PringleJuly 2, 2026
3 hours ago
A test of Anduril's Altius drone.
NewslettersTerm Sheet
Defense tech could be entering its awkward teenage years. Is the boom a bubble?
By Allie GarfinkleJuly 2, 2026
4 hours ago
em
Commentary250 Years of Innovation
America’s 250th birthday has Elon Musk and a record IPO. Its 15th had Alexander Hamilton — and a stock market bubble
By Owen LamontJuly 2, 2026
4 hours ago
Meta CEO Mark Zuckerberg (left) and CTO Andrew "Boz" Bosworth in Menlo Park, California, on Wednesday, Sept. 17, 2025. (Photo: David Paul Morris/Bloomberg/Getty Images)
NewslettersFortune Tech
Meta prepares to join the cloud infrastructure fray
By Andrew NuscaJuly 2, 2026
4 hours ago
Sam Altman seeks new world order for AI as OpenAI slowly loses ground to Google and Anthropic 
AIMarkets
Sam Altman seeks new world order for AI as OpenAI slowly loses ground to Google and Anthropic 
By Jim EdwardsJuly 2, 2026
5 hours ago

Most Popular

As Big Tech showers employees with perks to win the talent war, Nvidia built a nearly $5 trillion company by making people pay for their own lunch
Big Tech
As Big Tech showers employees with perks to win the talent war, Nvidia built a nearly $5 trillion company by making people pay for their own lunch
By Marco Quiroz-GutierrezJuly 1, 2026
1 day ago
MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year
Success
MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year
By Sydney LakeJune 25, 2026
7 days ago
Current price of oil as of July 1, 2026
Personal Finance
Current price of oil as of July 1, 2026
By Joseph HostetlerJuly 1, 2026
1 day ago
Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster
Success
Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster
By Preston ForeJune 27, 2026
5 days ago
The Supreme Court's birthright citizenship ruling hands the U.S. economy a $7.7 trillion win
Newsletters
The Supreme Court's birthright citizenship ruling hands the U.S. economy a $7.7 trillion win
By Diane BradyJuly 1, 2026
1 day ago
Trump got a $78K pension from the Screen Actors Guild in 2025 because he appeared in Home Alone 2 in 1992
Politics
Trump got a $78K pension from the Screen Actors Guild in 2025 because he appeared in Home Alone 2 in 1992
By Sasha RogelbergJuly 1, 2026
21 hours ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.