This interview with an AWS leader isn’t aging well, from CBS Sunday morning:

Pogue asked, "I don't mean to give anyone ideas, but let's say I figured out that one of these unmarked buildings was an AWS data center, and I blew it up. Are you saying that it's so backed up and redundant that you probably wouldn't notice?" Wood replied, "Yeah, you wouldn't notice. I mean, we might be a bit upset, but you wouldn't notice!"

https://www.cbsnews.com/news/cloud-computing-loudoun-county-...

I think this is due to the data residency requirements in UAE. I'm working with a client in the health space and the government requirements requires me to store data only in UAE! Tried with AWS but they were not allowing any new instances and I had to go with Azure.

My personal bet is there's an 80% chance this is caused by some internal bootstrapping problem that they've messed up. AIUI all the main cloud vendors are in trouble here. The automation project I work on is expressly designed to help folks solve this DR/bootstrapping problem. Soo many people get this wrong. Of course missiles don't help things, but I'd bet AWS is primarily to blame here. I'd love an actual technical report of why they can't recover things.

11.3 Force Majeure. Except for payment obligations, neither party nor any of their affiliates will be liable for any delay or failure to perform any obligation under this Agreement where the delay or failure results from any cause beyond its reasonable control, including acts of God, labor disputes or other industrial disturbances, electrical or power outages, utilities or other telecommunications failures, earthquake, storms or other elements of nature, blockages, embargoes, riots, acts or orders of government, acts of terrorism, or war.

I wonder if this will cause a mini cyber insurance crisis. I don't think any of those data loss plans have been tested at scale.

I wonder what kind of "some data" is: military installations in Arab states and Israel?

They guaranteed 11 9's durability, didn't they?

e: Yep

https://aws.amazon.com/s3/storage-classes/

No disaster recovery plan? No offsite backups? Someone failed to applied the most basic principles that have existed for decades.

They can recover most of it? I assumed that those AZs had been offline for so long because they were like gone gone.

If all your backups are with AWS you don't have backups at all.

Well… they have a good excuse.

   me (Middle East)
   ├── me-south-1 (Bahrain)                                DOWN since 2026-04
   │   ├── mes1-az1                            me (Middle East)

├── me-south-1 (Bahrain) DOWN since 2026-04 │ ├── mes1-az1 DOWN │ │ └── mes1-mct1-az1 (Oman, Muscat) │ ├── mes1-az2 DOWN since 2026-03-01 │ └── mes1-az3 DOWN ├── me-central-1 (United Arab Emirates) │ ├── mec1-az1 │ ├── mec1-az2 DOWN since 2026-03-01 │ └── mec1-az3 DOWN since 2026-03-01 └── il-central-1 (Israel, Tel Aviv) ├── ilc1-az1 ├── ilc1-az2 └── ilc1-az3 DOWN │ │ └── mes1-mct1-az1 (Oman, Muscat) │ ├── mes1-az2 DOWN since 2026-03-01 │ └── mes1-az3 DOWN ├── me-central-1 (United Arab Emirates) │ ├── mec1-az1 │ ├── mec1-az2 DOWN since 2026-03-01 │ └── mec1-az3 DOWN since 2026-03-01 └── il-central-1 (Israel, Tel Aviv) ├── ilc1-az1 ├── ilc1-az2 └── ilc1-az3

Does this mean that even with 3 availability zones for Amazon S3 storage, that some data is lost?

The data center layout should look something like this:

    me (Middle East)
    ├── me-south-1 (Bahrain) DOWN since 2026-04
    │   ├── mes1-az1 DOWN
    │   │   └── mes1-mct1-az1 (Oman, Muscat) ???
    │   ├── mes1-az2 DOWN since 2026-03-01
    │   └── mes1-az3 DOWN
    ├── me-central-1 (United Arab Emirates)
    │   ├── mec1-az1
    │   ├── mec1-az2 DOWN since 2026-03-01
    │   └── mec1-az3         DOWN since 2026-03-01
    └── il-central-1 (Israel, Tel Aviv)
        ├── ilc1-az1
        ├── ilc1-az2
        └── ilc1-az3

Not sure about the Muscat local zone, whole me-south-1 region has been reported down despite Muscat still being operational.

If someone had told me a year ago that a whole AWS region could go down I'd called them crazy, but now me is close to exactly that happening.

See also previous discussion: https://news.ycombinator.com/item?id=49033240

They say "some" data, i wonder what percentage that really is. I haven't seen pictures but I find it hard to imagine all of me-south-1 was completely leveled to the point where's there's just nothing left. On the other hand, if you have 100 rows of racks and then randomly take out a contiguous 10% across both rows and columns it may be functionally equivalent to taking out everything.

I wonder if they'll start adding an underground bunker to new data centers so you can put an S3 replica there?

I suppose this is why Microsoft and Google want to build data centers in the Netherlands. Its not cheap but it is safe.

So many AWS simps in this thread that don't actually understand how AWS scrimped out and fucked their customers. You can't all suck up to AWS at the same time, you need to serialize.

Seems like their multi-AZ redundancy didn't account for missile strikes. Good reminder to always have your own backups.

There are many regulations over there which prevent data from leaving the country.

Sometimes those rules can have serious consequences.

"..even if something happens only once in a billion requests, that means it happens multiple times per day within S3."

but one in a trillion...

Guess those multi-AZ promises have an asterisk when actual missiles are involved. Makes you double-check your own off-site backups.

Hey but that's exactly as per design. It is the customer's responsibility to store stuff elsewhere as DR backup, not AWS.

What a nightmare scenario to tabletop. How do you even begin to recover from something like this?

Since roughly zero of the articles I've seen link back to the source of their quotes, here it is: <https://status.aws.amazon.com/#multipleservices-me-south-1_1...>

We collectively gave this company trillions over the years and they still can’t get it right.

us-east-1 is finally looking pretty stable for once.

Everything not saved will be lost.

Ahhh the downside of “data sovereignty” rules and the de-globalization meme strikes again. I’d bet a bajillion dollars the reason this happened is due to government thinking it’s a good idea to make it illegal to store data outside their country.

Hey EU, take note of this next time you create silly data residency requirements that don’t allow data to travel outside your region. Encryption is an easy solution to multi-region residency…as long as the European Commission doesn’t stupidly keep trying to make encryption illegal too!

Hint: Russia absolutely knows where your data centers are.

Backup both locally and everywhere no matter what.

50%+ of companies that lose all of their data go out of business in 6 months.

DR/BCP costs are readily justified by doing a Business Impact Analysis (BIA).. budget up to some fraction of risk cost * risk probability.

And a friendly reminder that replication isn't a tested data backup.

I'm sorry but your data is in another castle.

Uh.

Uh-oh.

It's not clear from their messaging if multiple availability zones were severely damaged, or if the damage to one availability zone was simply more than they planned for. If it's the latter, that's a big uh-oh.

The wording certainly seems very careful:

"The damage to our infrastructure spanned multiple availability zones and exceeded what our regional and multi-AZ services are designed to withstand"

It was always a bad bet for billionaires like Bezos to become Trump enablers. You weren't buying a seat at the table, or the privilege of being left alone, you were just signing yourself up to be force-fed shit sandwiches over and over (And the shit-to-bread ratio gets worse as time goes on)

You should have used your considerable resources to fight. If only billionaires would oppose aspiring tyrants with the same zeal with which they oppose even minor tax increases.

[dead]

[dead]

[dead]

[flagged]

[flagged]

This is the flipside of data residency requirements that countries are now starting to require. If the EU wants to keep data in the EU, then great, but when the war comes and energy and infrastructure are hit, people would have wished for backups in North America, Asia, and the middle east.