Azure EastUS outage

FYI

Impact Statement: Starting at 23:07 UTC on 18 Jun 2024, you have been identified as a customer using Virtual Machines in US East who may receive error notifications when performing service management operations on a zonal level - such as create, delete, update, scaling, start, stop - for resources hosted in this region. This issue affected various Intel and AMD general purpose VM sizes such as Dasv5, Dadsv5, DDSv5, Dasv4, Dsv5, DDsv5, LSv3, Easv5, Dsv4, Easv4, BS, Dsv4, Dv2, Av2, Eadsv5, Esv5 etc. series.

 

Workaround: Alternatively, customers can deploy their resources in East US 2 as alternate region.

 

Current Status: Efforts to further investigate this matter and restore capacity to healthy buffers in relation to capacity are still ongoing. As our engineers continue their mitigation workstreams, we will continue to monitor progression and provide an update within 24 hours, or as events warrant

From Azure Service Health - you need to be signed with at least Reader role on a subscription to see it

1

Comments (7 comments)

Avatar
Carl Long

Peter Yasuda  -Thank you for bringing this to the community via our forums!

0
Avatar
Andre Zanoncello

It appears that there is no easy way to use an alternate region. Moving your golden image is a real pain if you don't already have it in the other region. 
East US seems to be allowing the creation of VM's now, but some stuff is still not working correctly (like deleting old broken VM's). 
By the time we got things moving to East US 2, East US started working again.

0
Avatar
Steve Hopler

This outage wasn't a problem for us until this morning and only affected one client.  I also didn't get far trying to get a session host setup in another region.  The existing session host booted about an hour ago.

0
Avatar
DStephenson

For what it's worth, it's not "fun" or "easy" to move your resources to a new region, but it's not "horrible" either.
I wrote-up this brief post about how to move your AVD resources to a different region that makes it seem a bit less scary. 🙂

Move AVD to a new Azure Region – Nerdio Help Center (getnerdio.com)

Maybe this could be a topic for one of the "Beyond the Basics" webinars, Frank Albani?

2
Avatar
DStephenson

It looks like Hayden Zegley just added a create post about how to help prevent this situation!

Configuring a Secondary Region for AVD Deployment – Nerdio Help Center (getnerdio.com)

0
Avatar
Frank Albani

DStephenson Thanks for the recommendation! We're covering AVD Auto-scaling in Part 3 July 9th 12pm ET and perhaps I'll call out Azure Capacity Extender while we're in there at a minimum.

1
Avatar
Peter Yasuda

Service Health shows this: 

These failures have subsided, and customers should not be experiencing any more allocation failures. While we are aware of capacity constraints in the East US affecting particular Virtual Machines SKUs, these failures were exacerbated by an unrelated incident impacting our allocator service. This secondary issue has been mitigated and deployments should be returning to levels before 22:35 UTC on the 19 June. However, we acknowledge that it may be possible for customers to observe provisioning errors with the following SKUs. Dasv5, Dadsv5, DDSv5, Dasv4, Dsv5, DDsv5, LSv3, Easv5, Dsv4, Easv4, BS, Dsv4, Dv2, Av2, Eadsv5, Esv5 due to the ongoing capacity constraints.

Pre-configuring a secondary region might be a good idea for some workloads. Anyone know when East US 3 is going online? https://azure.microsoft.com/en-us/updates/new-planned-datacenter-region-in-georgia-east-us-3/

0

Please sign in to leave a comment.