Friday, June 15, 2012


*** UPDATE 4:47M EST ***

All services are now restored.



*** UPDATE 4:06AM EST ***

Data center has brought all affected instances back online. We are running a data consistency check before we bring the database servers back online.

*** UPDATE 2:36AM EST ***

Data center reports as follows:

"We continue to bring affected EC2 instances and EBS volumes back online"


*** UPDATE 1:35AM EST ***

Data center reports as follows:

"We can confirm a portion of a single Availability Zone in the US-EAST-1 Region lost power. We are actively restoring power to the effected EC2 instances and EBS volumes. "


*** UPDATE 12:54AM EST ***

We can report some progress: problem is only affecting only two of our database servers. Data center reports as follows:

"We have identified the issue and are currently working to bring effected instances and volumes in the impacted Availability Zone back online. We continue to see increased API error rates and latencies in the US-East-1 Region"

*** UPDATE 12:30AM EST ***

Data center just posted the following update:

"We continue to investigate this issue. We can confirm that there is both impact to volumes and instances in a single AZ in US-EAST-1 Region. We are also experiencing increased error rates and latencies on the EC2 APIs in the US-EAST-1 Region"

*** UPDATE  12:23AM EST ***

Data center has not reported anything new yet. Last update read "We are investigating degraded performance for some volumes in a single AZ in the us-east-1 region."

*** ORIGINAL MESSAGE ***

Data center reports problem in storage system for some of the database server. They are investigating the problem.