Tuesday, May 19, 2009

At 13:13 EST one of the routers in the California data center started experiencing high packet loss, blocking service access to some customers. Technicians worked to resolve the problem and the affected services where brought back online at 13:56 EST. We apologize for the inconvenience.

*** UPDATE *** May 22, 2009

Our data center provided the following reason for outage:

"One of the routers had an issue with memory fragmentation, causing packet loss for about half of the data center. We rebooted the router, and that solved the problem.

We have also determined that memory fragmentation was not being properly monitored on that router in accordance with our monitoring policies. This is being corrected, so that we will be able to deal with this issue proactively if it resurfaces."