One of the side effects of the recent RackSpace outage in their Dallas/Fort Worth data centre has been finding out just how quickly their data centre heats up when the air conditioning system fails.
Our backup generators kicked in instantaneously, but the transfer to backup power triggered the chillers to stop cycling and then to begin cycling back up again a process that would take on average 30 minutes. Those additional 30 minutes without chillers meant temperatures would rise to levels that could result in data loss and irreparably damage customers’ servers and devices. We made the decision to gradually pull servers offline before that would happen.
WOW! 30 minutes from air-con failure until temperatures reach a level when servers start being damaged. I knew the temperatures would go up fast, but I didn’t think they’d heat up that fast.