Calculating Power Requirements for our Data Center

Calculating power requirements is pretty straight-forward. I gathered up all the data sheets on all the equipment we planned for our data center and recorded the avg and max values in Watts. Some manufacturers provide the power requirements in amps at a specific voltage, and some provide a value in watts. Some manufacturers provide both average and max values, but if they only provide a single value, its usually the maximum.

Its simple to get the wattage: volts · amps = watts

Next I designated which devices should be grouped and started playing with the elevations. Then I added up all the max watts of the individual pieces of equipment in each rack and divided by the number of circuits going into that rack (4). Essentially, the load will be distributed over four circuits. Should one of the circuits go down, we'd want to be able to run everything connected to a given PTXL unit on a single branch circuit. Because we can only draw 80% of the power available on a single circuit at any given moment (16A / circuit in this case), our goal is to stay well below 40% utilization on each circuit to allow for a complete failover and allow some room for growth.

So, to begin I subtracted that sum of all the equipment destined for a single rack from 3840W ((120 · 20)2 · 0.8) , which is 80% of the total watts available between the two primary circuits, and that gives me a gross approximation on the max wattage for that rack. Next I divided the equipment between the 2 PTXL units within each cage (see below about PTXL). Typically when a server starts up, the device temporarily draws more power during the POST, spinning up all fans and whatnot. However, a device will typically idle at half or a third of its max wattage. So, in a rack where your power margins are getting thin, you'll want to consider a device like the PTXL which allows you to prioritize power ports and set delays on them. Why? In a catastrophic power-loss scenario, a well designed implementation would stage the start-up sequence such that the storage devices would come up first, then probably the switches would come up, then the routers, and then the hosts. Each environment is different, so it varies.

To get the most for our money, and to use the power most efficiently, we'd want to pull in 3-phase power, but for some reason the data center services are all freaked out about an extra wire, the cost is 3-times that of single phase power. Anyhow, I contacted the data center (Equinix) to learn their max power/sq ft. It basically worked out to 80A/rack or four 120V/20A branch circuits for each rack (two are primary, two are redundant). I picked the NEMA L5-20 twist-n-lock plug type, specifically because I knew that the data center facility provides the power drops above the rack, with the plug receptacle facing down.

Another consideration when ordering power at a data center is where the circuit will be internally sourced. Be sure that redundant circuits from are sourced from different power banks than the primary circuit. Get a map from the data center on their power layout, it will allow you to double check the work that has been done by the facility engineers. Its always a good idea to make friends with these guys...

I bought two ServerTech Power Tower XL (PTXL) HF16 units per rack and everything has been working out very well. Each unit is IP addressable and has a web interface control panel, that's rather smartly designed, though I have never used it and disabled it. I can also fold the monitoring of these systems into my own monitoring framework because of SNMP (and it does support SNMPv3), and I can simply walk up to the rack and see what our load is at any time. Much to my surprise, the tty and www administration consoles are equivalent. Nice work ServerTech!

My Favorite Citrix/Virtualization Websites

My Faves:
http://www.rtfm-ed.co.uk/
http://blog.scottlowe.org/
http://www.vmguru.com/
http://www.dabcc.com
http://virtrix.blogspot.com/
http://www.stefanschuller.com/
http://www.thin-world.com/
http://www.brianmadden.com

And a few others...
http://www.vmwarez.com/index.html
http://blogs.technet.com/virtualization/
http://x86virtualization.com/