dcm magazine

News

Banner
Electrical resilience – the premium for uptime
Wednesday, 22 April 2009 00:00

Electrical infrastructure can account for half, or possibly more of the expenditure in creation of a new Data Centre. The level of resilience has an enormous effect on cost of this infrastructure. Gareth Evans of Northcroft explores some of the cost drivers

A recent white paper published by the Uptime Institute entitled ‘Dollars per kW plus Dollars per Square Foot Are a Better Data Centre Cost Model than Dollars per Square Foot Alone’ urges project managers to engage Tier Technology and Cost Experts with multiple project experience to provide cost advice relative to the uptime performance sought by the client. It goes on to provide examples that demonstrate how metre or foot square rates cannot be relied upon when calculating Data Centre construction costs.   In a series of articles we shall explore the effect on cost of electrical infrastructure, mechanical infrastructure, communication and protective infrastructure plant space requirements and compartmentation requirements relative to resilience and IT power demands.

Of course the total electrical power load of a data centre is made up of much more than just the IT load. Uninterruptible Power Supply, Cooling, Lighting and Ventilation together with the efficiency of each are major contributors to electrical power consumption and have major effects on the sizing of  switchgear, plant and equipment.

Data Centre fault tolerance, maintainability, and power load are the main drivers in the cost of Electrical and Mechanical Services. These in turn make up in the region of 50-95% of the construction cost of a new Data Centre, excluding land acquisition, and an even greater percentage of construction cost in any upgrade in power capacity or resilience of an existing Data Centre. Therefore the design of these installations and the required reliability of power supply and resultant cooling load are of paramount importance to anyone involved in the funding for such a project. This must be weighed against the ramifications of downtime on a business in order to determine how much expenditure is justifiable; this exercise along with element and component level life cycle cost models referred to later in this article are appropriate at feasibility stage when the engagement of a Consultant Engineer and Cost Consultant would be advisable.

Electrical Installations in the majority of instances represent the largest single investment for a Data Centre. The most significant cost items are Generators, Transformers, Uninterruptible Power Supply equipment, Switchgear and Power Distribution Units. In addition the power carrying infrastructure represents a major investment. Mains power supply has not been considered as cost, for connection to the national grid may run into millions of pounds.


Approximate Cost of Major Plant Items
The main plant items can be broken down as follows:

Generators & UPS
Generators including exhaust systems, day/bulk fuel storage tanks, ventilation ductwork and commissioning cost anything between £190 per KVA and £300 per KVA. Generators when combined with kinetic UPS units are referred to as Dynamic Rotary Uninterruptible Power Supply (DRUPS) solution; expect costs for these units in the region of £350 per KVA to £500 per KVA including peripherals; these have not been used for this graphic cost comparison.

For kinetic UPS units, which we have used in our graphic example,  expect costs in the region of £300 per kVA to £700kVA. Simply put, Kinetic UPS units have a heavy rotating wheel driven by an electric motor powered from the low or, more likely, medium voltage infrastructure. When there is a mains power failure the wheel continues to rotate its kinetic energy generating power and meeting the essential electrical load for approximately 15 seconds. Some load shedding may be essential during this period and some form of short break control may be required. After 15 seconds the generators will have reached operational speed to meet the load. Kinetic UPS units can be sited remotely from generators and in other plant areas. Cooling and mechanical ventilation are required with, for example, a 750kVA kinetic UPS unit that is 98% efficient potentially producing waste heat of 15 to 20kW. The battery version relies on large numbers of cells and is in the region 30% less space-efficient. It will require cooling and ventilation however expect lower costs than the kinetic UPS and in the region of £200KVA to £400KVA. Essential loads can be met for minutes rather than seconds. Running costs will differ but batteries are more maintenance intensive and a life cycle cost analysis will guide the purchaser to the best £/kW value for the economic design life of the facility. 

Power Step-down
Transformers costs of £40 to £90 per kVA can be expected though a complete external packaged sub-station could cost anything between £20,000 and £150,000.

Distribution derating
Switchgear- this is normally of the low/ medium voltage variety. A Form 4 type 5 LV Switchpanel currently costs in the region of £2500 and £5000 per circuit breaker. The main Switchpanels will require Power Factor Correction the costs of which are in the region of £60 per kVAR.

Power carrying infrastructure- Cable or Busbar?
Electrical distribution is a major cost component in the electrical infrastructure. Cost comparisons recently carried out between the supply and installation of cable and busbar revealed savings of between 15% and 25% in favour of cable.

The merits and demerits of busbar and cable have to be assessed. In terms of spatial coordination and the often congested building services that exist in a Data Centre, busbar may be preferable. Whilst costs at first glance for busbar do appear high, the power carrying capacity per metre square of space take-up is lower and can be a major factor in the designers’ decision making process. Cable often requires costly containment systems that can attract a large cost. An example is 300mm wide heavy duty cable tray which including fittings and support steelwork will cost in the region of £40 on average per metre.

Cost Differential Analysis basis
The power load and resilience of a Data Centre will establish the quantity and size of plant required. Power distribution topology can have a major influence on cost. We have produced a graph of the step-up in costs, based on Uptime Institute Tiers one to four and the following assumptions:-
•   White Space Area:  2000 metres square.
•   IT Power Load: 1500 watts per metre square.
•   It has been assumed for the purposes of the cost model that the backbone busbar will be included however Power Distribution Units and connections onto servers area excluded.
The Uptime Institute schematic layouts published in the aforementioned paper would be a good point of reference when referring to the plant and infrastructure schedules below. It should be emphasised that these simple cost models are for electrical power provision to IT equipment and mechanical plant. They exclude small power, lighting, cooling and ventilating plant, security and access control, lifts, building envelope, land purchase, Operational expenditure and HV connection.

Tier 1
The basic Electrical Infrastructure to include:
•   1Nr. 2000kVA Transformer
•   1Nr. 1000kVA Generator & Peripherals (standby only)
•   1Nr. Main Switchpanel with Power Factor Correction,1Nr. Rotary Fly Wheel Uninterruptible Power Supply Unit.
•   3Nr. Switchpanels complete with Air Circuit Breaker & Moulded Case Circuit Breakers.
•   Power Distribution Units
•   A predominantly Busbar- power carrying infrastructure
•   1Nr. Motor Control Centre (for control of Mechanical Plant items such as Fans, Pumps, Air Handling Units and Chillers)

Tier 2
The basic Electrical Infrastructure to include:
•   2Nr. 2000kVA Transformer
•   2Nr. 1000kVA Generators & Peripherals (standby only)
•   2Nr. Main Switchpanels with Power Factor Correction Air Circuit Breaker & Moulded Case Circuit Breakers
•   2Nr. Rotary Fly Wheel Uninterruptible Power Supply Units
•   4Nr. Switchboards or Panel Boards, Power Distribution Units 
•   A predominantly Busbar- power carrying infrastructure and, 1Nr. Motor Control Centre (for control of Mechanical Plant items such as Fans, Pumps, Air Handling Units and Chillers)
 
Tier 3
The basic Electrical Infrastructure to include
•   2Nr. 2000kVA Transformer
•   2Nr. 1000kVA Generators & Peripherals
•   2Nr. Main Switchpanels c/w Power Factor Correction & Air Circuit Breaker & Moulded Case Circuit Breakers
•   2Nr. Rotary Fly Wheel Uninterruptible Power Supply Units
•   2Nr. Generator Output Switchboards
•   2Nr. Supply Switchgear  for critical pumps & fans (all mechanical plant is excluded from the analysis), 1Nr.Alternative Switchgear for Power Distribution Unit, 2Nr. Generator Output Switchboards
•   2Nr. Critical Motor Control Centre
•   2Nr. Mechanical Services Switchpanel
•   1Nr. UPS Input Switchboard
•   1Nr. UPS Output Switchboard 
•   Power Distribution Units and a predominantly Busbar- power carrying infrastructure

Tier 4
The basic Electrical Infrastructure to include:
•   2Nr. 2000kVA Transformer
•   2Nr. 1000kVA Generators & Peripherals
•   2Nr. Main Switchpanels with Power Factor Correction Air Circuit Breaker & Moulded Case Circuit Breakers
•   2Nr. Rotary Fly Wheel Uninterruptible Power Supply Unit
•   2Nr. Output (Alternative Output) Switchboards 
•   2Nr. Supply Switchgear  for critical pumps & fans and MCC’s, (all mechanical plant is excluded from this analysis)
•   2Nr. Supply Switchgear for Power Distribution Unit
•   2Nr.Generator Output Switchboards
•   2Nr. Critical Motor Control Centre
•   2Nr. Mechanical Services Switchpanel
•   2Nr. UPS Input (Alternative Input) Switchboards,
•   Power Distribution Units a predominantly Busbar- power carrying infrastructure.
This simplified approach to cost differentials gives us the graph below. It must be emphasised that, the same computing power is being served in all four instances.

Construction cost increases dramatically when comparing Tier 2 and Tier 3 engine generator plant requirements. Generators would predominantly be considered to be stand-by rating only for Tier 1 and Tier 2 Data Centres whilst in Tier 3 and Tier 4 Data Centres; generators are required by the Uptime Institute to be the primary power source. Generators that are required to operate continuously attract significantly greater cost and demand fuel storage requirements on a different scale from those required for back-up only; capacities will reflect the continuous running requirement with all the concomitant costs that fuel storage legislation will attract.  This may preclude consideration of Tier 3 or 4 infrastructures by all but major financial institutions and multi-national conglomerates’. There is a possibility that, in these days of energy- uncertainty, market and legislative demands for energy efficiency that Combined Heat and Power Units could be considered for Tier 3 and 4 Data Centres. The waste heat could be harnessed to power Absorption Chillers and reduce site power consumption accordingly or for the benefit of for instance an adjacent municipal swimming pool.

Salient differences in functionality between Tier 3 & 4 from an Uptime Institute performance criteria viewpoint are that Tier 4 requires compartmentalization of distribution paths and continuous cooling. As we are, in this instance, solely analyzing the electrical infrastructure in the cost model comparison we have not considered these requirements. Tier 4 requires two electrical paths to be simultaneously energised whereas Tier 3 requires one to be active and the other to be available as maintenance or failures dictate. Whilst Tier 3 requires concurrent maintainability, Tier 4 requires tolerance to failure of a complete path without disruption to IT.

Prominent performance differences in operation between Tier 1/2 & 3/4 are:-
•   Engine generators for Tier 3 & 4 are required to have no limitation on consecutive hours of operation when loaded on ‘N’ demand. (Disruptions in utility power are considered to be an expected operational condition).

•   Engine generators that have a limit on consecutive hours of operation at ‘N’ demand are suitable only for Tier 1 & 2 Data Centres. (Disruptions in utility power are considered a failure).
 
Life Cycle Costing
BS ISO 15686-5 is an international standard for Life Cycle and Whole Life Costing. It refers to various levels of costing analysis including:
•   System (sub-element) level 
•   Component level.

At component level, options open to the Data Centre operator could be considered. These ideally should be carried out at Feasibility Stage or at Outline Proposals Stage. Early strategic Life Cycle Cost analysis can reveal the optimum commercial and energy efficient option. Obviously they must be considered in tandem with the best engineering solution to meet the Data Centre operator’s future requirements.

Life Cycle Cost calculations consider, amongst other things, the following salient factors:
•   Capital Cost
•   Economic Life Factor (the normal life expectancy of an item of plant)
•   Decommissioning and end of life costs
•   Maintenance routines and costs
•   Energy consumption

It is preferable to carry out a comparison of two or more options that may be incorporated into the design. Over a 25 year duration, for example, the capital cost, the energy cost, maintenance cost, replacement costs and de-commissioning costs are analysed on a Net Present Value basis. Sub-elements, systems or items of plant should be compared. Examples of useful comparisons are: Chillers/ chilled water plant options, free cooling plant systems, DRUPS versus Generators with Battery UPS, and Air Handling Equipment/ systems.

Design, functionality, maintenance and engineering demands may highlight a number of engineering solutions. Plant Space may ultimately displace computer space so it has a potentially high commercial worth. It is highly desirable to consider all plant solutions available in life cycle exercises before committing to a project .

Conclusion
There are a myriad of considerations to be made in the construction or enhancement of a Data Centre. Making the right decisions at the right time is pivotal in the functionality, economy and overall success of any project and business. Skimping on costs at an early stage may prove very costly later on. It is therefore advisable to approach all issues, be they matters of resilience, power demand, plant life expectancy, or cost, methodically and systematically.  Using any free cost information can be perilous and we strongly advise anyone embarking on the construction of or enhancement of a Data Centre to engage the services of suitably qualified and experienced professionals.