Real Time Energy Consumption Measurements

SM
Real-Time Energy Consumption Measurements in Data Centers
ASHRAE TC9.9 and The Green Grid
SM
First printing released at the ASHRAE Winter Annual Meeting, Jan 23 27, 2010
ASHRAE/TGG.2009.RealTimeEnergyConsumptionMeasurementsinDataCenters.Atlanta:American SocietyofHeating,RefrigeratingandAirConditioningEngineers,Inc.;coauthoredbyTheGreenGrid, Beaverton,Oregon.

2 Copyright 2010, The Green Grid The Green Grid Technical Forum
PURPOSE OF THE BOOK

Gartner, IDC, Forrester and other analysts continue to rank Power & Cooling (energy efficiency) as the #1 concern of data center owners/operators TGGs PUE is becoming the proxy for data center energy efficiency The PUE cannot be reasonably determined if energy consumption in the data center cannot be measured EPA reports an average PUE of 2.04 (2009 data)
SM
The book is designed to Provide an overview of the state of energy consumption measurements in the data center Educate the data center owner/operator with respect to making real-time energy consumption measurements in the data center Demonstrate how to consolidate the energy consumption data into a single energy efficiency value, e.g., PUE Provide guidance with respect to determining energy efficiency in mixeduse facilities The Green Grid Technical Forum
Copyright 2010, The Green Grid
TABLE OF CONTENTS
SM
PART 1 BASICS Chapter 1 Introduction Chapter 2 How, What, & Where to Measure Chapter 3 Measurement Devices Chapter 4 Measurement Collection Systems Architecture and Software PART 2 COOLING SYSTEMS Air Measurements Chapter 5 Air Handler Units Chapter 6 Computer Room Units PART 3 COOLING SYSTEMS Hydronic Measurements Chapter 7 Pumps Chapter 8 Cooling Towers Chapter 9 Chillers Chapter 10 Heat Exchangers 2010, The Green Grid The Green Grid Technical Forum Copyright
TABLE OF CONTENTS
PART 4 POWER SYSTEMS MEASUREMENTS Chapter 11 Introduction to Critical Power Distribution Chapter 12 Upstream Critical Power Distribution Chapter 13 Uninterruptible Power Supply Chapter 14 Computer Room Transformer and Power Distribution Unit PART 5 IT SYSTEMS MEASUREMENTS Chapter 15 Compute and Storage Systems Chapter 16 Networking Systems
Appendix Appendix Appendix Appendix Appendix
5
SM
A Pumps B Chillers C Mixed-Use Facilities D Uninterruptible Power Supply E Onsite Power Generation and CCHP in Data Center Applications
Copyright 2010, The Green Grid The Green Grid Technical Forum
KEY CONTRIBUTORS
SM
The Green Grid Technical Forum
KEY CONTRIBUTORS
SM
SM
PART 1 BASICS
SM
MEASUREMENT DEVICES (Chapter 3)

John Bean (APC), presenter
Part 1 Chapter 3 Sensors

3.1 Overview There are numerous attributes within the data center that require measurements
Temperature Pressure Flow Rates Voltage Current Humidity
SM
Sensors come in a variety of connectivity styles: direct, analog (010vdc / 420ma) and serial communication (ModBus, BacNet, etc) Introduce Terms: Span, Resolution, Accuracy

3.2 Accuracy What may appear to be small inaccuracies frequently may be compounded to give very misleading data
Example: Chiller with design temperature difference of 10F Return chilled water sensor reading 1F low Supply chilled water sensor reading 1F high Combined error of 2F Chiller reported capacity error of 20%
SM
11

3.3 Temperature
There are several types of temperature sensors available
Thermocouple dissimilar metals that produce a voltage the corresponds to junction temperature. Accuracy 1 to 5%. Care must be used when using hookup wire to avoid new dissimilar junctions with voltage potentials that could bias reading accuracy Thermistors semiconductor devices whose resistance varies in response to temperature change, typically are constructed of polymer or ceramic. Accuracy 0.1 to 2% RTD Resistance Temperature Detector, typically are wire wound or thin film devices whose resistance changes in response to an increase in temperature. Accuracy 0.01 to 1%. Are available in three and four wire version to compensate for lead wire resistance error
SM
All types may use a local transmitter to convert weak signal to more robust signal (010vdc or 420ma) to allow longer runs of wiring to measurement device
12

3.4 Pressure
There are several types of pressure sensors available
Bourdon Tube Strain Gauge Compound (vacuum and positive pressure) Absolute Pressure Gauge Pressure
SM
Need to ensure wetted surface material is compatible with fluid to be measured All types may use a local transmitter to convert weak signal to more robust signal (010vdc or 420ma) to allow longer runs of wiring to measurement device
13

3.5 Flow Liquid
There are several types of liquid flow sensors available
Paddle Wheel Turbine Wheel Shedding Vortices Venturi Ultrasonic Coriolis Variable Area
SM
Need to assure wetted surface material is compatible with fluid to be measured All types may use a local transmitter to convert weak signal to more robust signal (010vdc or 420ma) to allow longer runs of wiring to measurement device
14

3.6 Flow Gas
There are several types of gas flow sensors available
Pitot Tube Hot Wire Anemometer Flow Nozzle Cup Anemometer
SM
All types may use a local transmitter to convert weak signal to more robust signal (010vdc or 420ma) to allow longer runs of wiring to measurement device
15

3.7 Current
There are several types of current sensors available
Split Core Solid Core Resistor (voltage drop) Clamp-On Split Core
SM
Extra loops of conductor thru core act as current multiplier for current transformers All types may use a local transmitter to convert weak signal to more robust signal (010vdc or 420ma) to allow longer runs of wiring to measurement device
16

3.8 Voltage
There are several types of voltage sensors available
Potential Transformer (high & medium voltage) Voltage Divider Voltage Transducer
SM
17

3.8 Power Monitor
Typically use both current and voltage sensor and have necessary processing means to determine power in real time. Other common metrics from power meters may include
KVA (apparent power) KVAR (reactive power) Power Factor Individual Leg Currents Individual Leg Voltages
SM
May or may not be revenue grade Typically may have local display along with serial communication interface to BMS or SCADA System
18
SM
MEASUREMENT COLLECTION SYSTEMS

Architecture and Software (Chapter 4) Kenneth Uhlman, Ph.D, P.E., presenter
19
Topics
Business Questions Measurement Levels Scalable Hardware / Software Architecture
SM
20
Business Questions
Does this data metering and monitoring strategy support the organizations sustainability objectives? Will this project support current and future regulatory requirements domestically and abroad? Is there executive sponsorship to fund and support the project? How will this project be funded? What is the ROI and risks? Who currently pays the energy bills (IT or Facilities)? Is there a plan to charge back energy (and carbon) to the end users? Technology is not the limiting factor, but rather organizational structure and behavior
21 Copyright 2010, The Green Grid
SM
Measurement Levels
SM
Item Linked IT / Facility Business Goals Metrics Measure and log metrics Online data acquisition Monitoring Separate views (IT & Facilities) Linked IT/Facility dashboards Linked Business / IT / Facility dashboards Proactive Strategies Server managing servers Auto-discovery of IT / Facility devices Proactive IT/Facility Proactive IT/Facility/Smart Grid
22
Minimum Practical ?
Best Practical
State-of the-art
Typical Data Center Meters

Many data centers have plenty of meters for each sub-system Typically installed for capacity or component health, not enterprise efficiency May be able to aggregate and recast data to measure and log efficiency Solutions at various price points to meet business objectives
M 23 IT Copyright 2010, The Green Grid
FACILITY HEALTH CAPACITY EFFICIENCY
SM
M M M M M M M
M M
M M M M M
M M
Data Center Meters

M Facilities
M
Utilities
Enterprise Energy Dashboard

While a single-pane of glass may be an ultimate goal, there are many tactical activities for improvement The cost for enterprise views continues to decrease with the proliferation of networks and web-based equipment and gateways
24
SM
Initiate program with existing metering data where available

Sensing Hierarchy
SM
IT and Facilities equipment have different constraints bu are implemented similarly The application dictates realtime monitoring acquisition rates from seconds, minutes days, weeks or months - IT Applications - Networks - Power or cooling systems
Collecting Data Center Telemetry in an Enterprise
SM
Sites may be internal or external, domestic or international Network security is crucial Typically many vendors and vintages of Power, Building Automation, HVAC, Security, IT, etc. Local and central storage of data and acquisition rates
26
Managed server monitored by a manager server
SM
Proactive management of servers enables IT to match optimal energy rates, facility capacities and carbon footprint Virtualization enables IT loads to be shifted globally Facility systems must be linked to process
27
Data Collector Software Architecture

Is the heart of a linked system Secure and customized screens for: - Executives - IT - Facilities Proactively optimize IT, Facilities and Utilities to meet the organizations business objectives
SM
SM
PART 2 COOLING SYSTEMS Air Measurements
29
SM
AIR HANDLERS (Chapter 5) COMPUTER ROOM UNITS (Chapter 6)

Daryn Cline (Evapco), presenter
30
Part 2 Cooling Systems
SM
Air Measurements
Chapter 5 Air Handlers 5.1 Overview 5.2 Measurement Levels Chapter 6 Computer Room Units 6.1 Overview 6.2 Measurement Levels
31
Part 2 Cooling Systems-
SM
Air Measurements
5.1 Overview
Chapter 5 Air Handlers
32
SM
Air Measurements
Chapter 5 Air Handlers

5.2 Measurement Levels
33
SM
Air Measurements
Chapter 6 Computer Room Units

6.1 Overview CRAC Computer Room Air Conditioners Direct Expansion (DX) Chilled Water Dual Source -Chilled Water Primary, DX Backup Downflow -raised floor Upflow -free discharge or ducted Ceiling units Environmental Control Humidity Control RH, Dew point Sensible Heat

M =Measuring Points for Power Consumption
SM
35
SM
36
SM
PART 3 COOLING SYSTEMS Hydronic Measurements
37
SM
PUMPS (Chapter 7) COOLING TOWERS (Chapter 8) CHILLERS (Chapter 9) HEAT EXCHANGERS (Chapter 10)
Daryn Cline (Evapco), presenter
SM
Hydronic Measurements
Chapter 7 Pumps 7.1 Overview 7.2 Measurement Levels-Electrical 7.3 Measurement Levels-Fluid Chapter 8 Cooling Towers 8.1 Overview 8.2 Measurement Levels Chapter 9 Chillers 9.1 Overview 9.2 Measurement Levels Chapter 10 Heat Exchangers 10.1 Overview 10.2 Measurement Levels
39
Part 3 Cooling Systems Hydronic Measurements

Chapter 7 Pumps
7.1 Overview
SM
40
Part 3 Cooling Systems Hydronic Measurements Chapter 7 Pumps

7.2 Measurement Levels-Electrical
SM
41
Part 3 Cooling Systems Hydronic Measurements Chapter 7 Pumps

7.2 Measurement Levels-Electrical
SM
42

Chapter 7 Pumps
7.3 Measurement Levels-Fluid
SM
43
Part 3 Cooling Systems Hydronic Measurements Chapter 8 Cooling Towers

8.1 Overview-Open Counter and Cross Flow Cooling Towers 8.1 Overview-Closed Counter and Combined Flow Circuit Cooling Towers
SM
44

Chapter 8 Cooling Towers
SM
45

Chapter 8 Cooling Towers
8.2 Measurement Levels 8.2.1 Minimum Practical Measurement-True RMS Power
8.2.2 Best Practical Measurement-Current Transducers 8.2.3 Best Practical Measurement-State-of-the-Art
SM
46

Chapter 9 Chillers
9.1 Overview
SM
47

SM
48
SM
PART 4 POWER SYSTEMS MEASUREMENTS
49
SM
INTRODUCTION TO CRITICAL POWER DISTRIBUTION (Chapter 11) UPSTREAM CRITICAL POWER DISTRIBUTION (Chapter 12) UNINTERRUPTIBLE POWER SUPPLY (Chapter 13) COMPUTER ROOM TRANSFORMER & POWER DISTRIBUTION UNIT (Chapter 14) Steve McCluer (Schneider/APC), presenter
50
Chapter 11 Critical Power Distribution

Critical Power vs. Essential Power
Focus is on the Critical Power Path Can be several meandering paths Supports those loads with zero tolerance for unplanned downtime Mostly (but not entirely) in the IT Equipment Room Essential Power Path can tolerate a brief disruption Highly precise meters are typically required for power quality and forensic analysis, overkill for routine efficiency studies. Part 4 describes a data center rated somewhere between 200 kW and 5000 kW
SM
Generator
Utility Switchgear & Transformers ATS
SM
Distribution Switchgear UPS Mechanical Switchboard Distribution Switchgear Distribution Switchgear
PDU RPDU Mechanical Loads

Non-Data Center Loads
IT Loads
Non-Critical Loads
Critical Path Essential Path

52
Other Loads Path Green Grid Technical Forum The
Ch 12 Upstream Critical Power Distribution

Service Entrance Metering
Typically highly precise revenue grade Typically includes kW & kVAR May be difficult to separate IT loads
SM
Automatic Transfer Switch (ATS)

Typically less precise May capture variances between sources
Primary Electrical Distribution Switchgear

May or may not be available for each circuit Ideal place to capture power into IT equipment room
53
SM
54
Chapter 13 Uninterruptible Power Supply (UPS)

Centralized UPS
Measured outside of IT Equipment Room Output of UPS may be ideal place to capture IT room critical power consumption
SM
Distributed UPS
Measured close to the IT loads in the IT Room May be multiple UPSs May be highest power consumption in the critical power path Possible to capture power going to zones or rows
UPS room
Metering can be at power module &/or at the system level

Distributed UPS
UPS Metering
SM
56
CHAPTER 14 COMPUTER ROOM TRANSFORMER & POWER DISTRIBUTION UNIT (PDU)

Transformers can appear in many places in the power path
Efficiency lost at every conversion Typically no metering
i.e., measurements taken by hand-held instruments
SM
PDUs
With transformer, typically meter output
May be best place to aggregate actual IT equipment energy consumption
Without transformer, metering may be optional

SM
58
PDU
Figure 14.3 Power Flow Through a Power Distribution Unit
SM
59
Rack-mounted PDU (RPDU)

Measurements taken inside the equipment rack or cabinet
Generally impractical for aggregating equipment power consumption
Good for equipment measurement May be multiple units in one rack powering the same equipment
SM
Typically have lower accuracy than upstream devices Many levels of metering options
60
Appendix D - UPS
Overview of UPS types Redundancy and availability
Challenges for power measurement on multiple paths
SM
Rules of thumb for minimum practical level of UPS instrumentation

Example for quick estimation of PUE / DCIE inside the computer room
Sample case study: PUE and DCIE determination for the critical power path within the data center (Partial PUE)
SM
PART 5 IT SYSTEMS MEASUREMENTS
SM
COMPUTER AND STORAGE SYSTEMS (Chapter 15) NETWORKING SYSTEMS (Chapter 16)
Mike Patterson, Ph.D., P.E. (Intel), presenter
Motivation
SM
IT equipment power is the denominator in the PUE Calculation Overview

Workload Characteristics Issues Measurements
Minimum Measurement Best Practical State of the Art
64
Power thru the IT gear

Room Rack a
(Server, Stor, & Comms)

rack power strip or PDU PSU b CPU, etc..
SM
PDU
Server
Measurement location
(a) can give aggregate power (b) best location if actual data is avaiable
65
Process for Measurement

IT Equipment Inventory
Age & self reporting capability Location; remote storage, etc
SM
Data Center Workloads

Power draw can have unexpected patterns; power vs. energy considerations Enterprise, HPC, Financial, Internet, e-mail, etc
Time of day, weekly variations, batch processes Server to server variation due to workload
66
Issues and challenges

Different organizations, different systems Different protocols

SM
IT
IPMI DCMI SMASH
Facility
-BACnet -Modbus -other
Where is it brought together?

Management console, spreadsheet, or back of an envelope?
Redundant Power Supplies

Must measure all feeds and attribute accordingly
Power over Ethernet (POE)

SM
68
Chapter 16 Network; essentially 2010, The Green Grid book for details Technical Forum the same, see the The Green Grid Copyright
Minimum Practical Measurements

Equipment nameplate
Better than doing nothing, but just barely
See Green Grid White Paper #23, Proper Sizing of IT Power and Cooling Loads
SM
PDU Output
Can give actual rack-level power (sometimes energy) Not always tracked on-line
Single server/network/storage measured value

Actual portable power meter Workload, timeframe, scalability questions
69
Best Practical Measurement

Rack or row level PDU output
Rack power aggregate No visibility to individual servers or workloads Easily tracked/trended in BMS system
Majority of PDUs have this capability
SM
Ignores line losses

This is a reasonable assumption
70
State-of-the Art Measurement

On-line server power consumption
New Energy Star for Servers requires this! Challenge is integrating server manageability protocols with BMS systems
IPMI, DCMI, SMASH, Vendor-specific
SM
Extensive server data available

CPU utilization Front panel temperature sensors
On-line network and storage power consumption

Somewhat behind Server; Energy Star on the horizon Capabilities vary on self reporting
Wireless remote sensors

Wide range of data capture
SM
Thank you for attending The Green Grid Technical Forum 2010 For more information, visit www.thegreengrid.org
72

Real Time Energy Consumption Measurements

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Real Time Energy Consumption Measurements

Hochgeladen von

Copyright:

Verfügbare Formate

SM

Real-Time Energy Consumption Measurements in Data Centers

ASHRAE TC9.9 and The Green Grid

ASHRAE/TGG.2009.RealTimeEnergyConsumptionMeasurementsinDataCenters.Atlanta:American SocietyofHeating,RefrigeratingandAirConditioningEngineers,Inc.;coauthoredbyTheGreenGrid, Beaverton,Oregon.

PURPOSE OF THE BOOK

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Copyright 2010, The Green Grid

The Green Grid Technical Forum

MEASUREMENT DEVICES (Chapter 3)

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Part 1 Chapter 3 Sensors

Copyright 2010, The Green Grid

The Green Grid Technical Forum

MEASUREMENT COLLECTION SYSTEMS

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Copyright 2010, The Green Grid

The Green Grid Technical Forum

The Green Grid Technical Forum

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Typical Data Center Meters

Data Center Meters

The Green Grid Technical Forum

Enterprise Energy Dashboard

Initiate program with existing metering data where available

Copyright 2010, The Green Grid

Collecting Data Center Telemetry in an Enterprise

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Managed server monitored by a manager server

Copyright 2010, The Green Grid

The Green Grid Technical Forum

Data Collector Software Architecture

The Green Grid Technical Forum

PART 2 COOLING SYSTEMS Air Measurements

Copyright 2010, The Green Grid

The Green Grid Technical Forum