ANSI/BICSI 002, Data Center Design

and Implementation Best Practices

Rick Ciordia, PE, RCDD, DCDC, RTPM, CT
ETTG, PSC.
CALA District Chair - BICSI
 BICSI International
Standards Program
• Develop standards within all facets of
Information & Communications Technology
(ICT) infrastructure design and installation

• Details:
– Over 450 member worldwide
– Accredited by ANSI
– Develops international open to use/“royalty free”
standards and best practices
 BICSI Standards Within ICT
ISO/IEC
IEEE BICSI
& TIA
Defines the message Defines the transmission Defines how to design
and transmission media and system solutions using transmission
characteristics specifications media and systems
 Standards Development Cycle

Industry Needs
Technology Trends Use
Designers
Suppliers
Clients

Idea to
Consensus Review of
Consensus Draft Ballot Results
Body
 BICSI Published Standards

• Building/Facilities • Installation
– K12 / Primary Education – Cable Installation
Facilities (001) (NECA/BICSI 568)
– Data Centers (002)** – ICT Bonding and
– Healthcare Facilities (004) Grounding
(NECA/BICSI 607)*
• Systems
– ESS (005)*
• Other
– DAS (006)** – BIM for ICT (003)
* Available in Spanish
** Translation into Spanish planned or in progress
 Current Activities
New Standards
• DAS Design
– Moving to publication
• ICT Systems for Intelligent
Buildings
• OSP Installation and
Construction
– General Principles
– Aerial Pathway
Revision Activities
• Revision to BICSI 005, ESS
– IP Fire Alarm
• Revision to BICSI 001,
Educational Facilities
– Expanded scope to include
secondary, graduate,
post-graduate facilities
 About ANSI
(American National Standards Institute)

• Promotes standards use within United States

• Accreditation body
– Standards Development
– Credentialing Bodies (ISO 17024)
– Testing Laboratories (ISO 17025)
• Ensures open and unbiased standards
development processes
• Does not create standards
Reach of BICSI Standards
ANSI/BICSI 002-2014

Data Center Design and

Implementation Best Practices
 Data Center Standards

ISO/IEC 24764 (2010) ANSI/TIA-942-A (2012)

Information technology — Generic cabling Telecommunications Infrastructure Standard
systems for data centres for Data Centers
44 pages 118 pages

CENELEC EN 50173-5:2007 /
ANSI/BICSI 002-2014
A2:2012 Data Center Design and Implementation
Information technology — Generic cabling Best Practices
systems - Part 5: Data centres
534 pages
48 pages

CENELEC EN 50600 (2012-) ASHRAE TC9.9 (2012)

Information technology — Data centre Thermal Guidelines for Data Processing
facilities and infrastructures Environments, 3rd edition
Multiple Documents 150 pages
Comparing the Data Center Standards

• Data Center Design

– BICSI 002, CENELEC 50600
• Computer Room and Cabling Infrastructure
– TIA 942-A
• Cabling Infrastructure
– ISO/IEC 24764, CENELEC EN 50173-5
• Environmental Conditions
– ASHRAE TC 9.9
• Others
– Local / Country
– Certifications
 Design Standard - CENELEC 50600
EN 50600-2-1 (2014)
Building Construction
36 Pages
EN 50600-2-2 (2014)
Power Distribution
40 Pages
EN 50600-2-3 (2014)
Environmental Control
EN 50600-1 (2012)
32 Pages
General Concepts
32 Pages EN 50600-2-4
Telecommunications
Cabling Infrastructure
EN 50600-2-5
Security Systems

EN 50600-2-6
Management and
Operational Information
 BICSI 002, Data Center Design and
Implementation Best Practices

• More than just cabling

• Expands requirements
from other standards
• Covers additional
topics not within other
standards
BICSI 002-2014 Breakdown
Other
7% Site /
Security Building
14% 18%

Cabling /
Networks Electrical
22% 26%
HVAC /
Mechanical
13%

Represents 408 pages of normative content across 13 sections

 Content Revision and Expansions

• Availability Classes • Building Structure

• Modular Data Centers* • Site Hazards
• Hot/Cold Aisles • Data Center Services
• Mechanical Systems* Outsourcing Model*
• DCIM* • Bonding & Grounding
• Circuit Maps and DC • Commissioning
Power • Network Security*
• Cabinet Airflow and • Telecommunications
Cabling Capacity Cabling
• “Green” / Efficiency* • (And More …)

* Indicates all new content to this edition

BICSI Design Classes and
Selection Methodology
 Finding the Right Design

• Methodologies include:
– Loss prevention vs. initial build cost
– Total Cost of Ownership (TCO)
– Desired availability level

• All provide a starting point for design

requirements
 BICSI Design Class Determination

• Based on three questions

1. How much downtime per year will be allowed

for maintenance?
2. During scheduled operation, what is the
maximum allowed downtime?
3. What is downtime’s impact to operations?

• Answers will indicate design class for starting

point of requirements
 Interaction of Answers

Define
Define
Operational Define Impact
Operational
Availability of Downtime
Requirements
Requirements

Identify Required
Availability Class
 BICSI DC Design Classes

• Class 0: Single path, and fails to meet one or

more criteria of Class 1
• Class 1: Single path
• Class 2: Single path with redundant
components
• Class 3: Concurrently maintainable & operable
• Class 4: Fault tolerant
 Availability Class Prefixes

• Class Fx: Facility (Electrical & Mechanical)

• Class Cx: Cable Plant

• Class Nx: Network Infrastructure
• Class Sx: Data Processing and Storage Systems

• Class Ax: Applications

Class F1 Electrical Example
Alternate
Transfer
Utility Power
Switchgear
Source

Electrical Distribution

UPS Mechanical
Static Maintenance Switchgear
Bypass Bypass

PDU

Critical Non-Critical Mechanical

Loads Loads Loads
Class F2 Electrical Example
Alternate
Transfer Power Source N
Utility
Switchgear Alternate
Power Source +1

Electrical Distribution

Static
UPS UPS
Bypass
Mechanical
N +1 Switchgear

Maintenance Bypass

PDU

Critical Non-Critical Mechanical

Loads Loads Loads
 Electrical Class F3
Alternate
Power Source
N
Transfer
Utility
Alternate Switchgear
Power Source
+1

Electrical Distribution Electrical Distribution

Mechanical Mechanical
UPS UPS Switchgear Switchgear

Static N +1 Mechanical
Bypass Loads

Maintenance
Bypass Mechanical
Loads

Output Distribution Switchboard Alternate Switchboard

Critical Mechanical Critical Mechanical

Switchgear Switchgear

PDU Critical PDU

Fans/Pumps

Non-Critical Critical Non-Critical

Loads Loads Loads
 Electrical Class F4
Alternate Alternate
N Power Source
N
Power Source Transfer Transfer
Utility Switchgear
Alternate Switchgear Alternate
+1 Power Source
+1
Power Source

Electrical Distribution Electrical Distribution

UPS UPS Mechanical Mechanical UPS UPS

Switchgear Switchgear
Static N +1 N +1 Static
Bypass Bypass
Maintenance Mechanical Maintenance
Bypass Loads Bypass

Output Distribution Switchboard Output Distribution Switchboard

Critical Mechanical Critical Mechanical

Switchgear Switchgear

Critical
PDU Fans/Pumps PDU

Non-Critical Non-Critical
Loads Loads
Critical
Loads
Class F4 Electrical “Catcher”
Utility Utility

High Voltage High Voltage High Voltage

Receiving 1 Receiving 2 Receiving 3

High to Medium Voltage Alternate Alternate + High to Medium Voltage

Power Source
N Power Source 1 Switchgear
Transformer 1 Transformer 2

Medium Voltage Medium Voltage

Distribution 1 Distribution 2

Medium Voltage Electrical Distribution 1 Medium Voltage Electrical Distribution 2

Source 2
Source 1

Source 2

Source 2
Source 1

Source 1
Source 2
Source 1
Source Source Source Source
1 2 1 2

Stand- Stand- Normal Stand-

Normal Normal
Static Bypass Input
Static Bypass Input

Static Bypass Input

by by UPS by
UPS UPS
UPS UPS UPS
N N N
+1 +1 +1

Main Stand-by Main Stand-by

Overlap Switch To Mechanical Systems Overlap Switch

(Manual) See Section 10 for configuration (Manual)
of mechanical systems
PDU-A PDU-B

Non-Critical Critical Non-Critical

Loads Loads Loads
Telecommunication Classes
Service Service Service Service
Provider Provider Provider Provider

Class C4
C2
C1
C3
Maintenance Maintenance
Hole Hole

Entrance Room Entrance Room

Offices,
Operations Center,
Computer
Support Rooms
Room

Telecomunicaciones Main Dist Area Main Dist Area

Room

Horiz Dist Horiz Dist Horiz Dist Horiz Dist

Area Area Area Area

Zone Dist Zone Dist Zone Dist

Area Area Area

Equip Equip Equip Equip Equip Equip Equip Equip

Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area Dist Area

ITE ITE ITE ITE ITE ITE ITE ITE

Only required for large computer rooms due to port

counts or distance limitations
 Air-Cooled
Condensers
Mechanical Class
Air-Cooled
Condenser
(Dry-Coolers)
F1
F2
(Dry-Cooler)

Chiller Chiller • No
Redundant
redundancy
critical
present
components
Chiller • All power feeds from
CRAH common upstream
CRAH distribution
• Only
No ability
redundant
to be
maintained under
components able to
load
be maintained under
load
 Air-Cooled Condensers (Dry-Coolers)
Air-Cooled
Condensers
Mechanical Class
(Dry-Coolers)
F3
F4
Chiller
Chiller Chiller Chiller
Chiller • Redundant equipment
Redundancy in all areasand
• piping for maintenance
Equipment and controls
• with redundant
Power feed so that
systems
cooling
fed
from different
capacity does not
electrical
drop
distribution
below "N" when
CRAH
CRAH
• maintaining
Maintainablemechanical
when actionsor
upstream electrical
do not reduce cooling
distribution
capacity below "N+1"
• Maintainable when actions
do not decrease cooling
capacity below "N"
Class N0/N1 and N4 Network
Class S4 System and A4 Application
 Availability and
Multi-Site Data Centers
• Prior to virtualization, subclasses aligned through
data center

• Today, a single data center may not have

alignment

• Availability class methodology can in discussions

about using multiple data centers to achieve
availability target
Example: Class 3 Availability Using
Three Class 2 Data Centers
 Modular Data Center Design

• Modular may describe:

– How a data center is segmented
– Method of construction/implementation

• Both can assist in allowing for different

availability needs for one site
Modular/Containerized Data Center
Space Adjacencies
F4
F3
F2
Modular Design Planning Aides
BICSI-002, 2014 Supplement
 Are BICSI & Uptime Similar?
• ANSI/BICSI 002-2014
This standard provides a reference of common
terminology and design practice … a framework for
the process to determine facility criticality and to
develop optimum design & implementation solutions

• Uptime Tiers
“Only data center benchmarking system developed by and
for data center owners
Performance-based on fundamental concepts
– Not a checklist, design menu, or cookbook”

Source: Uptime Institute: Tier Classification System & Operational Sustainability presented by
Dana Smith, Director of Development, Uptime Institute at BICSI Andino 2012
Standards

BICSI
 Colombia
o NTC-6064 - Tecnología de la Información. Cableado
Genérico para instalaciones de clientes.
 Especifica el cableado genérico para uso dentro de
instalaciones que pueden comprender una o varias
edificaciones en un predio (campus).
 Comprende el cableado balanceado y el cableado de fibra
óptica.
 Referencia: ISO/IEC 11801
 Colombia (Cont.)
o RITEL - Reglamento Técnico para Redes Internas de
Telecomunicaciones
 Establece las medidas relacionadas con el diseño/
construcción y puesta en servicio de las redes internas de
telecomunicaciones.
 Referencia: ISO/IEC 11801
 Costa Rica

o Colegio Federado de Ingenieros y Arquitectos

 Todos los planos de instalaciones eléctricas y de
telecomunicaciones, de voz y datos deberán cumplir
donde corresponda con:
 El NEC Código Eléctrico Nacional (NFPA 70) en su última versión en
español.
 NFPA 70 E “Norma para la seguridad eléctrica de los empleados en
los lugares de trabajo”, en su última versión en español.
 Las últimas revisiones y adenda aprobados de las normas
ANSI/EIA/TIA 568, 569, 570, 606, 607.
 Mexico
o NMX-I-108-NYCE-2006, Telecomunicaciones – Cableado – Cableado
estructurado – Puesta a tierra en sistemas de telecomunicaciones
o NMX-I-132-NYCE-2006, Telecomunicaciones – Cableado – Cableado
estructurado – Especificaciones de las pruebas de cableado balanceado –
Parte 01: Cableado instalado
o NMX-I-154-NYCE-2008, Telecomunicaciones – Cableado – Cableado
estructurado – Cableado genérico residencial
o NMX-I-248-NYCE-2008, Telecomunicaciones – Cableado – Cableado
estructurado – Cableado de Telecomunicaciones para edificios comerciales
– Especificaciones y métodos de prueba
o NMX-I-279-NYCE-2009, Telecomunicaciones - Cableado - Cableado
estructurado - Canalizaciones y espacios para cableado de
telecomunicaciones en edificios comerciales
 Mexico (Cont.)
o NMX-I-14763-1-NYCE-2010, Telecomunicaciones – Cableado – Cableado
estructurado – Implementación y operación de cableado en edificios
comerciales – Parte 1: Administración
o NMX-I-24764-NYCE-2013, Tecnología de la información – Sistema de
cableado genérico para centros de datos
o NMX-J-364/I-ANCE-2011, Instalaciones eléctricas – Parte 1: Principios
fundamentales, planeación de características generales, definiciones
o NMX-J-C-I-489-ANCE-ONNCCE-NYCE-2014, Centros de datos de alto
desempeño sustentable y energético – Requisitos y métodos de
comprobación
o NOM-001-SEDE-2012, Instalaciones Eléctricas (Utilización)
ANCE - Asociación Nacional de Normalización y Certificación del Sector Eléctrico
NMX - Norma Mexicana
NOM - Norma Oficial Mexicana
NYCE - Normalización y Certificación Electrónica
ONNCCE - Organismo Nacional de Normalización y Certificación de la Construcción y Edificación
 Standards In Use
Country NEC TIA ISO BICSI Uptime ICREA National Other
Chile X X
Colombia RL X R X X X RN
Costa Rica RLN R X
El Salvador X X X
Guatemala X X X
Honduras X X X
Mexico RN X X X X X RN
Panama X X X
Peru X X
Puerto Rico R X X
Republica
X
Dominicana
Uruguay X X X X
Venezuela RL R R R
Notes: X = Being Used Not Required RN = National Version
R = Required by law, regulation, etc. RL = National Version based on NEC
RLN = NEC version adopted as national
 DC Standards
Organization BICSI TIA ISO CENELEC ASHRAE ICREA
BICSI 002 TIA 942A ISO 24764 EN 50173-5 ASHRAE ICREA-Std-
Standard
(2014) (2012) (2010) / EN 50600 TC9.9 131-2013
Int’l
US /Canada EU US/Canada
Recognition Worldwide Worldwide (21
(Int’l) (Int’l) (Int’l)
Countries)

Electrical Yes Yes Yes

Mechanical Yes Yes

Areas Covered

Telecom Yes Yes Yes Draft Yes

Environment Yes Yes Yes Yes Yes

Site Yes
Security Yes Draft Yes
 Planned Interoperability
Government Regulations

(BICSI 002-2014)
Data Center Electrical Code
Building Code
Fire Code
Electrical Standards
Cabling Standards
Cabling Spaces Standards
Environmental Conditions
Building System Standards
Security Standards
Interoperability Demonstrated
Industry
Government
Canada,
Government
FCC, OSHA
Labour
Regulations
Program
States
(BICSI 002-2014)
Rica
Colombia CSA
Electrical
NFPA
C22.1
70 (NEC)
Code
(CEC)
International
National Building
Government
BuildingBuilding
CodeCodeCode (IBC)
of Canada (NBC)
Canada
Government,
NFPA
National FireFire
1,Code
CodeNFPA NFPA
72 (NFC)
of Canada
Electrical
IEEE,
IEEE,
CSA,
NFPA,
Standards
NFPA
IEEE
NECA
Costa
United

Cabling
RITEL,
TIAStandards
568
ISO
Cabling Spaces
TIAISO
942Standards
Environmental
ASHRAEConditions
Building
TIA 862,
System
ISO
AHSRAE
Standards
13
NFPA,
RCMP,
Security
Government,
Government
ASIS,
Government,
Standards
Government
ISOISO
 Why BICSI 002?

“If you fail to plan,

you are planning to fail!”
“¡Si fallas en planificar,
estas planificando fallar!”
- Benjamin Franklin
 Thank You!

Rick Ciordia, PE, RCDD, DCDC, RTPM, CT

ETTG, PSC.
rciordia@ettgpr.com