BPSDC

(Data Center Networks & Cloud

Computing Security)

Lecture 3

Data Center
Standars
Service Level Agreement

2
Building a Data Center is just a Start …
What is Service Level Agreement (SLA)?
An official commitment between the service provider
and a client
Can be a legally binding formal or an informal "contract"
Originally used by fixed line telco operators from 1980s
Commonly includes several components, from a definition of
the services to the termination of the agreement
Definition of type of service to be provided
The service's desired performance level (+ reliability and
responsiveness)
Monitoring process and service level reporting
Steps for reporting issues with the service
Response and issue resolution time-frame
Repercussions for service provider not meeting 3
commitment, especially financial
Where and how does SLA apply?
Where are we able to find SLA?
Backbone Internet Providers
Web services
e.g. the availability of REST API to customers
Data Centers (both shared, on-premise and outsourced)
Cloud computing shared resources SLA
Example SLA (one of the Czech/Itallian Providers)
100% uptime for power and cooling
99,95% Internet connectivity
99,95% physical node availability for Virt. infrastructure
Servers
99,8% access to provided physical nodes 4
What does SLA not cover?
“Higher power” aka “act of God” aka “Force Majeure”
wars, terrorism, strikes, traffic accidents,
sometimes also natural disasters (see previous lecture)
Extraordinary interventions to be carried out urgently
to avoid hazards to safety/stability/confidentiality/integrity
typically announced in advance to customers (e.g. 48h
before execution when possible or ASAP)
Unavailability or blocking of the infrastructure due to
Customer actions (shutdown of servers, abuse, misconf.)
3rd party OS or applications used
non-fulfillment or breach of Contract by customer
Internet or connectivity problems caused by customer or 3rd
parties 5
Planned maintenance (normal amount)
Data Center Standards

6
A Data Center must follow some …
Guidelines/Best practices
ANSI/BICSI 002, Data Center Design and Implementation
Best Practices (USA → International)

Standards
TIA 942 (USA)
ISO/IEC 24764 → ISO/IEC 11801-5 (Worldwide)
EN 50600 series (WiP) + EN 50173-5 (EU)
…

Certification requirements
Uptime Institute Tier certification (Worldwide) 7
Building Industry Consulting Service International 002
DC Design and Implementation Best Practices (1)
Site selection – hazards, Electrical systems – utility serv.,
environments, access, distribution, mechanical, UPS,
regulations standby and Emergency,
Space planning – capacity, Automation & Control, Lighting,
power, cooling, supporting Protection, …
spaces, IT Equipment Fire Protection – walls, floors,
placement, network ceilings, aisle containment,
Architectural – design extinguishers, protection,
concepts, access paths, detection, …
planning details, construction DC Management and Building
components Systems – building automation
Structural – general, specific systems, electronic safety and
Mechanical – classes, cooling security systems 8
conditions, thermal, mech., …
Building Industry Consulting Service International 002
DC Design and Implementation Best Practices (2)
Security – physical s. plan, risks Backbone & horizontal c.
& threats, regulatory & Installation, testing, racks
insurance, DC security plan, Information Technology –
crime prevention, access control, disaster recovery, c. room
alarms, barriers, lighting, layout, communication,
surveillance, guards, disaster operations center, network
recovery, building site infrastructure reliability, security
considerations, building shell, Commissioning (+testing)
DC security Maintenance (of all systems)
Telecommunications, Cabling, Annexes (informative)
Infrastructure, Pathways, Spaces Design Process
C0-C4 Cabling class, Reliability & Availability
topologies, spaces, pathways, Alignment, Outsourcing 9
access providers Multi-DC arch., energy efficiency
BICSI 002 – Annex B – Operational Requirements
Annual
Operational
Planned Description
Level
Downtime
Operational less than 24 hours a day & less than 7
0 > 400 h days a week. Scheduled maintenance “down” time
available during working hours and off hours.
1 100 – 400 h As above.
Operational up to 24 hours a day, up to 7 days a
2 50 – 99 h week, and up to 50 weeks per year. Scheduled
maintenance “down” time as above.
Functions are operational 24 hours a day, 7 days a
3 0 – 49 h week for 50 weeks or more. No sch. maintenance
“down” time is available during working hours
Functions are operational 24 hours a day, 7 days a
4 0h week for 52 weeks each year. No scheduled
10
maintenance “down” time is available
BICSI 002 – Annex B – Downtime Impact
Classification
Description
(Impact)
Local in scope, affecting only a single function or operation,
Isolated
resulting in a minor disruption or delay in achieving non‐critical
(Sub-Local)
organizational objectives
Local in scope, affecting only a single site, or resulting in a minor Minor
disruption or delay in achieving key organizational objectives (Local)
Regional in scope, affecting a portion of the enterprise or
Major
resulting in a moderate disruption or delay in achieving key
(Regional)
organizational objectives
Multiregional in scope, affecting a major portion of the enterprise
Severe
or resulting in a major disruption or delay in achieving key
(Multiregional)
organizational objectives
Affecting the quality of service delivery across the entire
Catastrophic
enterprise, or resulting in a significant disruption or delay in
(Enterprise)
11
achieving key organizational objectives
BICSI 002 – Annex B – Data Centre Class

Facility Availability Classes

F0/F1 – Single path (maps to T-1, R-1, AC-1)
F2 – Redundant components (maps to T-2, R-2, AC-2)
F3 – Concurrent Maintainability (maps to T-3, R-3, AC-3)
12
F4 – Fault Tolerant (maps to T-4, R-4, AC-4)
BICSI 002 – Annex B – Availability Requirements

Allowable Annual Downtime Allowable Availability

(minutes) (Uptime 9s next lecture)
> 5000 > 99%
500 – 5000 99% ‐ 99.9%
50 – 500 99.9% ‐ 99.99%
5 – 50 99.99% ‐ 99.999%
0–5 99.999% ‐ 99.9999%

13
TIA-942 – Telecommunications Infrastructure
Standard for Data Centers (1)
Specifications for DC telecommunications pathways & spaces
Recommendations on media & distance restrictions for structured
cabling system and applications over it (2005)
Telecommunication spaces and topologies
Cabling, pathways, redundancy, Informative annexes: Design,
administration, access provider information, equipment plans,
dataspace considerations, site selection, tiers, examples,
references
Components known from TIA-568
Addendum 1 (2008) – usage of 75 Ω coaxial cable
Addendum 2 (2010) – additional guidelines for DCs – lighting in 3
tiers, recommendation from CAT-6/6A to CAT-6A only
(minimum required category is Cat-6) 14
TIA-942 – Telecommunications Infrastructure
Standard for Data Centers (2)
TIA-942-A (2012)
harmonization with TIA-568C
left some limitations to other standards (removed from here)
removed 100m limitation for optical fibers
multi-mode cable possible for horizontal & backbone cabling
use of LC & MPO connectors for optical fibers
Introduced Intermediate Distribution Area (IDA)
Zone Distribution Area (ZD) can contain only passive components
energy efficiency recommendations, harmonized with IEC 24764
TIA-942-A Addendum 1 (2013) – mainly data center fabric
topologies examples, new switch topologies
Fat tree, full mesh, inter-connected meshes,
15
Centralized switch, virtual switch
TIA-942 – Telecommunications Infrastructure
Standard for Data Centers (3)
TIA-942 Revision B (2017)
Added Cat-8 cabling, recommended cabling Cat-6A or higher
Maximum EDA cable length 10 → 7m
at least 1200mm deep cabinets, considerations for 24”+ (600mm+)
cabinets,
pre-terminated cabling,
labeling, cable routing, adding/removing cords, …
MPO-16 and MPO-32 connectors for 200G and 400G
Wideband multimode fiber (WBMMF) cable added
ANSI/TIA-568-C.4 coaxial cables and F connectors may be used
Normative references to other standards, including revised
references to temperature and humidity guidelines
Modifications for use outside of US, optical cable quality req. 16
TIA-942 – Ratings of Data Centres (1)

Rated-1: Basic Site Infrastructure

Single capacity components and a single, non-redundant
distribution path serving the computer equipment.
Limited protection against physical events
May not even have a raised floor
Susceptible to disruption from planned & unplanned activities
28.8 hours of annual downtime permissible
1 entrance pathway from access provider to facility, single pathway
for all cabling

17
TIA-942 – Ratings of Data Centres (2)

Rated-2: Redundant Capacity Component Site Infrastructure

Redundant capacity components and a single, non-redundant
distribution path serving the computer equipment.
Improved protection against physical events
Does have to use a raised floor
Slightly less susceptible to disruptions
22.0 hours of annual downtime permissible
Requirements of Rated-2 must be observed, also
2 entrance pathways from access provider to facility exist
Routers & switches have redundant power supplies & processors
Vulnerability of service entering building is addressed
N+1 redundant UPS modules, single generator 18
TIA-942 – Ratings of Data Centres (3)
Rated-3: Concurrently Maintainable Site Infrastructure
Redundant capacity components and multiple independent
distribution paths serving the computer equipment (power, data,
cooling). N+1 rule for everything.
Typically, one single distribution path serves the computer
equipment at any time.
Protection against most physical events
The site is concurrently maintainable – each & every capacity
component incl. elements which are part of the distribution path,
can be removed/replaced/serviced on a planned basis without
disrupting the ICT capabilities to the End-User.
1.6 hours of annual downtime
Requirements of Rated-2 must be observed, also
requires at least 2 access providers + a secondary entrance room
backbone pathways have to be redundant
multiple routers and switches must be included for redundancy 19
Vulnerability of a single access provider is addressed
TIA-942 – Ratings of Data Centres (4)
Rated-4: Fault Tolerant Site Infrastructure
Redundant capacity components and multiple independent
distribution paths serving the computer equipment.
All redundant capacity components and independent distribution
paths are active at the same time. 2(N+1) for all components
Protection against almost all physical events.
The data center allows concurrent maintainability and one fault
anywhere in the installation without causing downtime.
All computer hardware must have dual power inputs
Can sustain at least one worst-case, unplanned failure or event
with no critical load impact
0.4 hours (18 minutes) of annual downtime
Requirements of Rated-3 must be observed, also
requires redundant backbone cabling, which should be in conduit or
have interlocking armor, optional secondary distribution area
optionally, horizontal cabling is also redundant 20
Addresses any vulnerability of the cabling infrastructure
ISO/IEC 11801-5 – Generic Cabling for Customer
Premises – Part 5: Data centers (1)
Latest revision ISO/IEC 11801-5:2017
Balanced & optical fibre cabling specifications, normative parts:
Structure of the generic cabling system
Channel performance requirements
Link performance requirements
Reference implementations
Cable requirements
Connecting hardware requirements
Requirements for cords and jumpers
Annex A - Combination of balanced cabling links

21
ISO/IEC 11801-5 – Generic Cabling for Customer
Premises – Part 5: Data centers (2)
Informative Annexes (optional):
Usage of high density connecting hardware within optical fibre
cabling
Examples of structures in accordance with ISO/IEC 11801-5
Data center minimum configuration
End of Row concept
Middle of Row concept
Top of Rack concept
End of Row and Middle of Row concept with redundancy
Top of Rack concept with redundancy
End of Row and Middle of Row concept with full redundancy
Top of Rack concept with (full) redundancy
Examples of networking fabric architectures: fat-tree, full-mesh,
interconnected meshes, centralized switch, virtual switch 22
ISO/IEC 11801-5 – Cabling
Cable classes
Twisted pair (100 Ω impedance)
Class EA: link/channel up to 500 MHz Cat-6A cable/connectors
Class F: link/channel up to 600 MHz using Cat-7 cable/connectors
Class FA: link/channel up to 1000 MHz using Category 7A
Class I/II: link/channel up between to 1600 and 2000 MHz using
Category 8.1/8.2 cable/connectors
2-4 mated connectors per copper channel, RJ-45 or TERA connector
Optical fiber interconnect using multi-mode fibre
OM3: Multimode fiber 50µm, min. modal bw of 2000 MHz*km at 850 nm
OM4: Multimode fiber 50µm, min. modal bw of 4700 MHz*km at 850 nm
OS1/OS2: Single-mode fiber type 1 dB/km / 0.4 dB/km attenuation
duplex LC (2 fibers) or MPO (3+ fibers) connector
Channel length is determined by media choice 23
ISO/IEC 11801-5 – Data Centre Topologies

Fat tree without port extenders

Standard 3-tiered architecture

24
Port extenders Full mesh Interconnected meshes
EN 50173-5 – IT – Generic cabling systems
Part 5: Data centres
Structure of the generic cabling system in data
centres
Channel performance in data centres
Reference implementations in data centres
Cable requirements in data centres
Connecting hardware requirements in data centres
Requirements for cords and jumpers in data
centres
25
EN 50600 series – IT – Data centre facilities
and infrastructures
EN 50600-1 – General concepts
EN 50600-2-1 – Building construction
EN 50600-2-2 – Power distribution
EN 50600-2-3 – Environmental control
EN 50600-2-4 – Telecommunications cabling infrastructure
EN 50600-2-5 – Security systems
EN 50600-3-1 – Management and operational information
EN 50600-4-1 – Overview of and general requirements for
key performance indicators
EN 50600-4-2 – Power Usage Effectiveness
EN 50600-4-3 – Renewable Energy Factor 26
EN 50600-2-5 Security Systems
Physical security – general, risk assessment
Designation of data centre spaces - Protection Classes
Protection Class against unauthorized access
Protection Class against fire events igniting within data
centre spaces
Protection Class against environmental events (other than
fire) within data centre spaces
Protection Class against environmental events outside the
data centre spaces
Systems to prevent unauthorized access
Informative Annex – Pressure relief: Additional information
27
EN 50600 – Availability classes

28
EN 50600 – Protection classes

29