9/21/2017 PAC World magazine : Digital Substation ?

the Next Generation Smart Substation for the Power Grid

Home . June 2016 Issue . Digital Substations . Digital Substation ? the Next Generation Smart Substation for the Power Grid

Digital Substation ? the Next Generation Smart Substation for the Power Grid
S. Richards, GE Grid Solutions, UK and J. Arnaud, GE Grid Solutions, France
Drivers towards Digital Substations
Increased reliability and availability, reduced maintenance cost:The extensive self-diagnosis
capability of digital devices ensures maximized up-time of the substation. Any degradation in the
performance of an asset is pinpointed in real-time. Inherent redundancy in the system may be employed
to self-heal the operation, which permits troubleshooting without the need for any primary system
outage.
Optimized operation of assets – situational awareness: The digital substation closely monitors all
substation assets. Intelligent systems analyze the data and provide recommendations on maintenance
and repair actions to conduct.
The intelligence within digital substation schemes allows close monitoring of the loadflow capacity of
plant equipment, compared to its design ratings.
Safety:Digital substations eliminate dangerous cross-site CT circuits, and remove the explosion risk by
using oil-free optical CTs.
Substation reduction footprint: Stripped out of their I/O boards, IEDs are much smaller (typically half-
size) and can be packed into fewer cabinets. Digital instrument transformers are small and light, leading
to a reduction in land used by the substation.
Generic Architecture
The process level in the switchyard: Digital applications in the substation are based on a
communicating architecture, whereby real-time operational measurements are polled from the primary
system. It is communicated to devices which must act on those measurements by means of a “process
bus.” (Figure 1)
Control commands (switchgear operator commands, protection trips) also are routed to the primary
devices via the process bus, in the opposite direction.
The protection and control level (“smart substation area”): Between the process bus and the station
bus are devices historically identified as the “secondary equipment.” These devices are IEDs (intelligent
electronic devices), interacting with the field via the process bus, and with other peer devices in the bay,
to other bays, and the digital control system via the station bus.
The station control area: The Digital Control System is the intelligence which binds together the digital
substation. It is central to the flow, management and presentation of all components in the digital
substation. Wide area control units (WACU) offer the possibility to exchange IEC 61850 GOOSE data
between voltage levels within a substation and also between neighboring substations.
Digital Instrument Transformers and Process Bus Devices
The root of many of the limitations of conventional instrument transformers is the reliance upon an iron
core. Instead of an iron core, the translation from primary to secondary measurement may use optical,
Rogowski or capacitive technology.
Examples of the principles are:

Small footprint optical sensors use the Faraday Effect, whereby a fiber optic loop sensor carrying a
polarized light beam encircles the power conductor. This light will experience an angular deflection
due to the magnetic field, generated by the primary current flow
Low power voltage transformers implement a capacitive divider stack to produce the output voltage
for digitization in the merging unit. The novel design does not have the traditional wound VT
output stage of a traditional CCVT (capacitor-coupled voltage transformer)

Merging units perform all the data processing necessary to produce a precise output data stream of
sampled values according to IEC 61850-9-2LE. In a fully-digital architecture, protection IEDs receive
currents and voltages as IIEC 61850-9-2LE sampled values, and issue trip or alarm signals using IEC
IEC 61850-8-1 GOOSE. For retrofitting, or where the client has a preference to retain traditional
instrument transformers, analog merging units are available, digitizing the CT and VT analog outputs at
any convenient kiosk out in the yard. Such kiosks may also embed the switchgear control unit (SCU),
which may not only be used to actuate the switchgear in the yard and marshal its positional and status
information, but also it monitors the oil and SF6 gas health.

System Architecture
A reliable protection scheme should be fully redundant, for example with system A in the substation
building and system B in a container in the yard. Each has internal control/relay panels reduced in size
and number due to the small footprint of the process bus relays and bay computers. Both such IEDs are
miniaturized, delivered in a 40TE footprint, typically half that of traditional equivalent devices.
The substation can mix technologies, with optical, low power and conventional instrument transformers
deployed within the architecture. For
https://www.pacw.org/no-cache/issue/june_2016_issue/digital_substations/digital_substation_the_next_generation_smart_substation_for_the_po… 1/4
9/21/2017 PAC World magazine : Digital Substation ? the Next Generation Smart Substation for the Power Grid
deployed within the architecture. For
interoperability, the conventional CT/VT
outputs are digitized by an analog merging
unit, time-optimized for transmission-class
protection duties (the analog merging unit is
able to publish IEC 61850-9-2LE sampled
values with less than ¼ ms of latency, and
the SCU can process an incoming IEC
61850-8-1 trip and energize the circuit
breaker trip coil contacts in less than 1ms).
In order to achieve a level of dependability
(the ability to operate when required to do
so) and speed which is equivalent to, or
better than, a traditional substation scheme,
the Ethernet architecture design is of
paramount importance. All digital
substation architectures can be set up as an
IEC 62439 standards-compliant self-healing
ring (HSR protocol) or dual-homing star
(PRP protocol); both of which are
“bumpless” redundant. This means that data is exchanged between devices via two diverse paths, and
should one of these paths fail, data is instantly available hot from the other, with zero delay. Both HSR
and PRP are used, as shown in (Figures 2 and 3.)
For PRP, each intelligent device is typically a doubly-attached node (DAN), connected to LAN A and LAN
B, to use a redundant network for common failure mode elimination. For HSR, the ring architecture
allows digital messages to pass both in the clockwise and anticlockwise directions, for communication
continuity in the event that a failure breaks the ring at one point. (see Figure on page 46)

System security
As the substation moves toward a digital
world and utilities' communication
networks improve, new communication
types in the substation LAN become
possible (such as remote access for
maintenance), bringing opportunities for
savings, but also security concerns.
Threats to control systems can come from
numerous sources, including hostile
governments, terrorist groups, disgruntled
employees, malicious intruders, accidents,
natural disasters, etc. In the past, the goals
of viruses and worms ranged from simply
destroying their host to transforming their
host into a spam or a DoS attack bot. But
in the last few years, new stealth malware
appeared that was specifically targeted at Industrial Control Systems, such as Stuxnet, Flame and
Havex, making the menace more real. More recently, in December 2015, hackers penetrated three
utilities in Ukraine and remotely operated the breakers, disconnecting 225 000 consumers.
Consequently utilities and vendors now face the burden of securing the substation, from a cyber-security
angle. Over the years, regulations such as NERC CIP, standards bodies such as IEEE, IEC and working
groups in CIGRE have published requirements, standards and recommendations to achieve better
security. Some of them have an impact on the communications architecture.
Defense-in-depth
The chosen solutions must therefore restrict
access to the substation to authorized users
and deny malware propagation without
changing the substation automation
software whilst, at the same time,
minimizing the management overhead. This
can be done with a series of security layers
which combine into a ‘defense-in-depth’
strategy. This layered arsenal is capable of
withstanding or minimizing the impact of a
failure in any one layer.
Security practices at the business process,
network, host and application levels provide
multiple layers of protection, like moats, high
walls, secondary fortifications and a
dungeon would protect a castle. (Figure 4)
Information systems need to communicate
with one another: the substation, SCADA
control center, network operation center, and
corporate site are interconnected, usually
sharing common infrastructure, for efficient
operation. All those systems interoperate
but also must be defended against threats
emanating from each other.
Network segregation, as its name implies,
serves to keep different systems as discrete
networks (LANs), so that they can’t communicate with one another. Network segregation adds rules to
control and monitor the communication exchanges between LANs.
Network segregation is important to limit network propagation or lateral movement after a first element of
the system is compromised.
When a piece of malware penetrates a system, it scans the network looking for other targets to
compromise, jumping from host to host towards its final goal or simply looking to replicate itself as much
as possible. Network segregation largely reduces the number of devices that can be reached and the
protocols that can be used to communicate with them. A variety of technologies are available to achieve
segregation:

Physical separation of hardware and cables

Separation of hardware and data
Virtual LANs (VLANs) and private VLANs
Network Access Control
Tunnels (GRE, VPN, IPSec)
Network firewalls
Host-based firewalls

https://www.pacw.org/no-cache/issue/june_2016_issue/digital_substations/digital_substation_the_next_generation_smart_substation_for_the_po… 2/4
9/21/2017 PAC World magazine : Digital Substation ? the Next Generation Smart Substation for the Power Grid
Application firewalls and intrusion prevention systems (IPS)

Network Segmentation at the Substation Boundaries

NERC CIP005-5 requires that the substation LAN resides with an electronic security perimeter (ESP)
protected by a single access point effectively providing network segregation between the substation LAN
and the rest of the communication network, such as other substations or the control center.

Network segmentation between different

types of traffic (remote maintenance,
SCADA, video surveillance…) is achieved
using virtual private networks (VPNs) over a
variety of technologies (BGP/MPLS, IPsec,
GRE,…) that require specific hardware and
networking expertise. This type of
segmentation usually extends to the
substation's perimeter.
Another type of segregation can be
achieved by using the remote desktop
protocol (RDP) to remotely access a PC in
the substation. This achieves separation of
hardware and data: the remote PC is
effectively just a remote keyboard/display
which neither hosts nor shares any data
used inside the substation perimeter
(Figure 5).
Network Segregation inside the
Substation
Inside the substation, the LAN is divided
into different “zones”. Communication
between each zone is controlled at several
levels:

Network access control: an access control list (ACL) is assigned to users or devices based on
authentication
The router authorizes communication based on the ACL: which IP addresses (OSI layer 3) and
VLANs (OSI layer 2) can be reached
The network firewall further authorizes communication based on the protocols and ports (OSI layer
4)
The application firewall (IPS) authorizes communication based on the payload content (OSI layer
7). The application firewall understands the application protocol

Note the use of “authorize” rather than “restrict”. A good practice is to implement a default “deny all” rule
and whitelist only the known traffic. Also, note that segregation happens at different levels of the OSI
model for heightened resilience.
The typical zones in the substation LAN are:

DMZ for remote access

Protection and control LAN
Physical security, video surveillance

All communications initiated from outside

the substation should be routed to the
DMZ only. Communication to the
protection and control LAN should be
authorized only from the DMZ.
Network Segregation in the Protection
and Control LAN
Network segregation can be further refined
within the PAC LAN. For example, a
SCADA local operator interface PC and an
IED should only communicate using the
IEC61850 protocol and don’t need to see
each other’s maintenance or time
synchronization ports. By putting the PCs
and IEDs in separate zones (or VLANs),
they are better protected from each other.
IEC61850, HSR, PRP and IEEE1588
IEC61850-8-1 GOOSE and IEC 61850-9-
2LE sampled value messages are non-
routable which means that they are
confined to a single LAN (or VLAN) so
devices that rely on these messages must
be in the same network segment.
IEC61850-90-5 opens the route to routable
GOOSE that could relieve that constraint
and allows securing of PMU and teleprotection applications. At time of writing, there are no PRP/HSR
compliant routers/firewalls on the market. It means that redboxes are needed, introducing more
complexity in the topology (Figure 6).
In addition, VLAN management in an HSR ring is cumbersome as the switch port configuration must be
done on each HSR device. IEEE1588 (and time synchronization in general) is a service that must
access all network segments as it is important that all devices on the network be synchronized from a
single source. It makes the time server a priority target for attackersas its traffic spans all zones.
Network Monitoring Center
All devices that participate in network security will generate logs, particularly alarms when traffic attempts
to violate a rule. It is important that the alarm is received by a network monitoring center where a security
group can then respond to the alarm. Too often, this group doesn’t exist and the utility forwards the alarm
to the SCADA, usually using an SNMP to IEC61850 converter. This is bad practice as it breaks the
segmentation between security and operation. In addition, a dispatcher may not be a security specialist
and may respond to an alarm inappropriately.
Scalability and Cost
/> The illustrations above show a network topology where the router, firewall (usually a single appliance)
and redboxes are in the substation. For availability reasons, all are doubled. A large TSO with 1000
substations would then have to install and maintain 8 000 devices.
With 10 protocols and ports and multiple destinations (SCADA, maintenance center, network operation
center…) the number of rules can run to the order of thousands (Table 1). This does not scale very well.
Some strategies can be used to simplify
https://www.pacw.org/no-cache/issue/june_2016_issue/digital_substations/digital_substation_the_next_generation_smart_substation_for_the_po… 3/4
9/21/2017 PAC World magazine : Digital Substation ? the Next Generation Smart Substation for the Power Grid
Some strategies can be used to simplify
management of the system:

Substations can be designed such

that the same rules apply equally to
them all: same zones, same IP
address plan. This maintains just a
single set of rules that can be
deployed on all firewalls
Firewalls can be centralized in a hub-
and-spoke architecture. Zones must
be carefully-designed such that a loss
of communication to the firewall does
not lead to a loss of local protection
and control availability. This usually requires
that a router with a simple ACL is installed in
the substation

Whichever is the architecture, the challenge

is significant. Global solutions must be
designed from the outset, achieving better
scalability than piecemeal substation-by-
substation approaches (Figure 7)
Biographies
Simon Richards is the Product Line
Leader for the Transmission Relay products
within Grid Automation at GE. He is
responsible for the global product offering
of protection relays, digital instrument
transformers, recording, etc., (including
GE's UR, MiCOM, HardFiber, COSI and
Reason ranges.) Previously, Simon led
product marketing within Alstom Grid,
before the current alliance with GE. Here
he was also the Digital Substation
marketing coordinator for Alstom Grid as a
whole. He held several protection
application engineering and regional
business development positions within
Alstom.Previously, Simon held a 25kV electrification Distribution Engineer position for the 500km West
Coast Main Line railway in the UK. He has a B.Eng (Hons) in Electrical and Electronic Engineering from
the University of Bath, and is a Chartered Engineer.
Jerome Arnaud is an engineering graduate of Ecole Centrale de Nantes, France, having majored in
Automation and Control. He has worked at GE Grid Solutions (formerly Alstom Grid) since 2003,
successively as a software engineer and project manager in charge of Electricity Market Management
Systems, as the R&D Manager for telecom products (Power Line Carriers and Teleprotections) and
currently as Product Manager in the Grid Automation product line in charge of cyber security and
network switches. When not at the office, you’ll find Jerome riding his mountain bike in the beautiful (and
rocky) back country of southern France.

Home | Current Issue | Tutorials | White papers | Books | Tools | Events | Advertising | Classified | Forum
Terms and Conditions of Use and Privacy Policy
© PAC World - Last updated: 14 Jul 2016

https://www.pacw.org/no-cache/issue/june_2016_issue/digital_substations/digital_substation_the_next_generation_smart_substation_for_the_po… 4/4