The use of an IP subnet for Level 1 nodes outside the subnet range of the Experion servers is not

recommended
as a best practice. This scheme is recommended for installations where Level 1 address reuse is
required. For
more information, refer to Section 8.3, “Reusing IP Addresses for Level 1.”
3.2.4 Series A Level 1 LAN cluster
The following diagram illustrates a Series A Level 1 LAN cluster. The main purpose of this cluster is to
allow
critical peer-to-peer traffic to flow only locally. With Experion R430, ENIM/EHPM is included in the Level 1
LAN cluster. Refer to the Integrated Experion-TPS User's Guide for further information.
Figure 6: Series A Level 1 LAN Cluster
Citizenship
• Controller (C200)
• Fieldbus Interface Module
• Cisco switches
• ENIM/EHPM
• Safety Manager (SM)
Level 1 Switches
• Provide point-to-point connectivity for FTE devices in the cabinet
• High reliability configuration
– Always redundant
– Configure CDA traffic as the highest priority switch queue
– Configure view traffic as the second highest priority queue
– Configure other traffic as low priority switch queue
3.2.5 Connecting Level 1 LAN clusters
The following diagram shows several Level 1 LAN clusters connected with a second layer of switches.
3 LEVEL 1 NODES
24 www.honeywell.com
With Experion R430, ENIM/EHPM is included in the Level 1 LAN cluster. Refer to the Integrated
Experion-
TPS User's Guide for further information.
Figure 7: Connecting Level 1 LAN Clusters
Citizenship
• L2 configured FTE switches
• L1 configured switches
• Level 1 LAN clusters
FTE Switches
• Connect Level 1 clusters
• High reliability configuration
– Configured bandwidth limits for broadcast, multicast storm suppression
– Ability to disable interfaces with high traffic conditions
– Automatic port enabling when traffic profile returns to normal
• Dual FTE switch faults impact inter-cabinet traffic only
3.2.6 Connecting Level 1 nodes that intercommunicate
The best practice is to connect Level 1 nodes that intercommunicate to the same switch pair, so that they
have
the shortest communication path. If this is not possible due to size or geographic dispersion, their
communications go through the Level 2 switches. The Level 2 switches must be configured with the same
quality of service approach as those used for Level 1 switches.
• TCP ports are given the prioritization scheme described for Level 1.
• The control traffic entering from a Level 1 switch is tagged with the highest priority at the ingress.
• The output queue to the destination Level 1 node sends the control traffic before any other traffic.
Communications redundancy is provided for this peer-to-peer traffic by always having two “pipes” from
peerto-
peer and using FTE to provide four possible paths. Additionally, Level 2 switches are configured to have
storm protection on the interfaces where Windows operating system nodes reside. This storm protection
prevents broadcast or multicast storms caused by a node that is infected and using a denial-of-service
attack. If a
node reaches a limit of 20% of the connection bandwidth being used for broadcast or multicast, the
interface is
cut off until the traffic level falls below 18%. Normal FTE traffic for broadcast and multicast is below 2% for
each. Recent switch configuration files for the latest switch types use explicit bandwidth limiting (defined
as
Mbps) rather than percentage based limiting. Refer to FTE Technical and Specification Data for more
information on types of switches supported.
3 LEVEL 1 NODES
25
3.2.7 Using a switch for level 1 and level 2 (split switch configuration)
It is possible to divide a single switch into a level 1 and a level 2 section. The sections are interconnected
by a
cable between a port on each so the switch actually has 22 ports instead of 24 ports. The switch still
counts as
one level in the network hierarchy. The split configuration reduces the number of switches needed to
implement
best practices for connecting a few Level 1 and Level 2 devices. If you must put the Level 2 Console
station
directly on the Level 1 switch, the best practice is to use the split switch configuration files. These files
provide
improved isolation between Level 1 and Level 2. Refer to section 9.6, “Honeywell’s Switch Configuration
Files” for information of switch configuration options.
3 LEVEL 1 NODES
26 www.honeywell.com

4 Level 2 nodes
Related topics
“About level 2 nodes” on page 28
“Level 2 Best Practices” on page 29
“Implementing level 2 best practices” on page 31
“Safety Controller Best Practices” on page 33
27

4.1 About level 2 nodes

Level 2 nodes are primary server, view and advanced control nodes for the process control system.
These nodes
are essential for operation of the process, but not as critical to control as the Level 1 nodes. For example,
servers, stations, ACE nodes, and PHD nodes.
4.1.1 Level 2 LAN
The following diagram illustrates an example of Level 2 LAN.
Figure 8: Level 2 LAN
Citizenship
• Experion server
• Experion console
• Application node
• Subsystem interfaces
• Domain controller
• FTE switches
• Experion App node
Level 2 FTE switches
• Point-to-point connectivity for Level 2 devices
• Pre-configured bandwidth limits for broadcast, multicast storm suppression
– Ability to disable interfaces with high traffic conditions
– Automatic port enabling when traffic profile returns to normal
• Configured CDA traffic in high priority switch queue (ACE-ACE, ACE-Controller)
• Configured non-CDA traffic in low priority switch queue
4 LEVEL 2 NODES
28 www.honeywell.com

4.2 Level 2 Best Practices

The nodes residing on Level 2 are vulnerable to attacks by virus or software glitches because of the open
nature
of the operating system and the customized software running on these nodes. Hence, the FTE switches
in Level
2 are configured to provide the security and reliability as described in “Connecting Level 1 nodes that
intercommunicate” on page 25.
4.2.1 Configuring level 2 switch
The following are configured in the FTE switches.
• Protection from broadcast and multicast storms on all interfaces to these open nodes.
• The display traffic, like the control traffic, has a higher priority so that the view to the process traffic takes
precedence over other traffic on the switch. This is important if there is a “bad actor” on the LAN that
generates high traffic. The traffic with higher priority control and view arrives first.
• BPDUguard is configured on non-uplink interfaces to prevent loops and unexpected uplink placement.
4.2.2 Avoiding multiple network connections
Avoid connecting PC nodes to multiple networks. For example, connecting a server to two networks turns
the
PC node into a router, which is not allowed. Instead, the Experion network structure provides the use of
routers
to combine Level 2 nodes to Level 3 nodes or to other Level 2 nodes. A built-for-purpose router must be
used to
provide security and reliability through the use of access list filtering. There are exceptions when a third
NIC
interface is used for private connection to a single Ethernet device. An example is the Honeywell DHEB
for
bridging to the Data Hiway.
4.2.3 Non FTE dual attached nodes within level 2
Non-FTE dual attached nodes connect to Level 2 switches and are compatible with FTE. Although these
nodes
communicate with FTE nodes, they do not have the same level of network availability as FTE. Examples
of
these node types are as follows:
• Terminal servers
• OPC servers
• PLCs
4.2.4 Non FTE single attached nodes within level 2
Non-FTE single attached nodes, such as terminal servers or subsystem devices connect to Level 2
switches. For
a large number of single attached nodes, use a separate switch to aggregate these nodes.
Following are guidelines for using a switch for this purpose.
• The switch is counted as a level for spanning tree. Hence, it must not be connected to an FTE switch at
the
third level.
• The switch must not be connected to any Level 1 switches.
• To avoid loss of data, nodes that have a single connection are divided into two switches, where some of
the
nodes are connected to the green switch and the others are connected to the yellow switch.
4.2.5 Nodes with embedded operating systems
Nodes with embedded operating systems do not have the processing power to handle multicast and
broadcast
traffic volume generated by FTE test messages and Address Resolution Protocol (ARP) packets. Connect
these
4 LEVEL 2 NODES
29
nodes to a Level 3 switch, or protect it with “access list filtering” on a separate Level 2 switch. Honeywell
recommends the use of a qualified Experion switch for this