Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B
Sood Pructices V4,0 SP0Z Redundunt Ethernet Network Document version 2.0 Creation 21/01/2005 Last update 30/O8/2005 Author Yann BOUFFARD-VERCELLI Category Network Language English File name GP02_Ethernet_Wiring_en.doc
CONTENTS
1. RESUME........................................................................................................................................ 2 2. RECALL - REDUNDANT ETHERNET..................................................................................... 2 3. RULES............................................................................................................................................ 3 3.1. RULE 1 WIRE QUALITY ......................................................................................................... 3 3.2. RULE 2 IDENTIFICATION OF PRIMARY/SECONDARY OPTICAL FIBRE....................................... 3 3.3. RULE 3 ETHERNET DATA LIST. ............................................................................................. 3 3.4. RULE 4 CONFIGURING SWR, H15X AND H35X NUMBER ...................................................... 4 3.5. RULE 5 CONTROL NETWORK WIRING BY INCREMENTAL DEVICE ADDITION........................... 4 3.6. RULE 6 USE OF OPTICAL FIBRE.............................................................................................. 6 4. TESTING CUBICLES FOR REDUNDANT NETWORK AND REDUNDANT C264........... 6
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B 1. Resume
After a recall of Redundant Ethernet principles, rules are expressed in order to get an efficient network. Correction of wiring and addressing on set-up or site are regularly costly operations.
2. Recall - Redundant Ethernet Redundant Ethernet network is an Areva proprietary solution based on ring topology (while a standard Ethernet can be only looped through routers). The network is only composed of Areva switches and optical links. The network is most usually multimode, possibly monomode (kind of optical fibre, price and characteristics, optical emitter/receiver in the switch). The only devices are:
MiCOM H15x (PCI board) MiCOM H35x (stand alone device) SWR Board (Slot C into MiCOM C264)
Switches have additional 4 to 6 standard Ethernet Port, non redundant, copper (RJ45 Class 5SE cable) that allow to connect other Ethernet device up to 100 meter length. The optical link is composed of 2 optical fibres, with connectors BFOC (called also ST), defined as 62,5/125 microns for multimode (max. 2000 meters), or as 9/125 microns for monomode (max. 15000 meters). Any optical fibre is connected for an optical Emitter in one device to an optical Receiver on second device.
The proprietary redundant Ethernet looks globally like a ring. It has a Primary net ring in one direction, and a Secondary net ring in reverse direction (fig.1). To simplify, data are transmitted on the main Primary loop in one direction. When optical link is broken, the 2 neighbouring switches are making a self-healing of the net, making only one loop using the secondary back-up net (fig.2).
MiCOMH35 MiCOMH15 MiCOMH35 MiCOMH15
It is not possible by CORTEC to mix monomode and multimode switches.
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B Each redundant switch has 2 output contacts that indicate self-healing (or network broken one side). It is necessary that the two DO are wired on C264 DI (Data List) to indicate or to alarm network failure and that redundancy operating. Else it is never corrected and fails definitively at second failure.
3. Rules Rule 1 Wire quality: Rule 2 Identification of primary/secondary optical fibre Rule 3 Ethernet Data List Rule 4 Configuring SWR, H15x and H35x number Rule 5 Control net wiring by incremental device addition: Rule 6 Use of optical fibre
3.1. Rule 1 Wire quality Before installation and commissioning optical link should be tested (neon light if not connected, visual inspection, dB attenuation measurement in site). Any suspicious cable should be refused or destroyed.
3.2. Rule 2 Identification of primary/secondary optical fibre. The link is composed of 2 fibres for Primary and Secondary. At both end the primary fibre should be clearly identified. Waiting the definition of standard cable, the default is to put a jumper around primary at both ends.
REM: Some cables have separate colours for each fibre.
3.3. Rule 3 Ethernet Data List. 3 attributes and the number of appearance on the physical net, define Ethernet device. The list of devices and their parameters has to be displayed may be pin-up on set-up or site, and shall contain: Mac Address (C264) IP Address Network Alias Name Net Number, as order of appearance. Each device should have a sticker with its parameters (good practice). A device is connected on the net only after control of its parameters.
REM 1: C264 Mac address is written on delivery form, sticked during manufacturing on CPU260 board, and Front panel, displayed on LCD service panel, CMT and SMT (in hex in this last case). REM 2: The Ethernet network fails with 2 devices with same Mac address. This Mac Address can be modified using CMT, possibly by error loading new software version. REM 3: A default list has been defined with alias name and IP for standardisation.
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B 3.4. Rule 4 Configuring SWR, H15x and H35x number At production line, all switch devices get same address 0 (called order number). This address is configured in binary code removing jumpers (SWR) or by macro-switches. This order number should be used only once on the net. The rule is to configure the order number to be equal to the end of C264s IP address where SWR is installed, or of the neighbouring PC (H35x) in the site.
C264 IP 10.22.91.31 => the order number of SWR is 31 Remove the 5 first jumpers starting from the top.
REM 1: The maximum order number is 127 (below to the max IP in the standard data list). REM 2: The decimal value of each jumper is written on the board under the jumper. The order number is then the sum of numbers written where jumpers have been removed.
3.5. Rule 5 Control network wiring by incremental device addition Pre-requisite Each device can be power on/off. All addressing should have been done before (jumpers or switch for SWR/H35x, Mac Address, IP Address, Name alias). Such operation on C264 can be done from front port serial link with Teraterm (or HyperTerminal) on a laptop.
Method Connect one by one the device forcing 2 self-healing.
1) Start from H15x, H35x linked to first PC disconnecting fibre Rp (Es). 2) Go to next device following Primary loop. Connect from the first device connectors Ep-(Rs) to the second device connectors Rp-(Es). 3) On second device force the self-healing toward third device: Can disconnect temporary the EP (Rs) fibre on second device Or can stop the third device by power off. 4) Check from first PC with ping IP or CMT that the new device is seen on the net, and stable". The goal is to verify that the new optical link and new device parameters are correct. When it is not operating, check the 4 Ethernet parameters, then optical link (and replace it). Stable: make 3 or 5 ping without failure on the new device IP, or with CMT that new computer can be reached and that the list of computers is constant. If new device is seen and connection is not stable, the optical link is crossed or there is two times the same Mac Address. Restart from step 2, connecting device N+2
5) Last optical to the starting switch/PC: Before puting back the Rp(Es) connection on the first device (the one removed in step 1), disconnect its link Ep(Rs) and check connection stability to the whole network. Then, reconnect the Ep(Rs).
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B Rp Pri mar y fi ber Secondar y fi ber Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Rp Es Rs Ep Ex : SWR f r om Rear
Method rear viewing from C264 , connections between SWR
REM: Method detects also ethernet parameters error (Mac @).
Do not wire globally the network when not understand the process. Usually global wiring leads to problem, because at power on, when net does not work, only crossing a fibre make it operate. Somewhere there is no respect of link Ep to Rs (resp. Es to Rp). There is a link Ep to Rp on another device. The network is crossed 2 times. In case of link breaks, neighbouring switch go in self-healing and sub loops appear on the net. The network becomes instable. Another trouble of global wiring is that at stat-up one fibre is broken and the network already in self- healing, then at first reboot test the network (no more redundant) fails.
EP RS RP ES Primary fiber Secondary fiber A B C D E 1 2 3 5 6 7 9 10 11 8 4 switch switch switch switch switch
Healthy net (each switch in cubicle with copper link to 1-3 C264)
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B EP RS RP ES Primary fiber Secondary fiber A B C D E 1 2 3 5 6 7 9 10 11 8 4 switch switch switch switch switch
Healthy net when redundant applies with self-healing : 3.6. Rule 6 Use of optical fibre The centre of glass fibre is few microns, and glass becomes elastic. But elasticity is limited an we recommend a minimum radius of 15cm. Below micro break appears, loss in dB increase and may reach limit of sensibility of the receiver (intermittent communication can be observed). Always care on fibre, if doubtful remove it.
4. Testing cubicles for redundant network and redundant C264
Before platform installation, it is important to provide to logistics department the redundant network mapping with available equipments. The objective is to have a reduce wiring network mismatches and also to avoid fragile optical fibre links between cubicles
It is also advised to mount the C264 in testing cubicles regarding their network rank.
In case of redundant C264 : Generally, on site, main and backup C264 are powered by different auxiliary supplies. All main C264 are grouped on one part of the redundant network and backup on another one. In case of loss of one of the 2 supplies, the system keeps on being operational. See installation recommendation as follow :
AREVA T&D Protection & Contrle Avenue de Figuires Font de la Banquire BP75 34975 Lattes Cedex (France) Tl. : +33 (0) 4 67 20 54 54 - Fax : +33 (0) 4 67 20 54 99 Socit Anonyme au capital de 16 000 000 -343 074 142 RCS Montpellier - Siret 343 074 142 00070 - Code APE 312B Rp Wall Wiring To OI Wall Wiring From OI CUBICLE 1
Primary fiber Secondary fiber COMP02_M
COMP01_M COMP03_M
COMP04_M
COMP05_M
CUBICLE 2
COMP02_R
COMP01_R COMP03_R
COMP04_R
COMP05_R
TESTING CUB.
SWITCHBOX
SWITCHBOX
IED
IED
Wall Wiring To Next MAIN
Wall Wiring To Next BACKUP
Main/Back-up Wiring
Use long copper wires between Main and Backup C264 (BIU redundancy wiring, IED wiring, ). This mounting allows the use of (in any) a wall installed optical network.