Faul t fi ndi nq &Troubl e shooti ng i n maxDNA control svstem

The visual aids and performance of the control system are the basis for fault
location and troubleshooting.
The visual aids are the following :-
l.Status LEDs on the power supply convertor modules.
2. Status LEDs on the l/O modules.
3. Status LEDs on the DPU.
4. Status LEDs on the network switches.
5.System diagnostic pictures on the maxStation.
The information given by the above sources is explained below.
The performance or behaviour of the control system is also a source for
-
predicting the nature of fault.
D'AGNOSING FROM STATUS NDICATORS
l.Status LED on power supply convertor modules:-
The power supply convertor modules will each have three status LEDs with
labels
' input'
,' fault'
and
' output' .
When the input power is connected to the
modul e then the correspondi ng LED i s on and green and
,si mi l arl y ,
when the
output i s bei ng generated then the' output' LED wi l l be on and green.
The' Faul t' LED
,
i n the centre, wi l l l i ght up i n case of modul e fai l ure.
2. Status LEDs on l/O modules:
-
-
On the front plate of all l/O modules at the lowest part is a green LED with
label
' ACTIVE' .
This will be steady ON when the module is powered up and
the DPU is communicating with it.
-
l f the DPU i s not communi cati ng wi th i t, then the LED wi l l be bl i nki ng
ON/OFF at a steady frequency. The reasons for this can be.
a. The DPU configuration is blank.
b. The DPU outputs are frozen or
c. The address set on the module is not matching that
mai ntai ned i n the DPU confi gurati on.
-
On certain l/O modules there is another green LED on the front with label
' FUSE' .
Thi s i s l ocated
j ust
above the
' ACTIVE'
LED.
\-t
v
The
' FUSE'
LED wi l l be steadi l y on when the common fuse of the modul e
is healthy, and the field supply (usually 24 V DC) is available through the
back plane. Otherwise the LED will be off (dark). In such a case, check the
fuse healthiness and ensure that the field supply is reaching the back
pl ane at connector P14.
Refer figure below for a view of the front
plate of a module'
3. Status LEDs on DPU modul es:
-
Overuiew
At various times the 9 status LEDs on the DPU front panel flash green, yellow,
red or display no color to indicate the associated hardware' s functional status.
Discussions in the following sections explain what the color changes may
indicate under different DPU states.
The status LEDs are divided into the following sections:
Network Status LEDs
(Located
at the corner of each of the network ports)
maxNET Network A health (The same applies for Network B LEDF
l
\rlt
E I
a z
E I T
t
E t t
e t c
E '
E C
r*r
c ! *
et rc
g l r
E r e
et tl
F t a
s t l
E
r a
v
Green
Yellow
-
steady
Yellow
-
bl i nki nq
off
The network
connection is
operational.
On power-up, the
network
"A"
and
"8"
LEDs start off blinking
yellow until the DPU
hears communication
from another station in
i ts domai n. The LED
goes solid
yellow and
then to green.
Some Network Error.
Each devi ce (DPUs and
Workstations) send messages
that contain a list of all stations
from which it is currentlY
receiving messages. lf the DPU
receives messages from a
station that does not rePort
receiving messages from it, the
DPU i l l umi nates the Yel l ow
LED. The problem may or may
not be thi s DPU.
DPU
does not
hear
from any
station
DPU not
Operational
or LED Bad
Backup LED (Located at the corner of the backup cable
port)
Green
Yellow
Red
off
Backup Network
Operational
Some Backup
network Failures
Communications
not
established to
backup DPU
DPU not
operational
or LED Bad
IOM LED (Located at the lower part of the DPU front
plate)
Green
Yellow
Red
off
Operational IOM N/A
IOM Fai l ed /
Timedout
Reset Required
IOM not
initialized
l
Vr
rther fault investigation of l/O
t / O LED (Located at the lower part of the DPU front
plate)
Green
Yel l ow
Red
off
All configured l/O
cards are working
correctly.
Some configured l/O
cards are working
correctly, and atleast
one is not.
No configured l/O
cards are working
correctly
No l/O cards
configured in the
database or IOM
failure OR l/O cards
are either taken off
scan or
placed into
si mul at e mode.
modul es can be done. Ref er 5. 1. 1 & 5. 1. 8 i n a l at er part of t hi s document '
CP LED (Located at the lower part of the DPU front plate)
Green Yellow Red
off
Bl i nk Yel l ow / Green
-
Healthy Heartbeat
Solid Green- CP
failure.
Blink Yellow / Green
-
Healthy Heartbeat;
Solid Yellow (<20sec)
-
Loading, or saving the
dat abase;
Solid Yellow (>20sec)
-
CP failure.
Card Failure Gard Failure
a
STATE LED (Located at the lower part of the DPU front plate)
4. Status LEDs on the network switches :-
The network switch (usually with 24 ports) has LEDs on its front for indicating
status of its operations.
i) Power supply availability : LED labelled as SYST. This must be steady
green. lf switch is faulty, then a few seconds after power up this LED does
not become green and instead becomes amber.
ii) By default the LED labelled as STAT will be lighted green.
iii) For each port
,
the port status LED above it will behave as per table below
provided the switch is in STAT mode
\tt
DPU States Active DPU State/LED lnactive DPU State/LED
Standalone DPUs Green Yellow
Backup Warming Blinking Green/Red Blinking Yellow/Red
Hot Backup Blinking Green/Yellow Blinking Yellow/Off
Backup Enabled / No Backup
Available
Blinking Green/Off Red N/A
Booting Red Red
Offline Off then Blinking Red Off then Blinking Red
Port status LED
colour
Meaning
OFF No l i nk
SOLI D GREEN Link
present
FLASHING GREEN Port is sendinq or receivinq data
GREEN/AMBER Link faults
present
like error frames, CRC errors, etc.
SOLI D AMBER Port not forwarding due to being disabled, address
violation. etc.
D'AGNOSING FROM OPERATOR DISPLAYS ON MAXSTATION
o
5. System diagnostic
pictures on the maxStation :-
From a single maxStation (be it Operator Work Station or Engineering
Work Station) theltate of each DPU in various control
panels and the state of
the network connections of the full system can be observed.
Viewing the DPU4F Status disPlaY
Through the SYSTEM STATUS main menu
,the
DPU4F STATUS
picture
can be accesled. In this display
,option
is given to click and navigate to a display
of the status of any DPU which needs to be observed in detail.
The DPU4F Status display shows a collection of objects called annunciators.
View this display to see a summary of the health of each of the maxDPtts in your
system. Each maxDPU is represented by an annunciator that contains color-
coded icons for:
CP
-
(maxDPU' s Control Processor) status,
BK
-
backup status,
l/O
-
l/O subsystem status,
A B
-
maxNET networks A and B status.
A colored outline is used to denote which maxDPU of the Pair is active. A
quick glance at the display can indicate if all is well with your DPUs or if attention
is reqrlired. To view additional information about DPU status, click on a specific
DPU annunciator to access a separate maxDPU Details display for the selected
DPU. See
"Viewing
the maxDPU Details Display," later in this chapter.
The maxDPU Pair annunciator
just
has two of each icon, the upper row
for the Primary DPU and the lower row for the Secondary DPU. To indicate the
active DPU, a cyan-colored border surrounds the row. lf the border is not visible,
neither DPU is active (or the display cannot communicate with the DPU)'Each
row or icon displays data as described in table below'
\t4
DPU Status
good
(Dark Green, if
DPU Fan and/or
Battery bad)
DPU Status
marginal
(Dark Yellow, if
DPU Fan and/ or
DPU Status bad
(Dark Red, i f DPU
Fan and/or
Battery bad)
SBP
communications
bad
SBP
communications
bad
Normal or Non-
Critical l/O card
bad
SBP
communications
bad
Network has Failed
( lf DARK-GREY:
Network is OFF)
(t
5.1.0 Viewing the DPU4F Details Displays
Click on a specific DPU annunciator,
apfiearing
!lt!:.
DPU4F Status display'
to
""""tt a DPU4F Details display for the selected
DPU'
The DPU4F Details Display simulates a front maxDPU
faceplate and
provides a list of DPU status detajls. When
you select an annunciator
object for a
specific
DPU
pair, animated
faceplates
"ppeat for
primary and secondary
DPUs'
iP
Refer drawing below for a view of the DPU4F
Details
picture as seen on
Understanding
the DPU4F Faceplate
| ' r l r f r nr : t f cr r r sr r r v
r ! ! v vr
- J
ar r angement
f Or an
TheDPU4Ff acepl at ef eat ur esLEDsi dent i ca| t ot heLt t ^ r L^ ^ ^ - ^ ai af ar l
;:l'#t"ti.
ii""il;
colors on a DPu faceplate
indicate
the associated
n - ^ L - - ^ ^ n r r a l l ,
f,l.,.:ffi;tr^t,'i";'
status. At various times the LEDs flash
green'
vellow'
red
r . . ^ ^ { i ^ nal ef af t t c
ft
61;;il;;-'i;inJicatethe-associated,hardwar::^l11tl:",:iL:t15;
il:''rEilJ"ili,".;;
,ho*n on the disptay shoutd
be identical
to the color states
o
on the actual DPU.
The significance
of the LED colours is already
described
in
point 3 on
pages 2 to
4 above.
Data on DPU System display
which is additional
to the
physical module
faceplate
5.1.1 l /O Detai l LEDs
ln addition to the l/o summary
LED on the DPU front
panel' the DPU4F
Detail
display also contains
four LEDs to indicate
card status organized
by typg of loM
operation.
This is ,irit"l. to the inOicators
on the DPU4E
display.
These LEDs are
located adjacent
to the faceplate'
The color
codes for the individual
LEDs are tne same as for the l/o summary
LED' listed
above(That
is otr,neqverrow
ano oreen and the significance
is the same
but
appficaOfe
to the specific
category
given below)'
The l/O oPeration
LEDs are:
Fast Digitil Inputs
(labeled Dl on.the display)
outputJ
(label' ed AO/DO
on the display)
lnputs
(labeled Al on the disPlaY)
Items
(iabeled ITEM on the display)
click the l/o status button
next to the maxDPU4F
faceplate
graphic to, access an
!
l/O buffers inu"nto-ry
rtatus displ"y.
i"";tlO
Status,"
later in this chapter'
The
information
on that dispray can exprain why an error condition
is being
indicated
by one of the l/O Status LEDs'
5.1.2 Time SYnc Information
Time Syncnronizaiion
iniormation
for the DPU is shown
in two fields near the
bottom
of the t"""pi"i".
in" top fielo oisplays-either
"Time
Slave"
or
"Time
Master,,
dependinJrpon
no* tire OiU is configured.
A Time Slave
gets the
current date and time from a Time Master'
The tower field displays
the lRlc-B;A1-tJ;
of the DPU' lRlG-B
is an international
standard
tor a tim-Jtransmission
tig-n"l.Jne
inro-a
signal is typically
supplied by
a high accuracy,our."
such as " 6ps receiver.
The maxDPU4F
may be
purchased witn an opltn to allowit io OecoOe
the IRIGB signal'
A DPU
(or DPU
pair)with the IRG-B decoder
will typically
be used as a Time Master
in the
;;;i;,
other DPUs will be configured
as Time Slaves'
On a Time Slave DPU, the lower OitpLV field will be.blank'
On a Time Master
DpU, the tower nJO *irf contain IRi6;6trs
information'
lf the lRlG signal is
bei ngdecodedpr oper l ybyt heopu, t nene| dwi | l : I 9* " l R| GoK" , l f t heDPUcan
not decode
the l Rl G si gnal , the fi el d' wi l t
show' l Rl G
Fai l ure"'
l f the DPU
i s a
\lt/
t
Time Master and if lRlG decoding
has not been enabled
(via a
ggttino
in ll]"
TimeSync atom), the field will show
"lRlG
is not enabled"'
Note that only the
active DPU of a opu
pair decodes
the lRlG signal.
Thus, in a
properly operating
DpU
pai r, the acti ve dpU *i tt i ndi cate' l Rl G
Ok" and the i nacti ve
wi l l i ndi cate
"l Rl G
Fai l ure".
5. 1. 3 DPU TemP
This field indicates
the temperature
(in degrees
Celsius)
measured
by a sensor
on the DPU.
5.1.4 DPU4F Statistics
For maxDpus
tne-ioirowing
information
appears
in the center of the Details
di spl ay.
The DPU name, |P address, current Time and Date, DPU serial number,
DPU
hardware revision and service count
(which corresponds
to the size of the
downloaded
configuration
).
Current revision ievels for CP, lOM, CPLD, IOM FPGA, and the Shared
Memory
FPGA. These fields correspond
to various hardware
and software
components
wi thi n the DPU.
current
DPU run mode and state, Backup mode, other DPU state, Active
Pulses state, Other DPU Active Pulses state'
Alarm severity information
for the specified
oe.!J
9r
DPU
pair' The alarm
fields
show total alarm counts for each opu tor severity
levels 1 (the lowest severity)
through
5 (the highest severitY)'
Fatal Error Reason indicates the reason that the cP has experienced
a fatal
error. When the DPU is running normally, the field will contain
"NA"
(not
"ppii""Of").
lf a fata1 error oc",ir", descriptive
text will appear
instead' when
possible.
soDlMM
Status displays information
about the soDlMM
memory
module
installed
in the DPU. if a' standard
memory
module. is-installed,
the field will show
,,Standard,,.
lf an error-chect<ing
soDltvttvt' 1e.g.,t81559)
is installed
and operating
properly, the field will show' Ok' .
lf an error-Jhect<ing
module is installed
and if it
has detecteO a memory "r1"o|', the field will show a message
such as
.coRRyNONCOci
Mem Erro/' . tt tnat happens, the Fatal Error Reason
field will
"h"ng" from
"NA"
to
"Memory
Error Detected"'
The Time since Boot field shows how much time has elapsed
since the DPU
was last started. Time since change shows the elapsed
time since the DPU
underwent
a major state change
1elg.,
if an Active DPU became
lnactive)'
5.1.5 Functi on Bl ock Ti me
This field displays the
percentage of available
time
(0-100%) that is being
consumedbyf unct i onb| ockswi t ni nt heDPU. Thef unct i onbl ocksar edef i nedi n
the DPU' s database
and are downloaded
to the DPU' Function
Block Time is a
measure of DPU loading. lt gives an indication of how much logic may be run in
the DpU. Function
gocfrir;e
is displayed
by means of a bar
graph and a
numeric display.
The color of the Uai i= gr"en for light loads and will turn
yellow
and then red as the loading approaches
100%'
s