Controller redundancy

The DeltaV system supports redundant controllers. A redundant controller consists of a pair of standard controllers on separate 2-wide power/controller carriers connected together. Each controller requires a separate power supply mounted on its carrier. One of the controllers in the pair is the active controller. The other controller is the standby controller. The standby controller contains the same configuration as the active controller. When an active controller fails, the standby controller takes over providing uninterrupted control operation without user intervention. The standby controller gets updates of certain parameter values in the active controller over the redundancy link, but does not execute control logic. When the switchover occurs, the new active controller reads back I/O data and the modules begin to execute. A limited amount of re-initialization occurs in order to resume control without disruption. For example, certain control and output function blocks begin executing in out-of-service mode and climb to their target mode in order to force handshaking with other blocks. In most cases control actions resume on the first scan after a switchover. For complex modules, one or a few scans of handshaking may be required. The apparent mode change on a switchover is expected and has no adverse effect on control. 0 comments Even though each controller in a redundant pair does have its own MAC address and node address, a redundant controller counts as a single node on the DeltaV control network in terms of network capacity. 0 comments A redundant controller requires a redundant controller license. Downloading a redundant controller without a license results in a description of "Redundancy Not Licensed" in Process History View. In addition, the RedEnb diagnostics parameter has a value of NO if no license is present. 0 comments Redundancy typically requires more controller CPU than the same configuration in a simplex controller. Additionally, larger configurations (more modules, more parameters) require more CPU time for redundancy processing than a smaller configuration. A majority of the redundancy processing time is allocated against the 35% of total controller CPU that is reserved for overhead processes. A minor amount of the redundancy processing will be allocated to the 65% of time reserved for user-configured control processing and decreases the FreTim parameter by several percent. 0 comments It is important to note that the FreTim parameter is the percentage of the available processor time for execution of the user configuration. Available processor time is approximately 65% of the total CPU that is reserved for user configuration tasks and the remaining 35% of controller processor time is reserved for other controller functions. The controller's idle time measures the entire CPU time available and is shown in the DeltaV Diagnostics' Time Utilization Chart. 0 comments It is possible that certain extraordinary configurations which perform properly with a simplex controller may not do so when a redundant controller is added. For example, additional CPU load from redundancy processing could cause the overhead processes to consume more than the allocated 35% of processor time. Additional load could be due to a higher than typical number of modules or parameters or other atypical usage (such as communications loading exceeding the recommended limits). When this occurs, overhead processing can consume some of the 65% of the CPU allocated the user configuration. If adequate FreTim is available, this is not a problem. However, when FreTim is very low, the overhead system processes and the user configuration tasks compete for the CPU resources. In most cases this results in module slippage. 0 comments You can use the DeltaV Load Estimator tool to assist in determining controller load by measuring the controllers' Free CPU in % (FreTim). However, the DeltaV Load Estimator might not adequately account for redundant CPU loading on systems with more than 500 modules if the redundancy creates additional overhead loading exceeding the 35% reserved for system functions. Monitoring the FreTim parameter and the Time Utilization Chart is also important. The DeltaV Load Estimator is provided on the DeltaV installation disk #2 in the _Support\Tools\LoadEstimator folder. 0 comments Switchovers 0 comments A switchover from the active to the standby controller can occur for the following reasons: 0 comments Hardware failure within the active controller 0 comments

Communications failure between the active controller and the I/O subsystem 0 comments Communications failure of both the primary and secondary network connections in the active controller 0 comments Removal of the active controller from the carrier 0 comments Manual switchover request 0 comments Loss of power supply for the active controller 0 comments Memory failure (the active controller detects a RAM or ROM failure) 0 comments Software watchdog timer trip 0 comments When a switchover occurs, the node status area on the Alarm Banner indicates the status change to the operator. The system regenerates active, unacknowledged and suppressed alarms. The system does not regenerate alarms that are both inactive and acknowledged. Alarm states are maintained during a switchover but General I/O Failure (IOF) Alarms may change state momentarily. 0 comments The system stores a record of each switchover and the reason it occurred (if known). The switchover is logged in the Event Chronicle. After a switchover, the controller attempts to communicate with all of its I/O. The Event Chronicle description: Switchover I/O Readback Timeout means that after a redundancy switchover, the controller was unable to communicate with its I/O before the timeout period. 0 comments If the standby controller should fail, the software disables switchovers until you replace the failed unit. 0 comments Generally, there is no user impact related to a controller switchover. However, there are certain situations that the user should be aware of when configuring control. These relate to unit modules and phases, and HART digital variables, external references to other controllers, and forced debug parameter values. 0 comments Unit module and phase runtime data are not transferred to the standby controller before PAVAIL=YES and therefore one should wait at least 1 minute beyond PAVAIL=YES before doing a switchover so that the standby controller is ready to take control. 0 comments HART digital variables will have a value of zero with Bad status for six or more seconds after a controller switchover. If you access a HART digital variable in an expression (Calc, Condition, or Action function block; SFC transition or action), be sure to also consider the status of the variable in the expression. For example, use the value only when the status is Good. Accessing HART digital variables in AI function blocks or by external reference parameters should be done for monitoring, not control purposes. 0 comments When a controller switches over it can take several seconds to re-establish communication with other controllers. External reference parameters that reference parameters in another controller will have BadNoCommNUV status during this brief period following a switchover. Understand how downstream function blocks and expressions will react to this bad status on a switchover so that your configuration will not create an output disruption. Refer to the 'status handling' subtopic of specific function blocks for this information. 0 comments A Condition function block that references a parameter in another controller has the potential to change its OUT_D parameter when the referenced controller switches over. You can prevent this by selecting the Abort on Read Errors option in ALGO_OPTS. This is recommended unless you want OUT_D.ST to be BadNoCommNUV during the switchover. In this case leave the Abort on Read Errors option deselected and change ERROR_OPT to Use Last. 0 comments Forced debug parameter values (that is, those set in Control Studio's debug mode) are not retained following a controller switchover. The value of the parameter is set to the default value as configured. 0 comments

In general, the system regenerates process and device alarms during a controller switchover. This includes all active, unacknowledged and suppressed alarms. Inactive and acknowledged alarms are not regenerated. General I/O Failure (IOF) alarms may change state momentarily. The Time In values (displayed in the alarm list) and the alarm states are not affected by the regeneration. The Time Last values on the alarm list are updated. Alarm strings may not be maintained. 0 comments Manual Switchovers 0 comments Users with the Diagnostic Key can initiate a switchover from DeltaV Diagnostics. In Diagnostics, a redundant controller has a Redundancy subsystem. To perform a manual switchover, right-click the Redundancy subsystem, and then click Redundancy switchover. 0 comments Controllers have three parameters that determine whether a manual switchover can occur: 0 comments PExist -- indicates whether the standby controller is physically present 0 comments PAvail -- indicates whether the standby controller has received a download and is ready to take over 0 comments RedEnb -- indicates whether redundancy has been enabled for the pair. In order for redundancy to be enabled, the active controller must have received a download. 0 comments In order for a manual switchover to occur, all three of the above parameters must have a value of Yes. If not, the Status parameter provides additional details about why the controller cannot switch over. 0 comments Creating a Controller Placeholder 0 comments To create a placeholder for a redundant controller in DeltaV Explorer, add a controller node, and then click the Redundant Controller option on the node's Properties dialog. 0 comments Installing a Redundant Controller 0 comments Installing a redundant controller is as simple as installing a single controller. The auto-sense feature in the DeltaV system recognizes two controllers on adjacent, connected carriers as a redundant controller. 0 comments Refer to Installing your DeltaV Digital Automation System for physical installation instructions. 0 comments After installation, drag the controller from the Decommissioned Controllers section to the control network or to a redundant placeholder. Assign an appropriate redundant license to the controller and download the controllers to enable redundancy. 0 comments Installing a Standby Controller 0 comments You can connect a second controller to an existing controller's carrier to introduce redundant control without interrupting your process. The system automatically commissions a standby controller when you install it. It is not necessary to remove or decommission the active controller. The active controller continues to operate without interruption. How the standby controller gets its configuration depends on the controller type: 0 comments MQ and SQ controllers The standby controller receives its configuration from the active partner over the redundancy link. 0 comments MD, MD Plus, MX, SD Plus, and SX controllers The system automatically assigns the standby with an address and downloads the standby controller with the latest download and with any online changes made to the active controller. 0 comments To install a standby controller follow these steps: 0 comments

Follow the steps in the order shown. Do not install a second 2-wide carrier that already has a power supply and controller installed. Doing so will result in a loss of configuration data for the active controller. 0 comments Using the DeltaV Explorer, assign an appropriate redundant controller license to the controller node that you want to make redundant in DeltaV Explorer. 0 comments Install a second 2-wide carrier to the left of the current 2-wide carrier. 0 comments Insert the appropriate Power Supply in the left slot of the 2-wide carrier plug in the power cord. 0 comments Connect a controller to the carrier. The version of the controller you connect must match the existing controller's version. 0 comments The DeltaV Explorer displays a redundant controller icon in place of the simplex icon. 0 comments If you are changing a simplex controller to a redundant pair, there is a one-time step required. In the DeltaV Explorer, right-click the controller, click Download, and then click Setup Data. This turns on redundancy for the pair but does not disrupt any existing process control. 0 comments Replacing a Failed Controller 0 comments When a switchover occurs, remove the failed controller and plug in a replacement. The system automatically commissions the replacement and makes it the standby controller. 0 comments Removing Controllers 0 comments The commissioning and decommissioning function in the DeltaV Explorer affects both controllers in the pair. You can remove one controller in a redundant pair without decommissioning it. For example, if you remove the standby controller and reinstall it, the address is reestablished, the standby receives a current download and is ready to accept a switchover. If you remove the active controller without decommissioning, the standby takes over control. When you reinstall the controller, the address is reestablished and the reinstalled controller becomes the standby controller. Anytime one controller is removed from a pair, the DeltaV Explorer continues to identify the controller as redundant. If you want to change a redundant controller to simplex, you must change the properties of the controller in DeltaV Explorer and download the controller's setup data. 0 comments Do not disconnect a carrier and its controller while the controller is powered and operating. If a controller and carrier must be removed, power down the controller before disconnecting the carrier and its controller. 0 comments Before removing both the controllers of a redundant pair, decommission the node through the DeltaV Explorer. 0 comments Diagnostics 0 comments The DeltaV Diagnostics displays a redundant controller icon in place of the simplex icon for redundant controllers and placeholders. The DeltaV Diagnostics enables you to view diagnostics, MAC addresses, and node (IP) addresses for both controllers in a pair. Diagnostics also enables you to flash the lights on either controller in the pair independently using the Identify Controller menu selection. All diagnostic and address information for the standby controller is accessed through the active controller at the Standby level (below the Redundancy Subsystem). Diagnostics requires a commissioned communicating controller to be able to flash the lights (unlike Explorer, which can use the database MAC address when the controller is decommissioned). However, you can Telnet or perform a TCP/IP PING directly to the primary or secondary network connection node addresses on the standby. 0 comments Upgrading Controller Firmware 0 comments If the redundant controllers require a firmware upgrade (such as for a DeltaV version upgrade), the controllers contain flash ROM that allows you to update the firmware while your process is running. Simply upgrade the standby

controller, then perform a manual switchover from Diagnostics. You can then upgrade controller that you recently switched to standby at your convenience. There is no downtime. 0 comments