New features in Superdome 2 partition management

Technical white paper

Table of contents Introduction ..................................................................................................................................3 What are nPars and vPars? ...........................................................................................................3 Superdome 2 partition management overview .................................................................................3 nPar new features in Superdome 2 .................................................................................................5 Finer socket granularity ..............................................................................................................5 Integrated nPar/vPar management ..............................................................................................5 Partition specifications or parspecs..............................................................................................5 OS-agnostic resource paths ........................................................................................................6 Improved partition run-states .......................................................................................................7 OS specific default values for nPar attributes ................................................................................7 Improved h/w diagnostic display................................................................................................7 vMedia/DVD support ................................................................................................................8 Memory granularity is now an nPar attribute ................................................................................8 nPars differences in Superdome 2 from legacy implementation ..........................................................8 Resource specifications use new OS-agnostic resource paths ..........................................................8 Transition from chassis/cell (or global cell) usage to enclosure/blades ............................................8 Assigning I/O to nPars ..............................................................................................................9 Finding the resource path of I/O ................................................................................................9 OA CLI for partition operations ...................................................................................................9 Force and override option for OA CLI .........................................................................................9 Partition configuration from the Superdome 2 OA GUI ................................................................10 Root permissions needed for status display commands ................................................................10 Remote complex management through partition commands no longer used ...................................10 Socket local memory instead of cell local memory .......................................................................10 Blade online addition, deletion, or replacement (Blade OL*) no longer used ..................................10 VFP no longer used .................................................................................................................10 Obsolete nPar commands (fruled, frupower, cplxmodify, parunlock) .............................................10 Parconfig command for configuring LORA partitions no longer used .............................................10 HT is enabled for HP-UX by default ...........................................................................................10 No interleave blade failure usage policy is not supported .........................................................11 Core cell no longer used .......................................................................................................11 Obsolete partition command IP address .....................................................................................11 Unique partition names ............................................................................................................11 Change in reboot for reconfiguration method .............................................................................11 Genesis partition no longer used...............................................................................................11 Complex Profile no longer used ................................................................................................11 ILM/SLM ratio ........................................................................................................................11 vPars new features in Superdome 2 vPars ......................................................................................12 Integrated nPar/vPar management ............................................................................................12 Max 16 vPars per nPar ............................................................................................................12 Maximum of 4 vPars/blade as opposed to 8 vPars/cell ..............................................................12 Separate console for vPars .......................................................................................................12 vMedia/DVD support ..............................................................................................................12 vPar management through the Superdome 2 OA ........................................................................12

vPars differences in Superdome 2 from legacy implementation .........................................................13 No vPars monitor or vPar monitor commands .............................................................................13 Booting all vPars simultaneously ................................................................................................13 OA commands for vPar operations ...........................................................................................13 vPar commands from OA need to specify the nPar context ...........................................................13 OS-agnostic resource paths ......................................................................................................13 Transition from chassis/cell usage to enclosure/blades................................................................14 Transition from SBA/LBA to iohub/rootcomplex/rootport as well as ioslot usage ............................14 vPar configuration files (vPar database) reside on the Superdome 2 OA (not on HP-UX) ..................14 vPar boot follow nPar boot conventions .....................................................................................14 vPar install is now the same as how nPars are installed ...............................................................14 Memory granularity no longer a vPar attribute ............................................................................15 Change in default memory granularity .......................................................................................15 Change in minimum memory granularity....................................................................................15 vPar permissions management from the OA ...............................................................................15 Socket local resources instead of cell local resources ...................................................................15 vPar EFI shell ..........................................................................................................................15 Memory range support no longer used ......................................................................................15 vPars created with zero CPUs by default ....................................................................................15 Addition/deletion of CPU by path increases CPU count ...............................................................15 PIM data retrieval ...................................................................................................................16 OS version display in vparstatus ...............................................................................................16 Changes in mode switch functionality ........................................................................................16 Obsolete commands................................................................................................................16 Memory distribution among vPars .............................................................................................16 Partition state and run-states .....................................................................................................16 Change in reboot for reconfiguration method .............................................................................16 Support only for HP-UX versions that run on Superdome 2 nPars ...................................................17 Default values for SLM/ILM ratio following a mode switch ...........................................................17 Parspec change policy.............................................................................................................17 Both cpu and core keywords supported with the -a option ...................................................17 vPar numbers..........................................................................................................................17 Interactive commands no longer used ........................................................................................17 New Features in the Dynamic Cores release of Superdome 2 firmware .............................................18 Dynamic Cores .......................................................................................................................18 Core migration across vPars .....................................................................................................18 Core activation/deactivation in nPars........................................................................................19 Intended Active parameter (IA) .................................................................................................20 Difference in core migration behavior on Superdome 2 compared to legacy Superdome.................20 LORA ....................................................................................................................................20

Introduction
HP Superdome servers are scalable, cell-based systems, which support HP nPartitions (nPars) and HP-UX Virtual Partitions (vPars). This paper discusses the differences in managing partitions on the next-generation of HP Superdome servers, referred to as Superdome 2, as compared to the previous generation of HP Superdome, rx8640, and rx7640 servers, sometimes referred to as legacy systems. Legacy Superdome systems are referred to as SD1 in this paper. This document is primarily targeted toward users who have prior nPar and vPar experience.

What are nPars and vPars?

HP nPar are hard partition technology, within the HP Virtual Server Environment that enables you to configure a single server complex as one large server or as multiple smaller servers. Each nPar has one or more cells (containing processors and memory) that are assigned to the partition for its exclusive use. Any I/O that is attached to a cell belonging to a partition is also assigned to that partition. Since each nPar has its own processor, memory, and I/O resources consisting of the resources of the cells allocated to the nPar, resources may be removed from one nPar and added to another without having to physically remove and add hardware. Additionally, dynamic creation and modification of nPars is supported. With HP-UX 11i v3, nPars become dynamic, meaning resource reconfiguration can be done without shutting down the affected nPars. This document covers the new features of the partition management solution on Superdome 2 servers and the key differences from previous platforms. In addition, legacy SD1 partition management features that are not available in the first release of Superdome 2 servers are also stated.

Superdome 2 partition management overview

There are three main aspects of partition management: 1. Partition configuration and re-configuration 2. Partition start/stop 3. Management of the OS running on the partition This paper focuses on the first two aspects, partition configuration and partition start/stop. The partition management architecture has changed on Superdome 2 servers to adapt to the new hardware and firmware architecture. The core of partition management functionality now resides on the built-in Superdome 2 Onboard Administrator (Superdome 2 OA). The new OA-based partition management architecture supports a unified nPar and vPar management model based on the fact that partitioning (both nPars and vPars) is now entirely firmware functionality. There are no longer any dependencies on software tools, no need for an external management station, or a special partition to run tools. The result is faster, easier, partition configuration, and partition start/stop. Both GUI and CLI are supported on the OA to manage partitions.

Note: On servers prior to Superdome 2, partition configuration management software primarily ran on the system processors on the partition side. Hence, in order to ease the transition to the new management model, legacy partition management command interfaces with minor modifications are still supported from the partition side.

The partition management aspects described in this document covers the management tools available for partition configuration/reconfiguration operations and partition start/stop related operations.

Figure 1: Interacting with Superdome 2 partition management

HP-SIM and Plug-ins (For example: vman, capad, gWLM)

Remote System

SMH gWLM

nPar CLI, vPar CLI, iCAP CLI

An nPar or vPar

WBEM (CIMOM and providers)

Superdome 2 Complex

IPMI

Requires network connection between the users desktop and the MP

OA GUI + nPar, vPar GUI

OA

OA CLI + nPar, vPar, iCAP CLIs iCAP Usage Rights Mgr Can use network or serial connection between the users desktop and the MP

Figure 1 depicts the different ways a user can interact with the system to manage partitions. There are primarily three ways users can do partition management on Superdome 2 servers. 1. From the Superdome 2 OA OA GUI for all nPar and many vPar uses OA CLI for all nPar and all vPar commands 2. From the partitions Legacy (SD1 and earlier) partition management command interfaces are still supported from the partition side 3. From a data center management environment such as HP Systems Insight Manager (SIM) Get health information for the entire Superdome 2 complex and each partition Click down into the Superdome 2 OA to access the Superdome 2 partition management tools

nPar new features in Superdome 2

In general, Superdome 2 nPars are a super-set of previous systems nPar characteristics. Superdome 2 has finer-grained resource granularity (processors and I/O) than offered on previous platforms, while retaining the electrical isolation and native hardware performance supported by all nPars. Users can manage their nPars from both the new Superdome 2 OA methods, and from the legacy OS side tools. The new features as well as the differences between new and old systems are described in detail here.

Finer socket granularity

Superdome 2 supports finer-grained nPars, with a minimum of two processors per nPar instead of four processor nPars as defined by SD1. Superdome 2 also supports more I/O granularity, with I/O expansion chassis that can be split between partitions.

Integrated nPar/vPar management

The new Superdome 2 OA is a single-point of management for the whole complex and provides a new architecture, which integrates nPar and vPar management. In prior systems, vPar management required a separate set of tools and commands. In the Superdome 2 OA, each vPar is presented more or less like a finer-grained nPar, and so similar tools can be used to manage both, reducing administrator time and effort. User interfaces for common partition operations like boot, shutdown, reset, and the like are the same and are available from the OA CLI. Each vPar has a separate console and presents an EFI shell just like an nPar. OS installation operations for an nPar and vPar are the same, and so are the OS boot methods. The partition status display commands are also better integrated. The vPar configuration information (legacy vPar database) is now a part of the parspec, which defines the overall nPar configuration. Parspecs are described in the following section.

Partition specifications or parspecs

A parspec is a collection of data that defines the configuration of a specific nPar, including the vPars in the nPar if any are defined. Following are the key features of parspecs.

Every parspec defines only one nPar. Multiple nPars must be described in multiple parspecs. An nPar can be created from a specified parspec. Every nPar that exists has a parspec associated with it (referred to as a current parspec), and there can be only one
current parspec for a given nPar.

Any parspec not associated with an existing nPar is an alternate parspec. Every parspec has a unique name (scope of uniqueness is a complex). Parspecs can be managed through the partition commands. The vPar configuration information (legacy vPar database) resides within a parspec. Parspecs are stored in the OA, and are therefore available regardless of the partition state. Parspecs change policy attributes allow customers to specify a modification behavior should a parspec not be able
to attach all the resources requested in the definition.

Use parspec functionality to do the following:

Save nPar configurations Modify nPar configurations Create nPars Maintain alternate partition definitions for use at different times of the day, week, or month Quickly restore or reconfigure a system from tested nPar definitions
For more details on parspec, refer to the parspec manpage [parspec (5)] or the appropriate section in the HP Superdome 2 Partitioning Administrator Guide.

OS-agnostic resource paths

Resource path is a notation that can be used to specify a hardware resource. Each resource path consists of a hierarchical list of entity numbers. The hierarchy describes parent-child relationships. The parent is listed to the left of the child. Each entity number is relative to the parent. The separator between the entity numbers is the slash (/) character. The numbers that form the resource path are a user-friendly way of representing the actual hardware hierarchy of a resource. In addition, the resource paths are OS-agnostic, in the sense that, they are independent of how any OS chooses to represent a resource. Specifically, a resource path is not the same as an HP-UX hardware paththey are in different namespaces. You can map one to the other with the new m resourcepath option to ioscan (1M) Resource paths can be specified in three different formatsshort, medium, and long. Partition commands support all three forms of resource path specification. Table 1 shows a few examples.
Table 1: Partition command resource paths Resource Blade I/O Bay Socket CPU-core Rootport I/O Slot Short form 3/6 7/1 3/6/0 3/6/0/2 3/6/0/1/2 3/6/5 Medium form blade-3/6 iobay-7/1 cpusocket-3/6/0 cpucore-3/6/0/2 iorp-3/6/0/1/2 ioslot-3/6/5 Long form enclosure3/blade6 enclosure7/iobay1 enclosure3/blade6/cpusocket0 enclosure3/blade6/cpusocket0/cpucore2 enclosure3/blade6/iohub0/iorc1/iorp2 enclosure3/blade6/ioslot5

For more details on resource paths, refer to the resource path manpage [resource path (5)] or the appropriate section in the HP Superdome 2 Partitioning Administrator Guide.

Improved partition run-states

The partition run-states displayed by the status commands and OA GUI reflect the actual state of the partition varying from a firmware boot state to a state where an OS has been successfully booted in a partition. The virtual front panel display on legacy platforms has been replaced by the OA GUI display of the partition run-state information, as well as the parstatus and vparstatus commands on the OA CLI. The run-states are also common to both nPars and vPars. Both nPars and vPars have a state and a run-state. The state is either active or inactive. The various run-states that an nPar can be in are listed in Table 2. Table 2: Partition run-states
Run-state Down Activating FWBOOT EFI OSBOOT Up Shut Deactivating Resetting MCA INIT ALT_DEFINED RUN_VPARS Unknown DETACHED Description The partition is inactive and powered off. A boot operation has been initiated for this partition. The boot process is in the firmware boot phase for this partition and the partition has transitioned into the active state. The partition is at the EFI shell. The boot process is in the OS boot phase for this partition. The OS in this partition is booted and running. A shutdown/reboot/reset operation has been initiated on this partition. The partition is being deactivated (powered down) as part of a shutdown or reboot operation. A partition reset is in progress. A machine check (MCA) has occurred in the partition and is being processed. An INIT (or TOC) operation on the partition is being processed. This is a partition definition in an alternate parspec. The nPar is either running vPars or is ready to run vPars. The status is not known. This may reflect an error condition or a transitory state while partition states are being discovered. Firmware services are not available to the OS running on the partition. Examples of firmware services are the partition console, partition management, OS IPMI logs, boot settings, and error handling. The partition must be forcibly powered off to recover it to the inactive state. The following OA CLI command can be used to forcibly power off the partition. Once powered off firmware control will be restored and normal operation following a boot will be possible.

POWEROFF PARTITION <nParID> FORCE

Typically, you can see that an inactive partition is in a defined, down, or activating run-state. But there may be rare occasions where you can notice an inactive partition to be in deactivating, resetting, MCA, or INIT run-states. This can happen if a partition is subjected to these operations while the partition was in the activating run-state.

OS specific default values for nPar attributes

Parspecs support using OS-specific default values for many of the nPar attributes. When an nPar is created for a specific OS type, the default values specific to that OS type are applied to the nPar attributes. For example, HP-UX may want hyper-threading to be enabled by default when nPars are created to run HP-UX. These default attribute values are applied without a need for user intervention. The partition management software applies predetermined OS specific defaults when the user creates a partition for a specific OS type. However, there is also support for a new pardefault command which displays the OS specific attribute values and allows users to change them if needed.

Improved h/w diagnostic display

Partition status display commands show better resource health status and partition health status information. The resource hierarchy information is also reflected in the resource health status display.

vMedia/DVD support
Previous Superdome systems required an external DVD-ROM be attached to each partition. In Superdome 2, the Integrity iLO 3 Virtual Media (vMedia) built into every cell blade provides a virtual DVD-ROM/USB key. Using vMedia commands provided in the Superdome 2 OA, a partition can be given use of the physical DVD-ROM, which is built into every Superdome 2 enclosure. Alternatively, the OA web GUI can be used to connect to a remote DVD-ROM or iso image located in the GUI users PC. This allows an Superdome 2 partition to boot and use standard media without having to purchase and attach a separate physical DVD-ROM to each partition. A physical DVD-ROM may still be attached to a partition by use of the USB port provided on the SUV cable. This is a temporary attachment to an individual blade.

Memory granularity is now an nPar attribute

Memory granularity is now managed as an nPar attribute independent of vPars. All memory in an nPar is considered to be slices of memory each with size equal to or less than the memory granularity value for the nPar. The granularity of memory does not have any relevance in an nPar running in nPars mode in the first release of Superdome 2 servers. The nPar commands parcreate and parmodify can be used to specify the memory granularity value. The vparenv command in HP-UX will continue to support specifying the memory granularity value for the next boot of the nPar. On Superdome 2 servers, default memory granularity value is chosen based on the OS type of the nPar and may be adjusted by firmware based on the total memory in the nPar and the overall distribution of memory among the blades in the nPar. HP recommends that users do not change the default memory granularity value chosen by the system, unless there is a specific need to do that. In case there is a need to set a different value for the memory granularity, users are urged to use the nPar commands (and not vparenv) to manage memory granularity on Superdome 2 servers.

nPars differences in Superdome 2 from legacy implementation

Resource specifications use new OS-agnostic resource paths
As explained earlier, partition commands on Superdome 2 servers require that resources be specified using resource paths. On previous platforms, partition commands accepted HP-UX hardware paths as input. The status display commands also used HP-UX hardware paths for displaying resources. This has changed to the new OS-agnostic resource path format.

Transition from chassis/cell (or global cell) usage to enclosure/blades

Superdome 2 servers have a bladed architecture. This means that the equivalent of cells on previous platforms is now referred to as cell blades. You will find cells and blades used interchangeably in the documentation. The resource path format for addressing blades is enclosure#/blade#. Partition commands display cell blades in this format and accept this format as input to partition commands. Cell blades are explicitly assigned to nPars using the -a option of parcreate command. The legacy -c option is also supported and can be used to assign blades.

Assigning I/O to nPars

Superdome 2 I/O Expansion Enclosures (IOX enclosures) contain two bays, each of which may be separately assigned to partitions. Unlike previous platforms, IOX enclosures are not directly attached to cell blades on Superdome 2 serversthey have to be assigned to specific partitions, allowing more flexibility in how I/O gets assigned. Each I/O bay in the IOX enclosure is assignable to an nPar. So, in addition to cell blades, every nPar must have I/O explicitly assigned to it. The resource path format for specifying an I/O bay is IO_enclosure#/iobay#. I/O bays are assigned using the -a io:enclosure#/iobay# option of parcreate command. Note that there are some I/O slots on the blade itself. These are two dual-port LAN on Motherboards (LOMs) and three mezzanine card slots. The mezzanine slots are assigned the slot numbers 1, 2, and 3. The LOMs are assigned the slot numbers 4 and 5. iLO vMedia is assigned slot 6, and iLO vKVM is assigned slot 7. The mezzanine cards are not supported at the first release. If the I/O on the blade is not sufficient for the nPar, additional I/O may be assigned by using the IOX expansion enclosure.

Finding the resource path of I/O

The resource path of your blade-based I/O components can be found through the OA CLI or through EFI commands. Using the Superdome 2 OA, the command parstatus c {enclosure#/blade#} V. For example, parstatus-c1/2-V reports the path of the LOMs on blade 2 in enclosure 1. Using EFI, the info io command will return the results of all resources paths to all I/O. Here is an example result for a dual ported LOM (enclosure 1, blade 6, I/O slot 4) and an IOX Fibre Channel card (in IOX 9, bay 2, slot 6):
1/6/4 10/00 00/00 103C/402F //11 Bridge DevicePCI/PCI bridge 1/6/4 10/01 00/00 14E4/1650 103C/7105/00 Network ControllerEthernet control 1/6/4 10/01 00/01 14E4/1650 103C/7105/00 Network ControllerEthernet control 9/2/6 7D/00 02/00 103C/402F //11 Bridge DevicePCI/PCI bridge 9/2/6 7D/80 00/00 1077/2532 103C/3262/02 Serial Bus ControllersFibre Channel

LOMs (LAN on Motherboard) NICs are number 4 or 5.

OA CLI for partition operations

nPar operations that were supported on the management processors of previous platforms are still supported on the OA in Superdome 2 servers. These include the pe, rs, tc, bo, fpl, sel, co, and cl commands. In addition to these, there is a much richer set of OA commands that support these same nPar operations. These include the poweron, poweroff, reboot, init, connect, and other commands. These are OA partition-aware commands that are enhancements of blade management commands. They support both nPars and vPars.

Force and override option for OA CLI

Certain OA commands (like poweron, poweroff, reboot, and so on) support the force and override options for partition operations. A force option indicates that the user wants an ungraceful operation. For example, a poweroff partition 1 force means that the user wants the partition to be brought down, no matter what the state of the partition is. This is an ungraceful operation because the OS running on a partition is not given a chance to do a graceful shutdown. Similarly a reboot partition 1 force is equivalent to an rs command on previous platforms. The commands issued without a force option do a graceful operation by allowing the OS to go through an orderly shutdown or reboot process. Note that when an OS is not running (for example, partition is at EFI), a graceful operation is not possible and the user will have to use the force option. An override option indicates a mode (vPars/nPars) override when the user wants to perform an operation that is not supported in that mode. For example, a user wants to power on a vPar while in nPars mode. Normally, this would succeed only after the user has changed the mode to vPars. But, if the override option is specified, mode is automatically changed to vPars and the specified vPar booted. Similarly an override option is required for the poweroff partition command if all vPars in the nPar have to be shutdown simultaneously. For more details, refer to the HP Superdome 2 Partitioning Administrator Guide.

Partition configuration from the Superdome 2 OA GUI

The Onboard Partition Manager is a set of tools integrated with the Superdome 2 OA. The Onboard Partition Manager replaces the old ParMgr GUI which ran on an OS in a partition. The Onboard Partition Manager tools are a complete set of nPar and vPar configuration and management tools providing nPar and vPar configuration information as well as many other enhancements over the previous GUI tool. The OA GUI is accessible from the partition side through System Management Homepage (SMH).

Root permissions needed for status display commands

The new partition management architecture requires that all partition commands on the partition side be run only by the root user. In prior platforms, the parstatus command did not require root permissions while other partition commands required root permissions.

Remote complex management through partition commands no longer used

Remote complex management options (-g, -u, -h) are not supported from partition commands on Superdome 2 servers. Now that the OA in any Superdome 2 complex has in-built partition management capabilities, any user wanting to do remote management of an Superdome 2 complex can do so by logging into the OA of the remote complex.

Socket local memory instead of cell local memory

On Superdome 2 servers, memory controllers are associated with the processor sockets. Hence the concept of Socket Local Memory (SLM) is what is supported with Superdome 2 servers as opposed to Cell Local Memory on previous platforms.

Blade online addition, deletion, or replacement (Blade OL*) no longer used

Online addition and deletion of blades are not supported in the first release of Superdome 2 servers. Hence the parolrad command is not supported in the first release of Superdome 2 servers. The floating value for the blade_type attribute of the blade as well as the float value for the failure_usage_policy attribute of the blade are not supported as a result of this. All blades are considered base by default.

VFP no longer used

Virtual front panel or VFP display is not available on Superdome 2 servers. This is replaced by the partition run-state displays in OA GUI and the parstatus and vparstatus commands.

Obsolete nPar commands (fruled, frupower, cplxmodify, parunlock)

Certain nPar commands are no longer needed as part of partition commands. They are either not relevant on the new hardware architecture or have equivalent commands available from the OA CLI. Commands that are obsolete are fruled, frupower, cplxmodify, and parunlock.

Parconfig command for configuring LORA partitions no longer used

The parconfig command support for configuring locally optimized resource allocation (LORA) nPars is not available in the first release of Superdome 2 servers. However, some of the LORA attributes, like the more efficient SLM to ILM ratio, are applied by default to nPars created to run HP-UX.

HT is enabled for HP-UX by default

Hyper-threading (HT) is enabled by default in an nPar running HP-UX. In previous platforms, HT was not enabled by default.

10

No interleave blade failure usage policy is not supported

The no interleave value for the failure_usage_policy attribute of the blade is not supported on Superdome 2 servers.

Core cell no longer used

The core cell concept does not exist on Superdome 2 servers and consequently there is no core I/O concept. Hence, the -r option of parcreate command is not supported.

Obsolete partition command IP address

On platforms prior to Superdome 2 servers, users could assign an IP address to an nPar. However, there was no mechanism to keep this IP address in sync with the real IP address chosen by the partition OS. The IP address attribute of the nPar was deemed to be of very little use without being able to keep it in sync with the OS. Hence the -I option of parcreate command is not supported in the first release of Superdome 2 servers.

Unique partition names

On Superdome 2 servers, partitions can be specified either by the partition number or by the partition name. Any partition command accepts either partition name or number to denote a target partition. On previous platforms, nPar names did not have to be unique because the nPar commands did not support addressing an nPar by name. Hence on Superdome 2 servers, nPar names have to be unique within the complex. In addition, nPar and vPar names have the same format. Hence, nPar names can be 254 characters long, but cannot have space in between.

Change in reboot for reconfiguration method

On Superdome 2 servers, any pending configuration for the nPar comes into effect when the nPar is shutdown or rebooted. On earlier platforms, it was required (as per the documentation) to issue a reboot or shutdown with the option for reconfiguration (-r, -h). On Superdome 2 servers, a normal reboot or shutdown operation will result in the pending configuration to take effect. When nPars are configured with vPars, individual vPars have to be shutdown normally and then the nPar rebooted or shutdown for the pending nPar configuration changes to take effect.

Genesis partition no longer used

There is no genesis partition on Superdome 2 servers since users now have the ability to create the partition configuration of their choice from the OA.

Complex Profile no longer used

The Complex Profile as defined on previous platforms does not exist on Superdome 2 complex. Instead the complex profile is replaced by partition specifications and other complex configuration data on the OA.

ILM/SLM ratio
On previous platforms, an nPar created was by default configured with 100 percent ILM. On Superdome 2 servers, the default ILM/SLM ratio is determined by the OS type for which the nPar is created. Use the pardefault command to see the default values for each OS type. It is also worthwhile to note that on Superdome 2 servers, even when the user has specifically requested 100 percent SLM for an nPar, some amount of ILM may still be configured.

11

vPars new features in Superdome 2 vPars

Integrated nPar/vPar management
As described earlier in the nPar new features section, the new Superdome 2 management architecture integrates nPar and vPar management. Refer to the nPar section for more information on the integration of nPar and vPar tools. The vPar configuration information (legacy vPar database) is now a part of the parspec which defines the nPar configuration. Parspecs are described further in the previous nPar New Features section. The partition status and run-states are mostly common to both nPars and vPars. Permission management for partition operations in both nPars and vPars is done from the OA.

Max 16 vPars per nPar

On Superdome 2 servers, up to 16 vPars can be configured per nPar. On previous platforms, this was restricted to 8 vPars per nPar. Also see the restriction of 4 vPars/blade mentioned later in this document.

Maximum of 4 vPars/blade as opposed to 8 vPars/cell

Previously, the maximum number of vPars that could be configured in a single cell board was 8. The maximum number of vPars that could be configured in a single nPar (be it a single cell nPar or 8 cell nPar) was also 8. The Superdome 2 architecture increases how many vPars can be supported in an nParup to 16 vPars in a single nPar. However, only 4 vPars can be configured in a single Superdome 2 cell blade. Another way to express it is: the maximum number of Superdome 2 vPars per nPar is the lesser of 16 and 4 x N, where N is the number of blades in the nPar.

Separate console for vPars

The console is not shared on Superdome 2 vPars. Each vPar gets its own console, supported by dedicated hardware on each blade. The console commands available on the OA can be used to connect to individual vPars just as they are used to connect to nPars.

vMedia/DVD support
As described in the nPar new features section, Superdome 2 uses iLO vMedia to attach a virtual DVD-ROM/USB key to either an nPar or a vPar. For vPars vMedia assignment, vMedia can be attached to the vPar which has been assigned the cell blade iLO port. In Superdome 2 systems, there is one physical Integrity iLO 3 management processor built into every cell blade. Since there is only one iLO per cell blade, the number of vMedia/DVDs available for vPars is limited to the number of cell blades (the number of iLOs) in the nPar. The IO paths (rootports) having iLO capability are displayed by the parstatus c {blade} V command. vPar commands can be used to assign the rootport or IO slot hosting the iLO to a vPar, similar to assigning any other IO to a vPar. To get access to the enclosure DVD drive from a vPar, in addition to assigning the rootport having iLO capability to the vPar, the set partition dvd OA command (or the OA GUI) should be used to connect the DVD to a vPar.

vPar management through the Superdome 2 OA

A subset of vPar management is available through the OA GUI. vPars resource assignments and detailed vPar information is displayed in the Superdome 2 OA GUI. Details include resource assignmentsCPU, Core, Memory, and IO. vPars can be powered on and off from the Superdome 2 OA GUI. A complete set of vPar management is available through the OA CLI. At first release, vPars cannot be created or modified from within the Superdome 2 OA GUI. For user convenience, an OA CLI command prompt window can be launched from the partition configuration section of the OA GUI and the vPar CLI commands may be run from there.

12

vPars differences in Superdome 2 from legacy implementation

No vPars monitor or vPar monitor commands
On Superdome 2 servers, vPars functionality is entirely supported by firmware. There is no vPars monitor (unlike prior platforms). All vPar configuration and management can be performed from the OA. Since there is no vPars monitor, there are also no vPar monitor commands. Boot operations previously supported by the vPars monitor are now supported from the OA CLI. Note that users might see a /stand/vpmon image after installing the vPars product on HP-UX for Superdome 2 servers. This is because of the common packaging used for both Superdome 2 and previous platforms. Trying to boot vpmon in an Superdome 2 environment will result in an unsupported error message. You will see that OA CLI command (parmodify N {mode}) allows you to set the right mode (vPars or nPars) and boot an nPar as a single OS or as a set of vPars. Unlike prior platforms, users do not have to get to the EFI shell of an nPar to set the boot mode for the nPar or try to boot the vPars monitor.

Booting all vPars simultaneously

As a result of there being no vPar monitor, the way to boot all of the vPars at once is also different. Earlier, users could issue a vparloadall from the monitor prompt or boot vpmon with the -a option to boot all the vPars simultaneously. On Superdome 2 servers, this is achieved by doing a poweron of the nPar in vPars mode from the OA CLI.

OA commands for vPar operations

On Superdome 2 servers, partition operation commands (poweron, poweroff, reboot, INIT, and so on) are common to vPars and nPars and hence support all vPar operations earlier supported by explicit vPar commands. For example, OA poweron command supports boot of a vPar and provides the functionality of legacy vparboot command. Similarly, OA poweroff/reboot/INIT commands support vparreset functionality. Note that both vparboot and vparreset legacy commands are supported from the OA and partition side. However, users are urged to use the poweron/poweroff and related OA commands to manage vPars.

vPar commands from OA need to specify the nPar context

Since OA does not have a partition context, vPar commands executed from the OA should also include the nPar identifier which hosts the vPars. The new -N <nPar_id> option of vPar commands supports specification of the target nPar. From the OA, vPar can also be addressed using a combination of nPar and vPar identification in the format <nPar_num_or_name:vPar_num_or_name>. Note that the -N option is not supported from the partition side. Partition side vPar commands can only operate on the local nPar as on previous platforms.

OS-agnostic resource paths

As explained earlier, partition commands on Superdome 2 servers require that resources be specified using resource paths. On previous platforms, partition commands accepted HP-UX hardware paths as input. The status display commands also used HP-UX hardware paths for displaying resources and will now be displaying resources in resource path format. Users are required to use the partition status display commands (parstatus/vparstatus) to view all resources in the complex/nPars and use the resource paths as input to all vPar configuration commands. The partition status display commands show information to the level of vPar assignable resources. Note that I/O resources can now be assigned either by using the I/O slot resource path or the rootport resource path.

13

Transition from chassis/cell usage to enclosure/blades

As previously described, Superdome 2 servers have a bladed architecture. This means that the equivalent of cells on previous platforms is now referred to as cell blades. The resource path format for addressing blades is enclosure#/blade#. Partition commands display blades in this format and accept this format as input to partition commands. Blade specific resources assignable to vPars have to be specified in the resource path format, which includes the blade specified as enclosure#/blade#. For example, a socket local processor needs to be specified in the format 3/6/0/2, which equates to cpu-core2 in socket 0 of blade 6 in enclosure 3.

Transition from SBA/LBA to iohub/rootcomplex/rootport as well as ioslot usage

The I/O hierarchy for the I/O on the blade is as follows: blade_enclosure/blade/iohub/rootcomplex/rootport. Similarly, the I/O hierarchy within an I/O enclosure is io_enclosure/iobay/iohub/rootcomplex/rootport. The rootport is associated with the I/O slot hosting the PCIe port. Legacy vPar users familiar with the Local bus adapter (LBA) assignment for vPars should note that the I/O assignment is now at the rootport level. In addition to rootport assignment, Superdome 2 vPar commands also support assignment using the I/O slots. Since there is a one-to-one mapping between the rootport and the I/O slot on Superdome 2 servers, vPars I/O can be assigned by either specifying the rootport or the I/O slot. vPar commands support an ioslot keyword for the -a option, which can be used to specify an I/O slot for assignment. The I/O slot resource path format is blade_enclosure#/blade#/ioslot# or IO_enclosure#/iobay#/ioslot# for an I/O slot in blade and I/O bay respectively.

vPar configuration files (vPar database) reside on the Superdome 2 OA (not on HP-UX)
Prior to Superdome 2 servers, the vPar configuration file or database (vpdb) was a binary file that resided on /stand in an HP-UX OS image and contained configuration definitions for all vPars in the nPar. On Superdome 2 servers, with the OA-based partition management that supports an integrated nPar/vPar management in an OS-agnostic way, the vPar configuration is part of the parspec that describes an nPar. The parspec resides on the OA and is created/managed through the partition commands. So a /stand/vpdb file, if seen on HP-UX, does not have any relevance on Superdome 2 servers. On Superdome 2 servers, there is no concept of a vpdb as it existed previously. Each parspec stores the vPar definitions in addition to the nPar configuration. The equivalent of an alternate vpdb now refers to the vPar definitions in an alternate parspec. Consequent to that behavior, there is no -D option supported in vPar commands, instead users have to use the -Z option to target an alternate parspec.

vPar boot follow nPar boot conventions

On Superdome 2 servers, a vPar is presented pretty much like a finer granularity nPar. Each vPar has its own EFI shell. This means that an OS in a vPar can be booted from the EFI shell just like an nPar. Standard EFI/loader/OS mechanisms can be used to set boot disks, specify kernel paths and kernel boot options for a vPar. Consequently, there is no support for specifying boot disk, kernel boot path, or kernel boot options in the vPar commands.

vPar install is now the same as how nPars are installed

As mentioned earlier, each vPar has its own EFI shell. OS installation in a vPar is done in the same way as it is done for an nPar from the EFI shell (Lanboot, DVD, or vMedia). Consequently, vPar commands do not support the -I option to install OS in a vPar.

14

Memory granularity no longer a vPar attribute

As mentioned previously, memory granularity is now managed as an nPar attribute independent of vPars. Please refer to the previous nPar management section for additional information.

Change in default memory granularity

For HP-UX partitions, the default memory granularity value is 1024 MB as opposed to 128 MB on prior platforms.

Change in minimum memory granularity

The minimum memory granularity supported by Superdome 2 servers is 256 MB.

vPar permissions management from the OA

Permission management for partition operations for both nPars and vPars is done from the OA. Consequently, the vparadmin command is not supported on Superdome 2 servers. Instead, the parperm command from the OA has to be used to designate any vPar as an admin vPar. Note that the default vPar permission is now similar to the default nPar permission. Each vPar is by default allowed to target vPar operations only on the local vPar. To target vPar operations on other vPars, a vPar has to be given permissions using the parperm command. On previous platforms, vPars had permissions to target vPar operations on all other vPars in the same nPar by default.

Socket local resources instead of cell local resources

On Superdome 2 servers, the memory controllers are associated with the processor sockets. Hence the concept of socket local memory is what is supported with Superdome 2 servers as opposed to cell local memory on previous platforms. Cell specific resources like memory and CPUs that were assignable as vPar resources are now socket specific resources. So the clm and clp keywords in vPar commands is replaced by slm and slp keywords as appropriate. Consequent to that, the socket resource path has to be specified for assigning socket local memory or socket local processor.

vPar EFI shell

Unlike prior platforms, the EFI shell of a vPar is accessible to the user. This means that all standard EFI commands and utilities are available for use from the vPar EFI shell. As mentioned earlier, boot operations, install operations, and EFI driver update operations can be performed from the vPar EFI shell the same way it is done from an nPar EFI shell.

Memory range support no longer used

Assignment of memory ranges to vPars is not supported on Superdome 2 servers. Memory can only be specified by size or socket locality.

vPars created with zero CPUs by default

On previous platforms, if CPU resources were not specified at vPar creation time, by default one CPU was allocated to the vPar. There have been scenarios in the past where this approach has caused problems where users wanted to create dummy vPars to reserve some resources and ended up having one CPU reserved for allocation to the dummy vPar. On Superdome 2 servers, a vPar is created with zero CPUs by default. Consequent to that, the default cpu:::[min] value is also zero.

Addition/deletion of CPU by path increases CPU count

When CPU-cores are added to a vPar by specifying the CPU resource path, the count of total CPUs assigned to the vPar is also incremented. On prior platforms, the total CPU count was not adjusted when CPUs were added by path. Similar behavior is shown for deletion of CPU-cores by path as well.

15

PIM data retrieval

Previously, a vparreset command displayed the processor information module (PIM) data for the CPUs belonging to the vPar that got reset as part of the command. On Superdome 2 vPar implementation, the vparreset command does not display any PIM data. All CPU error diagnostic information is handled in the same way for nPars and vPars.

OS version display in vparstatus

The -P option of vparstatus command is not supported in the first release of Superdome 2 servers.

Changes in mode switch functionality

An Superdome 2 nPar has an nPars mode and a vPars mode just like the previous Integrity platforms. In nPars mode, the nPar is allowed to boot and run a single OS image. In vPars mode, the nPar is allowed to boot and run a set of vPars each having its own OS image. The difference from prior platforms is in the manner in which the mode can be set. The mode or boot mode is an nPar attribute that can be managed through the nPar commands. On previous platforms, the vparenv command was used to manage the boot mode. The vparenv command is still supported in the HP-UX OS, but not on the OA. Users are urged to use the nPar commands (parmodify N <boot mode>) to set the boot mode in all cases, but must use them if working from the OA. As explained earlier, the mode can be set at nPar creation time using the parcreate command or through the parmodify command. Mode switching can thus be performed from the OA CLI. For an nPar that is active, mode switch happens at next reboot time. For an nPar that is already inactive, mode switch happens when the inactive nPar is booted. On prior platforms, a mode switch required that an nPar be either booted to the EFI shell prompt, booted to the vPar monitor, or be running HP-UX to use the vparenv command.

Obsolete commands
Some vPar commands are no longer required in the OA based partition management solution supported on Superdome 2 servers. They are either not relevant in the new architecture or are replaced by other commands providing equivalent functionality. These commands are vparefiutil, vparutil, vparconfig, vparextract, vpardump, vparadmin, vpardbprofile.

Memory distribution among vPars

System firmware distributes some amount of memory below 4G equally among the maximum number of vPars that can be supported in the nPar. This is required because some of the I/O cards or devices need memory below 4G to function properly. Note that this memory is kept aside for the maximum number of vPars that can be supported in the nPar, not just for the vPars that have been created by the user. By default, the maximum number of vPars per nPar is 16. This means that there will be some unused memory reserved for future vPars that can get created in the nPar. If the user wishes to make better use of this reserved memory, the nPar can be created explicitly specifying a smaller maximum value (than default maximum of 16) of vPars/nPar, so that lesser memory below 4G is reserved. Note that this reserved memory does not have to be explicitly assigned by the user and is not visible in the vparstatus command output.

Partition state and run-states

As described in the earlier section on nPars, both nPars and vPars have a state and a run-state. The state is one of active or inactive. Please refer to the nPar section for more information.

Change in reboot for reconfiguration method

On previous platforms, any pending configuration changes for an nPar running vPars took place only after all the vPars were rebooted using the -R option of the shutdown/reboot HP-UX command and the subsequent reboot of the vPar monitor. On Superdome 2 servers, for pending nPar to come into effect, individual vPars have to be shut down and the nPar rebooted from the OA. The -R option is no longer used.

16

Support only for HP-UX versions that run on Superdome 2 nPars

In Superdome 2, only HP-UX 11i v3 is supported for both nPars and vPars. Superdome 2 does not support running older versions of HP-UX inside vPars.

Default values for SLM/ILM ratio following a mode switch

The default SLM/ILM ratio in an nPar in vPars mode may be different from what is used as default setting in nPars mode. In nPars mode, the default ratio is 7/8 SLM for HP-UX. In vPars mode, the default ILM/SLM ratio for the nPar is 100 percent ILM for the first release of the Superdome 2 servers. Note that this is only for the default value chosen by the system. If the user has created an nPar and specified a certain ILM/SLM configuration, that configuration will be in effect for both nPars and vPars mode.

Parspec change policy

Though the concept of parspecs itself is a new feature, it is worthwhile to mention about the parspec change policy. The parspec change policy is an attribute of an nPar, which the user can enable or disable. By default, it is enabled. The parspec change policy determines whether the OA can automatically modify a parspec in order to enable a partition operation (such as booting). In some scenarios, it is possible that the system cannot meet the customer requested resource or attribute specifications in the parspec for an nPar or vPar. This could be due to non-existent resources or degraded resources or other restrictions in hardware architecture. Under these circumstances, the partition configuration in the parspec may need to be modified to make progress. The parspec change policy attribute of the nPar gives the customer the ability to choose the behavior in these cases. If the parspec change policy is enabled (default), the affected partition operation is continued with available or modified resources or attributes after making the required changes in the parspec. If the parspec change policy is disabled, the affected partition operation is failed and no changes are made to the parspec. For example, consider the scenario where the memory granularity specified by the user cannot be accommodated by the system and a new memory granularity value has to be applied. This is likely to affect the vPar memory sizes specified by the user and stored in the parspec since they have to be rounded up to be a multiple of the new memory granularity. Users may want to make vPar memory size adjustments manually and not have the system make any changes in the vPar memory sizes they have assigned. In that case, they would have to disable the parspec change policy for the nPar before they configure and boot vPars.

Both cpu and core keywords supported with the -a option

vPars have CPU-core granularity. With Superdome 2 vPars, the core keyword can also be used where cpu is used in vPar commands. Hence both cpu and core keywords are considered equivalent for vPar operations. Note that both cpu and core keywords in vPar commands context refer to a CPU-core.

vPar numbers
On Superdome 2 servers, vPars can be specified either by the partition number or the partition name similar to nPars. Any partition command accepts either vPar name or number to denote a target partition. From the OA, a vPar can also be addressed using a combination of nPar and vPar identification as explained earlier. On previous platforms, vPars could only be addressed by names.

Interactive commands no longer used

On legacy servers, certain vPar commands (for example, vparremove) interacted with the user for confirmation of the operation. On Superdome 2 servers, these operations fail with a message requesting users to use the -f option of the command if they really want to perform the specified operation.

17

New Features in the Dynamic Cores release of Superdome 2 firmware

Dynamic Cores
The initial release of Superdome 2 did not support online resource migration in vPars and nPars. Beginning with the Dynamic Cores release (aka DC release), vPars will support online migration of cores across vPars within an nPar, nPars will support online activation and deactivation of cores. The DC release supports a complete integrated solution comprising of core migration capabilities with enforced authorization through integration of iCAP product on the OA and workload management capabilities through integration with gWLM (running in the OS partitions). In addition, this release also supports the full Dynamic Processor Resiliency feature, which allows for replacement of faulty cores in a partition with a healthy iCAP core subject to availability of free cores in the system.

Core migration across vPars

This can be achieved by using either the vparmodify or the icapmodify command on an Active vPar, from the OA or the OS. Cores may be added to or deleted from a vPar that is in the UP state (booted to HP-UX). All variants of adding cores that are supported on an Inactive vPar (a vPar in DOWN state), are also supported for an Active vPar. In other words, cores can be added in one of these three ways : Count based Examples (i) Using OS-based commands Adding cores to a vPar: vparmodify -p vpar1 -a cpu::2 icapmodify -p vpar1 -a 2 Deleting cores from a vPar: vparmodify -p vpar1 -d cpu::2 icapmodify -p vpar1 -d 2 (ii) Using OA-based commands Adding cores to a vPar: vparmodify -p 1:2 -a cpu::2 icapmodify -p 1:2 -a 2 Deleting cores from a vPar: vparmodify -d 1:2 -d cpu::2 icapmodify -p 1:2 -d 2 Socket based Note: Socket-based additions are possible only through the vparmodify interface. It is not supported in the icapmodify interface Examples (i) Using OS-based commands Adding cores to a vPar (add 2 cores from socket 1/1/0) vparmodify -p 2 -a socket:1/1/0:core::2 vparmodify -p 2 -a socket:1/1/0:cpu::2 Deleting core from a vPar (delete 2 cores from socket 1/1/0) vparmodify -p 2 -d socket:1/1/0:core::2 vparmodify -p 2 -d socket:1/1/0:cpu::2

18

(ii) Using OA-based commands Adding cores to a vPar (add 2 cores from socket 1/1/0) vparmodify -p 1:2 -a socket:1/1/0:core::2 vparmodify -p 1:2 -a socket:1/1/0:cpu::2 Deleting cores from a vPar (delete 2 cores from socket 1/1/) vparmodify -p 1:2 -d socket:1/1/0:core::2 vparmodify -p 1:2 -d socket:1/1/0:cpu::2 Path based Note: Path-based additions are possible only through the vparmodify interface. It is not supported in the icapmodify interface Examples (i) Using OS-based commands Adding cores to a vPar vparmodify -p 2 -a cpu:1/1/0/1 vparmodify -p 2 -a core:1/1/0/1 Deleting cores from a vPar vparmodify -p 2 -d cpu:1/1/0/1 vparmodify -p 2 -d core:1/1/0/1

Core activation/deactivation in nPars

For nPars, cores may be activated and deactivated online. This is achieved through the icapmodify command from either the OS partition or the OA. Core deactivation and activation on nPars can be achieved only through the icapmodify interface. Examples (i) Common to both OA-based and OS-based commands Activating cores in an nPar icapmodify -p 1 -a 4 Deactivating cores in an nPar icapmodify -p 1 -d 4

19

Intended Active parameter (IA)

This is used to determine the total number of cores that may be active in the nPar. This is an nPar attribute and can be set during partition creation using the -n option of parcreate. Or it may be modified by using the -n option of parmodify. However, modifications made through the parmodify interface will be effective only on next reboot. Modification of the IA value is supported through the icapmodify interface also ('-s' option). Executing an icapmodify on an Active nPar will cause core activation or deactivation (depending on whether the new value specified, causes the active core count to be reduced or increased). So changes made to the IA value through the icapmodify interface will be effective without a reboot.

Difference in core migration behavior on Superdome 2 compared to legacy Superdome

On Superdome 2, the Partition Controller on the OA picks the cores for assignment during a core addition. In a vPar context, the core assignments are made more efficient so that the cores that are picked are as close as possible to the sockets from which locality memory assignments are made. For core deletion, the OS picks the cores to be deleted. As a result, cores assigned by path or cores assigned by socket localities may also be chosen by the OS for deletion. This is a difference from the legacy systems where path-bound and socket-bound assignments were seldom disturbed. Allowing the OS to choose the cores for deletion helps ensure that the least loaded cores are deleted and locality awareness is still retained.

LORA
With the DC release, the assignment of cores to a vPar either during boot or online core addition is done such that the core distribution is as close as possible to the memory localities owned by the vPar. This helps keep latencies (due to larger distance of cores from the sockets on which memory has been drawn) minimal.

HP Superdome servers are scalable, cell-based systems, which support HP nPars and vPars. To know more, visit www.hp.com/go/Superdome2 and www.hp.com/go/integrity.

Copyright 20102011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 4AA2-0619ENW, Created July 2010; Updated November 2011, Rev. 3