Sie sind auf Seite 1von 8

The Association of System

Performance Professionals
The Computer Measurement Group, commonly called CMG, is a not for profit, worldwide organization of data processing professionals committed to the
measurement and management of computer systems. CMG members are primarily concerned with performance evaluation of existing systems to maximize
performance (eg. response time, throughput, etc.) and with capacity management where planned enhancements to existing systems or the design of new
systems are evaluated to find the necessary resources required to provide adequate performance at a reasonable cost.
This paper was originally published in the Proceedings of the Computer Measurement Groups 2004 International Conference.

For more information on CMG please visit http://www.cmg.org



Copyright Notice and License
Copyright 2004 by The Computer Measurement Group, Inc. All Rights Reserved. Published by The Computer Measurement Group, Inc. (CMG), a non-profit
Illinois membership corporation. Permission to reprint in whole or in any part may be granted for educational and scientific purposes upon written application to
the Editor, CMG Headquarters, 151 Fries Mill Road, Suite 104, Turnersville , NJ 08012.
BY DOWNLOADING THIS PUBLICATION, YOU ACKNOWLEDGE THAT YOU HAVE READ, UNDERSTOOD AND AGREE TO BE BOUND BY THE
FOLLOWING TERMS AND CONDITIONS:
License: CMG hereby grants you a nonexclusive, nontransferable right to download this publication from the CMG Web site for personal use on a single
computer owned, leased or otherwise controlled by you. In the event that the computer becomes dysfunctional, such that you are unable to access the
publication, you may transfer the publication to another single computer, provided that it is removed from the computer from which it is transferred and its use
on the replacement computer otherwise complies with the terms of this Copyright Notice and License.
Concurrent use on two or more computers or on a network is not allowed.
Copyright: No part of this publication or electronic file may be reproduced or transmitted in any form to anyone else, including transmittal by e-mail, by file
transfer protocol (FTP), or by being made part of a network-accessible system, without the prior written permission of CMG. You may not merge, adapt,
translate, modify, rent, lease, sell, sublicense, assign or otherwise transfer the publication, or remove any proprietary notice or label appearing on the
publication.
Disclaimer; Limitation of Liability: The ideas and concepts set forth in this publication are solely those of the respective authors, and not of CMG, and CMG
does not endorse, approve, guarantee or otherwise certify any such ideas or concepts in any application or usage. CMG assumes no responsibility or liability
in connection with the use or misuse of the publication or electronic file. CMG makes no warranty or representation that the electronic file will be free from
errors, viruses, worms or other elements or codes that manifest contaminating or destructive properties, and it expressly disclaims liability arising from such
errors, elements or codes.
General: CMG reserves the right to terminate this Agreement immediately upon discovery of violation of any of its terms.
Planning for Partitioned AIX Systems

Jaqui Lynch
Mainline Information Systems

The introduction of logical partitioning (LPAR) and dynamic logical partitioning
(DLPAR) with AIX 5 has caused a major change in the thought processes
involved in planning in the pSeries world. Previously, sizing a system correctly
involved pre-planning for resource consumption in terms of memory, CPU,
DASD and adapters. New dynamic partitioning features on pSeries partitionable
systems have introduced flexibility in resource allocation that minimizes these
constraints but also requires other types of planning.


Introduction:
LPAR technology enables workload
consolidation by running multiple
partitions on the same box. These
partitions can be either Linux or AIX
partitions, and both OSs can be run
on the same server concurrently. In
the future additional operating
systems will be added to the mix.

However, there are some important
items to remember when planning
for this environment. The number of
partitions available depends on the
model and the number of processors
installed. Some features such as
CUoD (Capacity Upgrade on
Demand) are only available on
certain systems. (Note: pSeries Facts
and Features (G320-9878) provides
feature codes for partitionable
hardware series and those machines
that allow for CUoD.) These are just
a couple of areas where it is easy to
get caught by surprise.

Why partition?
As more and more companies try to
cut costs server consolidation has
become increasingly important.
Consolidation brings benefits in the
areas of floor space, power and
cooling costs. But it also sharply
reduces management costs provided
planning is done. Partitioning
allows production and test systems
to be combined on the same
hardware such that they are isolated
from affecting each other. It allows
for multiple operating systems to be
on the same hardware and enables
the consolidation of applications that
may need to be in different time
zones. Partitioning also helps with
complying with license agreements,
as it allows you to put a product in a
correctly sized system. This does
depend on the vendor though.
Effectively, partitioning brings
mainframe capabilities to the UNIX
world

Terminology:
Partitioning introduces some new
terminology and some concepts that,
while they have been in the
mainframe world for a long time,
may be new to those in the
distributed world. Some of these are
detailed below:
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
1. Logical Partition this is
basically an operating system
instance with a set of user
defined system resources. These
resources consist of processor,
memory and I/O resources.
2. Full System Partition when all
the managed resources are
assigned to one operating system
image this is called a full system
partition. In this mode all
memory is allocated to the
system and there is no
Hypervisor overhead.
3. Managed systems are systems
that are physically managed and
attached to an HMC (hardware
management console). An HMC
is required by all partitioned
systems.
4. Affinity Partitions these are not
that common but refer to
partitions where the processor
and memory resources are
allocated to LPARs based on
which MCM (multichip module)
they are physically installed in.
5. Partition profiles refer to a
repository where information is
kept on assigned resources for a
partition. When a partition
profile is activated the LPAR is
activated. Resources assigned to
a partition profile are only
allocated when the partition is
actually activated. Thus it is
possible to have more resources
listed in partition profiles than
are physically installed on the
system they just cant all be
active at the same time.
6. System profiles refer to a
collection of partition profiles
that get activated at the same
time. This allows for some
automation and a grouping of
views of the system.

Role of the HMC:
The first major change when moving
to a system that will be partitioned
(i.e., currently p630, p650, p655, p670
or p690)

is the requirement for a
Hardware Management Console
(HMC). AIX V5 and the HMC are
required for all partitioned systems.
The HMC is also required if CuOD
(capacity upgrade on demand) is to
be used. The purpose of the HMC is
to manage the systems, to provide a
console to manage the hardware, to
act as a service focal point, and to
detect, report and store changes in
hardware. The HMC can be
attached to both partitioned and
non-partitioned systems.

The HMC is also used to create and
store partition profiles, start and stop
partitions, display status, and
provide a virtual console to connect
to a partition. Partitions cannot be
installed using a monitor attached to
a graphics card assigned to that
partition. For non-partitioned
systems, the HMC shows the LED
codes.

The HMC is required for
management functions, but in the
event of an HMC failure event, the
managed partitions will still remain
operational. The HMC must be
backed up regularly to DVD, and
these DVDs should be added to any
offsite backup plan. If the HMC is
not backed up and the hard drive
fails the HMC will try to rebuild as
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
much as it can from the LPAR
information saved in the various
pSeries boxes, however all custom
information, such as security and
network settings will have to be
manually rebuilt. It is safer to just
set up the backups!

HMC and Async Adapters:
The HMC has 2 serial ports the first
can be used to attach a managed
system using RS-232 and the second
is used for the Call Home feature. In
order to manage more than one
system, an 8 port or 128 port Async
adapter can be installed in the HMC.
This allows the HMC to control
multiple pSeries with many
partitions. The best time to install
the additional async ports is when
the HMC is first purchased this
avoids the necessity to upgrade it
later.

When using the 128 port async
adapter it is important to pay
attention to correctly configuring
cables, converters and other
necessary items to make the RANs
(remote asynchronous node)
useable. The 8 port adapter supports
both RS-232 and RS-422 (used by
7040-W42 rack) attachments. The
128 port adapter supports only RS-
232. For this reason most sites
automatically configure the HMC
with 2 x 8 port async adapters.


Resource Allocation:
Currently, each partition must have
exclusive access to its assigned
resources. The minimum assigned
resources for an LPAR include one
processor, 256 MB of memory, one
boot disk, an adapter to access the
disk, an Ethernet adapter to access
the HMC and methodologies to run
diagnostics (one LPAR must be
assigned as the service LPAR) and
for installation (e.g., NIM for
installs). Resources are assigned at
the slot level, which means that all
disks attached to an adapter are
assigned as one group to one LPAR.

There are also specific rules with
respect to assigning devices in the
media drawer. In some cases the
whole media drawer and the internal
disks must be allocated to the same
partition and on other systems the
drawer and disks can be split
between partitions. This varies by
pSeries system. (Note: Slot
placement should be in accordance
with the PCI Replacement Guide:
www-
1.ibm.com/servers/eserver/pseries/
library/hardware_docs/pci_adp_pl.
html.)

As an example the p630 requires that
PCI slots 1 and2, the internal
Ethernet and SCSI adapters and the
ISA based I/O (serial, keyboard and
mouse) all be assigned together.
Dynamic LPAR cannot be used for
those devices. The p650 can have a
split backplane which not only
allows the 4 internal disks to be split
between 2 partitions (2 disks each)
but it also enables Ultra 320 disk
speeds and the ability to float the
media drawer between partitions.

When assigning resources to an
LPAR, they can be assigned as part
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
of the minimum, desired or
maximum requirements. Minimum
refers to resources that must be
allocated to boot, desired refers to
resources the system should try and
allocate normally on boot, and
maximum refers to the maximum
resources for that partition to use for
dynamic allocation.

It is possible to assign one CPU as a
minimum, three for desired and
eight for maximum which means the
partition will not boot unless it can
get one CPU, it will always try to
boot with three and the ability is
there to increase the number
allocated to up to eight using
DLPAR.

Dynamic LPAR
DLPAR is the ability to dynamically
add, remove or redirect resources
(processors, memory and PCI
adapters) to and from partitions. At
a minimum this requires AIX 5.2 to
be installed on the O/S image. The
HMC must also be at the correct
level (post 10/2002 at least R3v2.2).
IT is recommended that the HMC be
kept at the latest maintenance
versions at all times.

DLPAR enables the use of Memory
and Processor Capacity Upgrade on
Demand (CUoD), which allow
currently unassigned processors and
memory to be turned on and off. The
added capacity can be either
temporary or permanent. Resources
can be added or deleted to/from a
partition or they can be moved
between DLPAR capable systems
(i.e. those running AIX 5.2).
Resources that can be moved include
processors, memory and PCI
adapters. It is important to note that,
when using DLPAR, the maximum
resource setting cannot be exceeded.
So, if maximum processors is set to 4
then that settings has to be increased
before 5 processors can be activated
for that LPAR. The maximum
setting cannot be increased for
memory without deactivating and
reactivating the LPAR.

One final point to note is that it is
important to note whether the
applications being run are LPAR
aware and LPAR safe. LPAR aware
means they notice if resources are
added or removed using dynamic
LPAR. LPAR safe means they do not
crash when this happens and that
they automatically adjust their own
settings if possible. This is an
important question to ask database
vendors if you plan to implement
and actively use DLPAR.

Memory
When planning for memory, it must
be noted that some of the memory is
used by the pSeries to retain critical
information. The amount that must
be reserved depends on a number of
factors, including the number of I/O
drawers, number of partitions and
how maximum memory for those
partitions is set.

At the bottom of real memory, 256
MB is reserved for the Hypervisor.
At the top of memory, the
Translation Control Entry (TCE)
reserves memory to be used to
translate I/O addresses to system
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
memory addresses. This ranges from
256 MB to 1024 MB (in 256 MB
increments) depending on the
pSeries model and the number of
I/O drawers. Additionally, Page
Table entries are required for each
partition. The minimum reserved for
Page Table entries is 256 MB, and
each partition reserves 1/64 of the
maximum memory value set in the
HMC for its use (rounding up to
nearest power of two). The Page
Table entries for an individual LPAR
must be in contiguous memory. On
a p690, its not uncommon for the
overhead to be 2 GB with only two
partitions. There are additional
overheads associated if the OS is
lower than AIX 5.2.

Memory Address Modes:
AIX v5.1 will only run in the default
LRM (Large real mode) address
region. This assigns memory on
256mb, 1gb and 16gb boundaries
depending on the size of memory to
be allocated, and this has to be
contiguous real mode memory. This
means that to allocate a 17gb region
the system needs to find 16gb
contiguous and then will obtain the
final 1gb in 256mb LMBs (logical
memory blocks). Because of the
space reserved by the TCE (top of
memory) and the Hypervisor
(bottom of memory) this partition
would require a system with at least
48gb of memory before it could be
activated as it would need to allocate
the 16gb contiguous starting at 17gb
through 32gb.

AIX v5.2 and most Linux
implementations can run in Small
Real Mode Address Regions (this
needs to be enabled for this partition.
This avoids the memory boundary
constraints mentioned above.



Operating System Versions:
Supported operating systems in a
partitioned environment include AIX
5.2, AIX 5.1, Suse Linux, United
Linux 1.0, Redhat Linux EL AS3,
TurboLinux and Connectiva Linux.
No version of AIX prior to v5 will
work on a partitioned system. These
disparate systems can be run
concurrently on the same pSeries
system provided they are in separate
LPARs. It should be noted,
however, that the true enabler of the
advanced features is AIX v5.2. AIX
v5.2 enables an LPAR to have more
than 256gb, dynamic LPAR, and
Capacity Upgrade on Demand. It
also enables dynamic processor
sparing.

It is critical that the required
microcode levels for the pSeries box,
PCI cards (especially fibre) and the
HMC be checked and updated. The
operating system maintenance levels
should also be checked. pSeries
support information can be found at
48gb boundary
TCE 256mb to 1gb



Extra gb
32gb boundary
17gb partition aligned on a 16gb
boundary AIX 5.1


16gb boundary



Page Tables (multiples of 256mb)

256mb Hypervisor
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
IBMs website
1
. As of March 2004
HMC maintenance became the
customers responsibility and HMC
corrective service can also be found
at the IBM website.

Math 101 and Workloads:
One area where many have been
caught out is when consolidating
workloads from multiple boxes onto
multiple partitions on one system. It
is very common to undersize the
system being consolidated to. This is
best illustrated using an example:

We have 4 workloads below that are
running on 4 separate boxes. Their
rperf needs are:
A 6.03
B 2.27
C 2.48
D 4.87
Total 15.65

When looking at the pSeries systems
the p650 8way 1.45ghz is listed at
16.88 rperf so this should be big
enough for this combined
workloads. BUT, the issue is they
will still be separate LPARs on the
new system and the granularity for
allocation is one processor. So, using
math 101:
Processors Listed rperf
8way 16.88
4way 4.43
A little division using the 4 way
number provides around 2.21 as the
rperf number to use for one
processor. Lets look at those
workloads again:


1
http://www-
1.ibm.com/servers/eserver/support/pseries/
Workload rperfs Processor
needed on p650

A 6.03 3 (6.64)
B 2.27 2 (4.42)
C 2.48 2 (4.42)
D 4.87 3 (6.64)
TOTAL 15.65 10 (22.21)

Since a p650 only has 8 processors
this system would not work here.
The issue is granularity of workload
and processors. For workload B 2.27
rperf was needed but one processor
was only 2.21 rperf so two had to be
allocated. Normally one might
choose to take the risk of only
allocating the one processor but the
p650 is maxed out at 8 processors so
there would be no easy way to add
processor power if the extra
processor really was needed.


Summary
As can be seen from the brief
summary above, planning is now
even more critical. In order to get the
best value from pSeries partitions,
its important to implement a
resource plan and maintain planning
worksheets and checklists to help
ensure that resources arent over
committed and that unnecessary
overhead is avoided.

Recommended Reading
For more information on LPAR
planning, check out the following
reading materials.

1. Complete pSeries Partitioning
Guide Redbook (SG24-7039)
12/21/03
Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e
2. pSeries Facts and Features (G320-
9878) 01/27/04
3. Redpiece on Planning for LPAR
(Tips0119) 05/07/03
4. Redpiece on TCE Table Memory
Sizes (Tips0357) 01/22/03
5. p690 Technical Support
Certification Redbook (SG24-7195)
Feb 03
6. Systems Management using the
HMC (SG24-7038) Aug 03
7. IBMs LPAR Checklist and
Worksheets (www-
1.ibm.com/servers/eserver/pseries/
lpar/checklist_worksheet.pdf)


Current Partition Capabilities:

System Max
Processors
Max GB
Memory
MAX I/O
Drawers
Max
Partitions
P690 32 1024 8 32
P670 16 256 3 16
P655 8 64 1 4
P650 8 64 8 8
P630 4 32 2 4
P630 deskside 4 32 0 3

Find a CMG regional meeting near you at www.cmg.org/regions
Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference
B
u
y

t
h
e

L
a
t
e
s
t

C
o
n
f
e
r
e
n
c
e

P
r
o
c
e
e
d
i
n
g
s

a
n
d

F
i
n
d

L
a
t
e
s
t

C
o
m
p
u
t
e
r

P
e
r
f
o
r
m
a
n
c
e

M
a
n
a
g
e
m
e
n
t

'
H
o
w

T
o
'

f
o
r

A
l
l

P
l
a
t
f
o
r
m
s

a
t

w
w
w
.
c
m
g
.
o
r
g
J
o
i
n

o
v
e
r

1
4
,
0
0
0

p
e
e
r
s

-

s
u
b
s
c
r
i
b
e

t
o

f
r
e
e

C
M
G

p
u
b
l
i
c
a
t
i
o
n
,

M
e
a
s
u
r
e
I
T
(
t
m
)
,

a
t

w
w
w
.
c
m
g
.
o
r
g
/
s
u
b
s
c
r
i
b
e