AIX Service Strategy Details and Best Practices

IBM AIX Operating System Service Strategy Details and Best

Practices

Julie Craft
AIX Development
jcraft@us.ibm.com

06/10/2009 1
 AIX Service Strategy Details and Best Practices

Contents

Contents .................................................................................................................................. 2
Introduction............................................................................................................................. 3
Enhanced Service Strategy Releases ...................................................................................... 3
Contents of a Service Pack ..................................................................................................... 3
Changing the names of the service packs ............................................................................... 4
Changing the M in VRMF ...................................................................................................... 5
Technology Levels Must be Applied as a Group.................................................................... 5
Recommendation to apply the whole SP instead of just individual PTFs .............................. 6
Service Pack schedules ........................................................................................................... 6
Technology Level Lifecycle and Support............................................................................... 6
Interim Fix Guidelines/Policy for Maintenance ..................................................................... 8
Certified Interim Fixes............................................................................................................ 8
Security Fixes.......................................................................................................................... 9
APAR Numbers for each TL .................................................................................................. 9
New Automatic Interim Fix Removal Function ..................................................................... 9
Hardware Support with a Service Pack ................................................................................ 11
Moving to a New TL............................................................................................................. 11
Staying Current ..................................................................................................................... 12
Maximum Stability Model.................................................................................................... 12
Additional Information ......................................................................................................... 12
Summary ............................................................................................................................... 12

06/10/2009 2
 AIX Service Strategy Details and Best Practices

Introduction
IBM has enhanced the AIX operating system (OS) Release and Service Strategy as part of an
ongoing effort to improve the manageability and stability of the AIX operating system for our
clients. The enhanced strategy will provide clients with:
• Longer support for each AIX Technology Level update (formerly known as
Recommended Maintenance Levels)
• Improved serviceability for AIX throughout the life of each Technology Level
• Support for some new hardware on existing Technology Levels
The previous paragraph came from the IBM AIX Operating System Release Service Strategy
paper, which should be read before reading this paper.
This paper will go deeper into some of the changes, give some best practice scenarios and
discuss some of the more advanced function available.

Enhanced Service Strategy Releases

The AIX enhanced service strategy will start with AIX 5.3 TL6 and continue with AIX 6.1. AIX
5.2 will not use the new service strategy; TL10 is the last technology level available on 5.2 and
we will continue to provide service packs until the end of support.

Contents of a Service Pack

Service packs contain fixes for:
• Customer reported problems (APARs) that cannot wait until the next TL
• Critical problems found by development or test teams
• Very, very limited number of changes to support new hardware. Examples: A new
device driver, a new ODM entry to allow for configuration of a new class or type of
device, small changes in the kernel to recognize a new processor speed, etc.

The only changes that are allowed in an SP are limited to minimal corrections that do not change
behavior or add new functionality. The development team employs a fix rating system to
enforce this. New function, both for hardware exploitation and software features, is only shipped
in Technology Levels or new releases.

While AIX TL’s contain RSCT and CSM fileset updates, those filesets are not updated in the
AIX Service Packs. The reason for this is scheduling. The RSCT and CSM teams line up their
schedules to meet the AIX TL schedules, but do not do the same for Service Packs. The fixes
are placed into the next TL. Occasionally, usually with applications or products that rely more
heavily on CSM or RSCT, you may be required to download a fix directly from the RSCT or
CSM download sites. This situation may be more suited for your environment because it does
not require application of a specific AIX SP, only the individual fix for CSM or RSCT. We are
continuing to explore possibilities here to simplify the maintenance of AIX and its components.

06/10/2009 3
 AIX Service Strategy Details and Best Practices

Changing the names of the service packs

Starting with TL6, Service Packs will be labeled with their release date, using the YYWW
format, where:
YY = 2 digit year
WW = 2 digit week
An example: AIX 5.3 TL6 SP3 was released in 2007 and the 32nd week of the year, therefore:
5300-06-03-0732
' ' ' '---------------------YYWW
' ' '-----------------SP3
' '------------TL level
'--AIX release

The ‘oslevel –s’ command was updated to report this new name. The nice thing about this is not
only can you tell when the SP was released, but you will also know which SP to move to on a
new TL. When moving up to a new TL, you must move to a SP that is the same or later than
your current SP. The SP number itself (IE SP3) will not be the same, because the Service Packs
will be numbered consecutively as they are released, but the dates will tell you where you need
to be on the new TL. In general, it is recommended that you apply the latest Service Pack when
moving to a new TL.

New Release Strategy Service Pack Name Detail (AIX V5.3 shown)
2007 2008 2009 2010
Fall Spring Fall Spring Fall Spring

0812 0830 0845 0903

SP SP SP SP
HP … …

SP4 SP6 SP8 SP9

Technology

Level 7

0830 0845 0903

SP SP SP
… …
SP2 SP4 SP5
Technology

Level 8
Service pack names will be changed to reflect the release dates:

VR00-TL-SP-YYWW 5300-07-04-0812

We will line SP’s up so the upgrade path will be clear

*All statements regarding IBM's future direction and intent are subject
to change or withdrawal without notice, and represent goals and objectives only.
* Not showing all SP’s above and example dates are not necessarily accurate

Figure 1: Service Strategy Service Pack Name Detail

06/10/2009 4
 AIX Service Strategy Details and Best Practices

Applying individual PTF updates after a Service Pack will not be shown with the ‘oslevel –s’
command. Instead, use ‘oslevel –s –g 5300-06-03-0732’ to show what filesets are greater than
the specified service pack.
The installp command was changed, as were the updates themselves, to not allow a system to
apply any updates that are ‘younger’ than what is currently installed. This would cause
regression. When moving to a new TL, we recommend moving to the latest SP, that way you are
guaranteed it will pass the installp regression test.
Using the example above, if you are currently at TL7 SP8 (5300-07-08-0845), then you would
not be allowed to install TL8 SP2 (5300-08-02-0830), because 0845 was released after 0830,
and therefore will have fixes and hardware enablement that 0830 does not have.
The oslevel –s command will now print out the new SP format, but the other options will not
change.
# oslevel
5300
# oslevel –r
5300-06
# oslevel –s
5300-06-03-0736
The installp command will also stop an update entirely if it sees any updates that are older
(released before) in the list it is trying to apply. This is to make sure that part of a SP is not
installed, if that was not your intention. If you see a message from installp about not being able
to install a fileset update because of regression, then go to a newer SP (the latest) and try the
install again.

Changing the M in VRMF

Starting with AIX 5.3 TL7, any filesets that are updated will get a new ‘M’ in the VRMF. For
example, if the bos.rte.lvm fileset was updated, its update in TL7 would be 5.3.7.0 and the first
update, in SP1, will be 5.3.7.1. The fileset updates won’t necessarily correspond to the SP
because an update for fileset bos.rte.install might not come out until SP3, but it would be called
bos.rte.install 5.3.7.1. Likewise, if bos.rte.lvm had not changed since SP1, it would still be
bos.rte.lvm 5.3.7.1 in SP3.

Technology Levels Must be Applied as a Group

Technology Levels must be applied as a group, using the ‘smitty update_all’ or
‘install_all_updates’ commands. Installing a Technology Level is an “all or nothing” operation.
Initially, the plan was to add requisites to glue the TL together, but this was not done because of
the complications of circular requisites. But, installing a partial Technology Level will not be
recognized from a support standpoint.
Before applying a TL, you should always create a backup and plan on restoring that backup if
you need to rollback to your previous level. Or, use alt_disk_install or multibos as a way to get
back to your previous level. Technology Level updates should always be committed because
they cannot be rejected. Committing the updates saves space in the / and /usr file systems and
also makes it easier to track and reject service pack PTFs.

06/10/2009 5
 AIX Service Strategy Details and Best Practices

After the TL has been successfully applied and tested, another backup should be taken for
disaster recovery situations.

Recommendation to apply the whole SP instead of just individual PTFs

For planned maintenance windows, customers are encouraged to apply Service Packs as a group,
to simplify inventory and make it easier to report levels to service representatives and auditors.
Fix Central and SUMA will now download the entire SP for a specified APAR (search function).
This was also done to simplify the ordering process.
For more information on Fix Central changes, see the paper:
Fix Central changes supporting AIX Service Strategy at
http://www14.software.ibm.com/webapp/set2/sas/f/best/home.html.
Even though the entire SP is downloaded, individual fileset updates or APARs can still be
applied with SMIT (‘smitty install_by_fix’) or from the command line (installp or instfix).
There is nothing in the Service Pack that glues all the updates in a Service Pack together. While
most of the testing that occurs in the regression test lab is done as a group, applying individual
updates is still fully supported.
For unplanned application of fixes (like security or HIPERs), changing the minimum amount of
code is certainly the desired outcome.
For example:
• An interim fix was just released for a security issue and it requires an update from a
service pack.
• The service pack would require a reboot of the system whereas applying just the
individual PTF would not.
In these cases, just applying the individual APARs/PTFs may be the best solution. Service
Packs, because they will always require a reboot, are recommended for planned maintenance
windows or if the application of the fix will require a reboot anyway.

Service Pack schedules

Because there will be more Technology Levels shipping new fixes and because the Service
Packs will be lined up to come out at the same time (with the same YYWW), the SP release
schedules will be increased to approximately every 12 weeks. Occasionally, if a very critical
problem is found (high severity security, for example) and it cannot wait for the next SP, a
‘standalone PTF’ will be released and will be made available via APAR search on Fix Central.
The PTF will be included in the next Service Pack.

Technology Level Lifecycle and Support

With a valid Software Maintenance Agreement (SWMA), customers can continue to open PMRs
to ask questions and report problems with AIX 5.2, AIX 5.3 and AIX 6.1. However, if a code
defect is identified (i.e. an APAR is created) then the release and TL must be in the ‘New Fixes
Available’ period to request a fix. This applies to both interim fixes and Service Pack updates.
More information about the System p Product Lifecycle dates can be found at:
http://www.ibm.com/software/support/systemsp/lifecycle

06/10/2009 6
 AIX Service Strategy Details and Best Practices

The following chart illustrates the entire lifecycle of a technology level, showing both the times
where new fixes are delivered and when it is still supported via PMR for how to and usage
questions and problem determination.

Technology Level Lifecycle

New Fixes Available
Period
Support Period

“How to” Q&A, problem determination

New Fixes created (Interim Fixes
and Service Packs)

End of New Fixes

End of support for the AIX release
Available Period”
Availability
of the TL

Figure 2: Technology Level Lifecycle

The chart below is a listing of the technology levels that have been released in the last two years.
The chart highlights the differences in the timeframes of when fixes were available. AIX 5.2
TL8 and TL9 fell under the old service strategy and had a shorter time period of fixes, about one
year from release. The new strategy allows new fixes to become available for approximately two
years after release. Use this chart for comparison and planning.

Release Fixes Available

Release Date Until
AIX 5.2 TL8 Feb-06 Feb-07
AIX 5.2 TL9 Jul-06 Nov-07
AIX 5.2 TL10 Jun-07 4/30/2009 **
AIX 5.3 TL5 Aug-06 11/30/2007
AIX 5.3 TL6 Jun-07 5/30/2009
AIX 5.3 TL7 Nov-07 2 years later
AIX 5.3 TL8 April-08 2 years later
AIX 5.3 TL9 Nov-08 2 years later
AIX 5.3 TL10 May-09 2 years later
AIX 6.1 TL0 Nov-07 2 years later
AIX 6.1 TL1 April-08 2 years later
AIX 6.1 TL2 Nov-08 2 years later
AIX 6.1 TL3 May-09 2 years later

06/10/2009 7
 AIX Service Strategy Details and Best Practices

Figure 3: New Fixes Available – NOTE: All future dates are subject to change.
* New Fixes will be made available until two years later or when the new TL is offered. In some cases this may
mean slightly less than two years of support, in some cases it may mean slightly more. AIX will only make new
fixes available for up to four Technology Levels for each release at any given time.
** For 5.2, the date for ‘Fixes Available Until’ and End of Support are the same date.
When selecting a TL for upgrade, keep the ‘New Fixes Available’ period in mind. The
recommendation is to move to the latest TL and the latest SP when performing an upgrade, to get
the longest period of support.

Interim Fix Guidelines/Policy for Maintenance

Interim fixes are fixes made available to our customers to offer relief for a problem until the
customer is able to update to a Service Pack or Technology level that contains the fix. The
general rule is that a customer can request an interim fix for Technology Levels that are in the
‘New Fixes Available’ period; it will then be evaluated by development. Development will
make every attempt to supply an interim fix, however, there may be some that cannot be supplied
due to complexity.

Here are the guidelines that the Service team will be following:
• If a client is at the latest level and finds a problem – they can request an Interim Fix
• If a client finds a problem that is already in a later SP - they will be asked to install the
latest SP (or, at a minimum, the Service Pack that includes their fix or individual PTF that
contains the fix)
• If it is a critical problem and/or the client can't install a SP - an interim fix can be
requested, however it is recommend they move to the latest SP during their next
maintenance window
• If a client finds a new, unique problem at any level that is making new fixes available -
they can get an Interim Fix on their current level (if it's possible)
• If a client finds a new unique problem on a prior SP and has multiple Interim Fixes
installed, which are included in a later SP - they must move to the latest SP before getting
another Interim Fix
• If a client has multiple Interim Fixes and only some of them are in a later SP - they may
be asked to move to the latest SP and another Interim Fix will be bundled with their prior
Interim Fixes on the latest SP
• Clients should update to the SP that contains the fix as soon as they are able

Certified Interim Fixes

Interim fixes are offered to customers as a faster way to get temporary relief for a problem.
Interim fixes are shipped in ‘emgr’ format instead of installp format. Interim fixes are also
tracked via the ‘lslpp –L’ and ‘emgr –l’ commands.
Some interim fixes get additional functional and regression testing before they are released.
These interim fixes are usually intended for a wider audience than other interim fixes, which are
specific to a single customer.
Examples of certified Interim Fixes are security fixes that are released thru a vulnerability
advisory or subscription notification.
06/10/2009 8
 AIX Service Strategy Details and Best Practices

Security Fixes
Security fixes for security vulnerabilities are published thru advisories. Customers will continue
to be notified of security fixes by notices from IBM Subscription services at:
http://www14.software.ibm.com/webapp/set2/subscriptions/pqvcmjd
The security fixes published in the vulnerability advisories are posted here:
ftp://aix.software.ibm.com/aix/efixes/security
In the past, security interim fixes were offered for a longer period than other fixes to allow
customers time to transition to the current levels. But, because the enhanced service strategy
allows new fix availability for a longer period (even longer than the previous security fix policy
of 3 total TLs), security fixes will be made available for the same time period as all other new
fixes (see chart above). This offers consistency to the service strategy in regards to types of fixes
offered in the New Fixes Available period.
To assist our customers with the transition to AIX 5.3 Technology Levels that support the
enhanced service strategy, we will continue to generate 5.3 TL5 security interim fixes until 5.3
TL9 is shipped in 2H08 (approximately 10/2008). AIX 5.2 TL8 and AIX 5.2 TL9 support of
security interim fixes will continue until 6/2008. AIX 5.2 TL10 will continue to be supported
via Service Packs and interim fixes until end of support in April, 2009.

APAR Numbers for each TL

In the past, if a problem existed in two releases, only one APAR number was created for each
release. Starting with AIX 5.3 TL7, a new APAR number will be created for every Technology
Level where the problem is fixed. When PTFs were only made available on the current TL, only
one APAR number was required. Now that multiple service streams exist per release, an APAR
must be created for each one where a new fileset update (PTF) will be shipped.
If you’ve been given an APAR number for a particular problem, do a ‘Fix Search’ in Fix Central
to verify the APAR number with the Technology Level. The search will return results for the
other ‘sister’ APAR numbers, APAR numbers for the same release (e.g. 5.3) but different TLs,
and also ‘cousin’ APARs for other releases (e.g. 6.1). The search will also show availability of
the APAR and allow signup for a subscription notice if it has not been released.
For more information on searching for APAR and fixes, see the Fix Central paper at:
http://www14.software.ibm.com/webapp/set2/sas/f/best/home.html.

New Automatic Interim Fix Removal Function

Because of the issues with multiple APARs numbers in a release (APARs differ per TL as
described above) installp was not able to map the fixes it was shipping to the interim fixes that
were installed. Starting with 5.3 TL10 and 6.1 TL3, installp and emgr have been enhanced to
map the APARs and automatically remove interim fixes if the fix is present in the TL, SP or PTF
that installp is applying.
Interim fixes will now be packaged with a reference number. That number will allow installp to
map it back to APARs for all the Technology Levels where the fix was shipped.
When installp performs a preview install/update or an actual update, information will be
provided to list any interim fixes that will be automatically removed. Preview installs are always
highly recommended, even if you don’t currently have any interim fixes applied. But, if you do,

06/10/2009 9
 AIX Service Strategy Details and Best Practices

then installp will list any interim fixes that are installed and that will be automatically removed
or any interim fixes that are installed and cannot be removed because the corresponding fix was
not available in the updates you were installing. In this later case, the install will fail stating
there are filesets locked by emgr, like it has in the past.
For example, the information about interim fixes that will be removed will appear directly after
the build date verification in the installp (apply or preview) output:
+-----------------------------------------------------------------------------+
BUILDDATE Verification ...
+-----------------------------------------------------------------------------+
Verifying build dates...done

The updates being installed contain the following interim fix labels,
which will be removed prior to installing the updates:
IZ39706
IZ23965

NOTE: Rejecting an update that contains an interim fix will not

re-install the interim fix on the system.
……
Please note that if an interim fix is removed as part of the installp update, and then that update is
later removed (rejected), the interim fix that was previously installed will NOT be re-installed.
You would need to manually install the interim fix again at that point. So, always make sure you
have a copy of your interim fix, use alt_disk_install or create a mksysb image before the update
for recovery.
Because this is new function, a decision was made to not ship it in Service Packs on the ‘older’
Technology Levels. But, for this particular case, we will allow and support installing one
individual fileset from a TL. The bos.rte.install 5.3.10.0 update may be installed on 5.3 TL7,
TL8 or TL9 by itself, and the bos.rte.install 6.1.3.0 update may be installed on TL1 or TL2 by
itself. This will give you the new automatic interim fix removal function but will not require you
to go all the way to 5.3 TL10 or 6.1 TL3.
You may also download the bos.rte.install fileset update individually from Fix Central by
searching on bos.rte.install.
From the http://www-933.ibm.com/support/fixcentral/ main site, choose:
Product Group – System p, Product – AIX, Version – 5.3, Fix Type – Fix Search
http://www14.software.ibm.com/webapp/set2/fixsrch/FixSearch?release=53
Enter search items: fileset bos.rte.install
Then, in your search results you should select "Fileset information for: bos.rte.install".
You will be taken to all the bos.rte.install fixes. Choose the latest update for your release so
you’ll get any additional fixes or function included. All the updates are cumulative, so you can
apply to any level since GOLD. You will need to uncheck the download options of ‘Include
requisites’, ‘Include fixes that correct regressions’ and ‘Replace superseded fixes with the latest’.
If you leave these options checked, then the entire TL will be listed for download.

06/10/2009 10
 AIX Service Strategy Details and Best Practices

It is also recommended to watch the bos.rte.install fileset to pick up any fixes or new function.
You can always update the bos.rte.install fileset to a lever higher than your current TL, so if new
function ships in a later TL, you can obtain it without updating your entire system.

All interim fixes shipping will have a reference field that allows installp to map the APARs to
the interim fix. When the interim fix is installed (if you have the new bos.rte.install fileset
applied) then a message will be printed by emgr to alert you to the fact that automatic removal is
enabled:

+-----------------------------------------------------------------------------+
Processing APAR reference file
+-----------------------------------------------------------------------------+
ATTENTION: Interim fix is enabled for automatic removal by installp.

To verify that your interim fix has a reference number, you can check the contents of the
/usr/emgrdata/DBS/aparref.db file:

# cat /usr/emgrdata/DBS/aparref.db
IZ39706|:|220

This file contains a list of interim fix labels with their corresponding reference number values.
Later, we’ll update the emgr –l command to show this reference number. So, again, it’s a good
idea to get the latest level of bos.rte.install to make sure you have all the latest fixes and
enhancements.

The bos.rte.install 5.3.10.0 or bos.rte.install 6.1.3.0 or higher update must be installed in order to
get the automatic interim fix removal function. If this fileset update is installed along with other
PTFs, in a Technology Level, for example, it will automatically re-invoke itself and it will then
perform the mapping and automatic removal, if applicable.

Hardware Support with a Service Pack

When new hardware is released, the required TL’s or SP’s will be published in the release notes
and RFA (Release For Announce). If the TL is still in the ‘New Fixes Available’ period, then
the latest SP will have the hardware support.
If new boot media is required for an existing TL, then it will be made available for web
download (ISO image) via the PRPQ process.
If you currently have the TL in your environment, then updating to the required SP and creating
a mksysb backup media or image (NIM) will allow boot and install of the new hardware
systems.

Moving to a New TL
You should move to a new TL:
• If your existing TL is out or is about to go out of the New Fixes Available period

06/10/2009 11
 AIX Service Strategy Details and Best Practices

• You want to use new function and/or features in a new TL. Hardware exploitation, such
as large page space or new software function, such as multibos, will only be released in a
TL.
• You are going to test a new level for distribution into your production environment and
want to get the longest period of new fixes available. In this case, you should move to
the latest TL.

Staying Current
If you are currently running on a TL where new fixes are being made available, then any SP will
be supported for your interim fix needs. But, the risk in not updating to a service pack that
contains your fix during a planned maintenance window is that you will require another interim
fix to the same file. See the Interim Fix Guidelines for more information.
Updating twice a year to a new SP or TL is recommended to stay current.

Maximum Stability Model

Realizing that customers appreciated and relied on the CSP for planning their maintenance, and
realizing that having a specific SP to recommend to customers that want to maintain the most
stable environment and do not plan on moving up to a new SP before moving to a new TL, we
would recommend using the SP that roughly would be delivered in the same time frame as the
CSP. We would recommend planning around SP3.
This is not to say that other Service Packs in the TL are less stable than SP3, but because the TL
has been out a few months and any major issues have usually been identified by that point in
time, and there is still more than a year left for service, SP3 is a good one to target and our
recommendation for maximum stability. This recommendation is meant as guidance for
planning. Individual situations may require applying earlier or later SP’s, depending on
circumstances.

Additional Information
The IBM Service and Support Best Practices for UNIX® servers Web page at
http://www14.software.ibm.com/webapp/set2/sas/f/best includes additional information on the
AIX Release and Service Strategy, and best practices for managing the AIX and System p™
servers.
The Fix Level Recommendation Tool (FLRT) is a planning tool to help administrators
determine what key components of your System p server are at the minimum recommended fix
level. It can be found at: http://www14.software.ibm.com/webapp/set2/flrt/home.

Summary
The new AIX service strategy will offer customers more options for maintenance in their
environment. Two years of new fixes per TL will help simplify and reduce cost for maintenance.
Supporting new hardware on existing TL’s will reduce complexity in customer environments.

06/10/2009 12
 AIX Service Strategy Details and Best Practices

© IBM Corporation 2009

IBM Corporation

Marketing Communications

Systems Group

Route 100

Somers, New York 10589

Produced in the United States of America

June 2009

All Rights Reserved

This document was developed for products

and/or services offered in the United States.
IBM may not offer the products, features, or
services discussed in this document in other
countries.

The information may be subject to change

without notice. Consult your local IBM
business contact for information on the
products, features and services available in
your area.

All statements regarding IBM’s future

directions and intent are subject to change or
withdrawal without notice and represent goals
and objectives only.

IBM, the IBM logo, AIX, AIX 5L, POWER5+,

System p are trademarks or registered
trademarks of International Business
Machines Corporation in the United States or
other countries or both. A full list of U.S.
trademarks owned by IBM may be found at
http://www.ibm.com/legal/copytrade.shtml.

UNIX is a registered trademark of The Open

Group in the United States and other
countries.

Other company, product, and service names

may be trademarks or service marks of others.

Copying or downloading the images contained

in this document is expressly prohibited
without the written consent of IBM.

Information concerning non-IBM products was

obtained from the suppliers of these products
or other public sources. Questions on the
capabilities of the non-IBM products should be
addressed with the suppliers.

The IBM home page on the Internet can be

found at http://www.ibm.com.

The IBM System p home page on the Internet

can be found at
http://www.ibm.com/systems/p.

The IBM AIX home page on the Internet can

be found at: http:///www.ibm.com/servers/aix.

PSW03011-USEN-00

06/10/2009 13