You are on page 1of 65

Sevilay Kurt, Technical Leader of IBM Turkey, Senior Certified Architect, sevilay@tr.ibm.

com

Overview of Mainframe
Operating System Facilities - II
Data Management

Agenda

Data Management
Data management
Catalog processing
DASD volume organization
Allocating new non-SMS DASD data sets
SMS Facilities
Data sets and files
Data Set Support Functions

Data Management

When the environment is being prepared for a program's execution, the JCL provides data set and device allocation
information. For uncataloged data sets, the JCL will contain not only the data set name and disposition, it will also have the
device type and volume name which contain the data set. For cataloged data sets, only the data set name and disposition
are required. The catalog will be searched for data set location information. When creating new data sets, the JCL may be
used to provide space information or SMS class information.
3

Data Management

Job Entry Subsystem (JES) receives the JCL and stages it to the spool queue. JES then converts the JCL into internal text
while waiting for an address space (initiator) to become available. The converter/interpreter text is then pasted to the initiator
address space and is interpreted by the initial task which starts the allocation process.
Allocation takes place after the initiator has selected a job for processing and after the interpreter has built the appropriate
internal work areas and control blocks in the scheduler work area (SWA) from parameters coded in the JCL.
4

Once the program starts executing and completes, de-allocation takes place before termination.

Data Management

Allocation must determine the physical location of the data referenced on the DD statement. This can be thought of as a twostep process:
1. Determine which volume.
2. Determine where on the volume.

The first step is to determine on which volume the data is located. The z/OS catalog will be used to determine the volume
information unless the DD statement provides that information with the VOL=SER or VOL=REF parameters. The second step
is to determine where the data is located on the volume. This occurs when the data set is opened. Directories on the volume
such as the Volume Table of Contents (VTOC), the Indexed VTOC, or the VSAM Volume Data Set (VVDS) are used to locate
the data set.

For an uncataloged data set, the JCL must include the data set name, the disposition, and the appropriate UNIT= and
VOL=SER= parameters. The system will locate the volume and then do data set location using the VTOC. IBM
recommends you always use cataloged data sets.
For a cataloged data set, the JCL needs to include only the data set name and disposition. The data set name will direct
z/OS to the appropriate catalog. The system does volume location using the data set name and the catalog. Once the
volume is located then the process is the same.
6

Data Management

The master catalog should contain only system-related entries. The master catalog contains pointers to system data sets and
user catalogs.
User catalogs contain pointers to user VSAM data sets and user non-VSAM data sets.
The master catalog is required by z/OS. You should always have an alternate master catalog available just in case something
happens to the current master catalog, because z/OS will not function without it. The master catalog contains a pointer to the
system data sets, SYS1.xxx datasets, a pointer to the user catalogs, and aliases which point to user catalogs. The user
catalogs contain pointers to user data sets. Only Integrated Catalog Facility (ICF) catalogs are supported for SMS-managed
data sets.
7

Data Management

Catalog functions are performed in the catalog address space (CAS). Most catalog modules and control blocks are located
in the catalog address space above 16MB. This reduces the required virtual storage in a user's private area needed to
perform catalog functions.
The CAS is also used as in-storage cache for the catalog records of open ICF catalogs. This cached information is used to
speed up the data set locate process.
During the initialization of a z/OS system, all user catalog names identified in the master catalog, their aliases, and their
associated volume serial numbers are placed in tables in CAS.
Changes to the master catalog are automatically reflected in the CAS tables. The information in the master catalog is
normally the same as the information in CAS. For shared catalogs, the catalog address spaces on all the sharing systems
are updated, maintaining data integrity for your systems.

Data Management

In Storage Cache (ISC)


- Default catalog cache
- Records cached in CAS
- Fixed amount of storage for each user catalog
- When change for a BCS is detected, entire ISC for
that BCS is released
- All catalogs are eligible unless in CDSC or explicitly
removed from ISC
- At first catalog open, a chunk of storage is
acquired from CAS private
- This chunk is fixed, and records are discarded
when the chunk is full
Catalog Data Space Cache (CDSC)
- Exploits VLF to maintain data space for storing
catalog records
- To enable, update COFVLFxx member
- Catalog must be put explicitly in CDSC
- Records cached in VLF-managed data space
- Catalogs not limited to set amount of storage
- When data space is full, least recently used record
is removed
- When change is detected, only the changed
record(s) is released

DASD I/O operations can be reduced by staging more catalog records in data spaces.

Data Management

Specifying CDSC In SYS1.PARMLIB(COFVLFxx):


CLASS NAME(IGGCAS)
EMAJ(BCS1)
EMAJ(BCS2)
MAXVIRT(4096|nnnn)

The site can selectively identify ICF catalogs for data space caching. After a locate is performed for a data set, the associated
catalog record is given to the Virtual Lookaside Facility (VLF) via the COFCREAT macro. Subsequent locates for that data set will
cause the catalog record to be retrieved from the VLF-managed data space.
An installation can selectively identify ICF catalogs for data space caching using the COFVLFxx member of SYS1.PARMLIB.
The catalog records in the data space are valid for as long as VLF is active. For catalog updates, only the affected records are
invalidated. If VLF becomes inactive, the CAS caching takes over.

10

Data Management
A catalog consists of two separate kinds of data sets: a basic catalog structure (BCS); and a VSAM
volume data set (VVDS). The BCS can be considered the catalog, whereas the VVDS can be
considered an extension of the volume table of contents (VTOC).
The basic catalog structure is a VSAM key-sequenced data set. It uses the data set name of
entries to store and retrieve data set information.
The VSAM volume data set is a VSAM entry-sequenced data set. A VVDS resides on every volume
that contains a catalog or an SMS-managed data set that is cataloged. It contains the data set
characteristics, extent information, and the volume-related information of the VSAM data sets
cataloged in the BCS.

11

Data Management

An alias is an alternate name for a user catalog. For example, PAY is an alias for the user catalog called UCAT1.
Alias names are defined in the master catalog and are directly related to the real name of the user catalog. For example,
UCAT1 is the real name of the catalog which has an alias of PAY.
The high-level qualifier (HLQ) in the data set name normally matches an alias for the user catalog where the data set will be
cataloged. For example, PAY.ABC is cataloged in UCAT1 since it has an HLQ of PAY.
Aliases are identifiers with pointers to the proper user catalog which contains a fully qualified data set name and which
volume it is on. Aliases are placed in the master catalog to revenue the master catalog from having entries for all of the user
data sets.
12

Data Management

Multilevel Alias (MLA) allows an alias for a user catalog to be defined with up to four levels of qualifiers. Segmenting similarly
named data sets into different catalogs enables an installation to have more flexibility in data set management.
MLA reduces catalog contention by spreading the data set across multiple user catalogs. The SYSCATxx member of
SYS1.NUCLEUS or the LOADxx member of SYS1.PARMLIB must be updated to indicate the number of levels of aliases your
installation will use.
13

Data Management

The order in which the catalogs are searched to find a data set is:
1. Any user catalog specifically identified by a JOBCAT or STEPCAT JCL statement.
2. The master catalog is searched for a match between the HLQ of the data set name and either the alias name or the exact
name of the user catalog.
3. The master catalog is searched for the fully qualified name of the data set.
The system searches the master catalog for an alias. If one is found, it will go to the user catalog specified and search for the fully
qualified data set name. If one is not located, the system will search the master catalog for the fully qualified data set name.
14

Data Management

All volumes contain a volume label and a Volume Table of Contents (VTOC).
The Volume Table of Contents and the VTOC index are system data sets that maintain extent and allocation information for a
volume. The VTOC is used to find empty space for new allocations and to locate non-VSAM data sets. For all VSAM data sets,
and for SMS-managed non-VSAM data sets, the VTOC is used to obtain information not kept in the VVDS.
The volume label is made up of a volume serial number (VOLSER) and a pointer to the VTOC. A VOLSER is used to identify
each volume in the system.
The VSAM Volume Data Set (VVDS) is a component of an ICF catalog. The VVDS describes the VSAM data sets and the
system-managed non-VSAM data sets on the volume.
15

Data Management

The VTOC is a data set that describes the contents of the direct access volume on which it resides. It is a contiguous data set that
resides in a single extent anywhere on the volume. Its location is determined via the volume label. The VTOC is composed of Data
Set Control Blocks (DSCBs) that describe the contents of the volume. The DSCBs can be of different types:
Format-0 Describes an unused record in the VTOC. This DSCB is used to mark a data set as deleted in the VTOC.
Format-1 Describes the first three extents of a data set.
Format-2 Describes the indices of an ISAM data set.
Format-3 Describes additional extents of a data set.
Format-4 Describes the VTOC.
Format-5 Describes the space on a volume that has not been allocated (free space).
There are several types of DSCBs. The main ones used in data set handling, are: format-0, describes unused space; format-1,
describes the data set; and format-5, describes free space. The VTOC is described by a format-4 DSCB.
There is also another DSCB type (not mentioned in student notes): format 7 DSCB, which is only used by some devices, to keep
track
of free space - like format 5 DSCBs,
16
In general. A VTOC consists of complete tracks and can be expanded without re-initializing the volume.

Data Management

The VSAM Volume Data Set (VVDS) is a VSAM entry-sequenced data set. A VVDS resides on every volume which contains a
VSAM data set or a SMS-managed data set, that is cataloged in an ICF catalog.
The VVDS contains the data set characteristics, extent information, and the volume-related information of the VSAM data sets
cataloged in the BCS. If using SMS, the VVDS also contains data set characteristics and volume-related information for the
non-VSAM, SMS-managed data sets on the volume.
VVDS records for VSAM data sets are called VSAM volume records (VVRs). Those for SMS-managed non-VSAM data sets
are called non-VSAM volume records (NVRs). Since a BCS is a VSAM data set, it also has an entry in the VVDS. A BCS does
not own a VVDS: more than one BCS may contain entries for a single VVDS. Every VVDS which is connected to a BCS has
an entry in the BCS.
All SMS-managed volumes will have a VVDS. VSAM will create a VVDS on a volume when a VSAM data set is created on
that volume. You may also have non-VSAM data sets identified in the VVDS if they are cataloged in an ICF catalog.

17

Data Management

A volume may be initialized with an indexed VTOC.


The indexed VTOC provides improved performance, primarily when z/OS is allocating new data sets. There is one index
entry per data set on the volume. The index entry points directly to the data set entry in the VTOC.
An indexed VTOC will allow a tree search to be done when processing the VTOC. A normal VTOC process would be a
sequential search without an indexed VTOC.
18
All

SMS volumes will have an indexed VTOC.

Data Management

The user may request that the data set be allocated on a specific device with the UNIT= and VOL=SER= or VOL=REF=
parameters in the JCL.
Alternatively, the user may include only the UNIT= parameter in the JCL and let z/OS choose the volume on which the data
set will be allocated.
In a non-SMS environment you must supply the system with all the information needed to allocate the data set, including
data set characteristics and device information.
19

Data Management

Each time a user submits a job, the UNIT= parameter on DD statements identifies the I/O devices that the job requires. Before the
job can continue execution, z/OS must allocate all of those devices to the job.
If the user requests a particular device (type), by specifying a specific device type on the UNIT= parameter, and that device is
available, z/OS allocates the device to the job. If the device is unavailable, however (it's allocated to another job, for example), the
user's job will have to wait until the device becomes available for allocation.
By requesting a specific device (number), the user limits the choice of devices to one. By doing so, the user increases the probability
that the requested resource the job needs will be unavailable when needed (for example, UNIT=123 or UNIT=/3390).
The user can increase the probability that a device will be available for allocation when needed by requesting a generic device type.
z/OS logically groups device types with similar characteristics and assigns the group a generic name (for example, UNIT=3390).
Device number: Specifies the three or four-digit hexadecimal device number. Precede a four-digit number with a slash (/). A threedigit number can be specified with or without a slash.
20
So, UNIT=3390 is not a device number, but UNIT=/3390 is a device number.

Data Management

An esoteric device group is an installation-defined and named collection of I/O devices.


When a user submits a job, the job can identify a device that it needs by specifying an esoteric device group instead of a
specific device number or a generic device type. A job that specifies an esoteric device group is requesting z/OS to allocate
a device from that group.
Users can specify esoteric device names on DD statements for input and output data sets. z/OS uses the eligible device
table (EDT) to select devices in response to allocation requests. The order in which devices are placed in esoteric device
groups controls the order in which z/OS allocates the devices.
An esoteric device group can include devices of more than one generic device type. To define esoteric device groups and, in
21
general, an I/O configuration to a z/OS system you have to use the Hardware Configuration Definition (HCD) product.

Data Management

If this is a non-specific allocation (no VOL=SER= or VOL=REF= Specified), z/OS allocation routines use a combination of JCL, the
EDT, and the Volume Attribute List (VATLSTxx member of SYS1.PARMLIB) to determine which device should be allocated for a
new data set. The list of candidate volumes is then passed to SRM. SRM selects the optimum volume.
VATLSTxx contains one or more volume attribute lists that predefine the mount and use attributes of direct access volumes. The
mount attribute determines the conditions under which a volume can be demounted. The use attribute controls the type of requests
for which a volume can be allocated.
The system programmer creates the EDT by providing UNITNAME macros as input to HCD. The UNIT= parameter specified in the
JCL is checked against the EDT to determine which volumes are eligible for allocation to this request. The system programmer
also defines volume attributes for the DASD devices via the VATLSTxx member of SYS1.PARMLIB. Volumes can be designated as
PRIVATE, PUBLIC, or STORAGE. Only volumes that match the characteristics of the request will be eligible for allocation. The
system resource manager makes the final decision as to which volume to use.
22

Data Management

Direct Access Device Space Management (DADSM) controls the allocation of space on a volume.
DADSM uses the VTOC to locate space on a volume. When DADSM allocates space on a volume, it updates the VTOC to
point to the new data set located on that volume. When DADSM deletes a data set from a volume, it updates the VTOC to
remove the data set pointer and add pointers to the additional free space.
The direct access space management program is used to create new data sets, extend existing data sets, and delete data
sets from a volume. DADSM is responsible for updating the VTOC with new DSCBs or information in existing DSCBs.
23

Data Management

During allocation, the Storage Management Subsystem (SMS) makes many decisions about the new data set that were
previously made by the end user through JCL. As a result, the JCL to allocate a new data set is much simpler in an SMS
environment.
SMS makes its decisions about the new data set based on input from the storage administrator.
SMS chooses the volume for allocation based on the space, availability, and performance requirements of the data set.
SMS runs in a system address space of its own.
With SMS set up and active, the end user only has to specify the data set name and the DISP= information. SMS will decide
what type of data set, what characteristics the data set will have, and where the data set will be placed. SRM will still make
the final decision about which volume to use based on utilization.
24

Data Management

The SMS manages the data by using centralized policies that define how the hardware resources should be used.

25

Data Management

26

Data Management
Data processing users face numerous decisions every day in dealing with their data. In addition to dealing
with issues which are unique to the data or application, they need to be aware of their installation's storage
management policies. They must also deal with issues related to the format, handling, and placement of
their data:
What should the block size be? How much space will be needed?
What type of device should I use? Should the data be cached? What about recovery?
How often should it be backed up? Migrated? Should it be kept? Deleted?
What volumes are available for data set placement?
If the use of data processing services is to be simplified and made available to everyone, then simpler
interfaces need to be provided to the system. In particular, JCL is one area where simplifications are being
made.
In a z/OS system, space management involves the allocation, placement, monitoring, migration, backup,
recall, recovery, and deletion of data sets. These activities can be done either manually or through the use
of automated processes. When data management is automated, the operating system determines object
placement, and automatically manages data set backup, movement, space, and security.
Data management includes the following main tasks:
Sets aside (allocates) space on DASD volumes
Automatically retrieves cataloged data sets by name
Mounts magnetic tape volumes in the drive
Establishes a logical connection between the application program and the medium
Controls access to data
Transfers data between the application program and the medium
27

Data Management

The goal of system-managed storage is to enable any data set to be accessed by specifying nothing more than the data set name
and disposition.
SMS design was dictated by two key objectives:
- To provide centralized control of external storage. This implies that many data management decisions can be automated via storage
management policies.
- To remove device awareness from users. Simpler JCL coding is a consequence of this second objective.
SMS enables an installation to implement storage management policies via four classes of predefined attributes called constructs:
- Data Class (DC): Contains allocation attributes that describe the logical data format.
- Storage Class (SC): Contains desired performance and availability objectives. SMS uses this criteria to determine where to place a
data set.
- Management Class (MC): Provides criteria that DFSMShsm uses in data set migration and backup.
28
- SG: Defines a list of volumes for data set allocation.

Data Management
Default DD statement parameters (such as LRECL, SPACE, and RECFM) can be predefined in the data
class. Constructs also contain non-JCL information (such as how often to back up a data set or the desired
response time). These attributes are inherited by a data set or volume when the constructs are assigned to
the data set or volume.
Even if an installation has SMS installed, not all data sets will be managed by SMS. Some volumes may be
SMS-managed while others will not be. However, SMS-managed data sets always reside on SMS-managed
volumes, and non-SMS-managed data sets always reside on non-SMS-managed volumes.
The heart of DFSMS is the SMS. Using SMS, the system programmer or storage administrator defines
policies that automate the management of storage and hardware devices. These policies describe data
allocation characteristics, performance and availability goals, backup and retention requirements, and
storage requirements for the system. SMS governs these policies for the system and the Interactive Storage
Management Facility (ISMF) provides the user interface for defining and maintaining the policies.

29

Data Management

Most installations have data sets with many different formats and a multitude of performance, availability, backup, and migration
requirements. Thus, numerous constructs of each type need to be defined. Automatic Class Selection (ACS) routines are used
by SMS to select and assign the proper constructs to a data set. The data set then inherits the attributes from the assigned
constructs.
SMS gets control during the creation of a data set to determine if the data set is eligible to be managed by SMS. If a new data
set is eligible, then the ACS routines receive control in the order shown. A maximum of one DC, one SC, and one MC can be
assigned by the corresponding ACS routines.
After the ACS routines complete the construct assignments, the z/OS Scheduler Allocation then gets control to complete the
data set allocation.
SMS receives control not only at data set creation, but also at numerous other times during the life of a data set. SMS
examines
constructs in managing allocation requests, data set placement, backup and retention, and to check performance and
30
availability requirements.

Data Management
On systems that do not use DFSMS, storage management consists mostly of manual operations performed
on individual data sets, and manual and automated operations performed on volumes. With SMS, you can
automate storage management for individual data sets and objects, and for DASD, optical, and tape
volumes. You use SMS classes and groups to define the goals and requirements that the system should
meet for a data set or object.
You use:
DC to define model allocation characteristics for data sets
SC to define performance and availability goals
MC to define backup and retention requirements
SG to create logical groupings of volumes to be managed as a unit
Certain data sets are not eligible to be SMS-managed. For example, a data set must be cataloged in an
ICF catalog to be eligible. Also to be eligible, a data set should reside on a pack containing an indexed
VTOC.
(Some of these constructs might not be assigned. For example, an MC is not assigned to temporary data
sets. An SC is not assigned to non-SMS-managed data sets.) However, up to fifteen SG constructs can be
assigned by the SG ACS routine.

31

Data Management

SMS provides nine DD statement parameters that can be used to define new data sets. If SMS is active, these parameters
can simplify JCL DD statement coding. That is, several old JCL DD parameters can be replaced by fewer parameters from the
list above. However,
if SMS is inactive, all of these parameters will be ignored.
DATACLAS, STORCLAS, and MGMTCLAS are DD statement parameters which are used to assign a DC, SC, and MC
respectively. (Therefore, the DC, SC, and MC can be assigned via ACS routines as well as JCL!)
STORCLAS and MGMTCLAS, apply only to SMS-managed data sets, whereas DATACLAS and the remaining parameters
apply to both SMS-managed and non-SMS-managed data sets. RECORG and KEYOFF, apply only to VSAM data sets. The
LIKE and REFDD parameters can be coded when creating a new data set to specify the allocation attributes to assign. LIKE
tells the system to use the allocation attributes from an existing cataloged data set. REFDD tells the system to use the
allocation attributes from a data set defined earlier in the job.
32
SECMODEL specifies a model RACF profile which RACF uses to create a discrete profile to protect the data set.

Data Management

A data class is a named list (for example, A, B, C) of data set allocation and space attributes that SMS assigns to a data set
when it is created.
The storage administrator creates different data class definitions to match the various types of data sets used at your
installation.
Both SMS-managed and non-SMS-managed data sets can have a data class assigned to them. Users can override attributes
in the data class definition by explicitly coding the appropriate JCL parameters. The data class provides data characteristic
information such as block size, LRECL, RECF, and others. There can be data classes for any type of data set, PDSEs, VSAM,
and so on.
33

Data Management

ACS routines are written by the system programmer or storage administrator using the class selection programming
language.
A separate ACS routine is used to assign a DC, an SC, an MC, and an SG to a data set.
The input variables for ACS routines can consist of:
Data set information (for example, HLQ, LLQ, size, retention)
Job and step information (for example, job name, program name)
RACF information
Environmental information (for example, interactive, batch)
The ACS routines determine what specific classes should be used in the creation of the new data sets by using information
passed at request time in the JCL. The information includes such things as data set name, user type, RACF information, and
job or step details. The ACS routines are coded by the storage administrators.
34

Data Management

Since the DC can be assigned via the DATACLAS keyword, then z/OS users need to have a method of listing the data classes
along with their attribute values. The Interactive Storage Management Facility (ISMF) provides this capability.
ISMF generated the screen above. This screen displays the data classes and summarizes their attribute values. Thus, z/OS
users can quickly scan the list to see what data classes are available.
DC attributes are assigned to a data set when it is created. DC attributes apply to both DFSMS-managed and non-DFSMSmanaged data sets. Attributes specified in JCL or equivalent allocation statements override those specified in a DC. Individual
attributes in a DC can be overridden by JCL, TSO, IDCAMS, and dynamic allocation statements.
35

Data Management

In order for a data set to be SMS-managed, the data set must satisfy two criteria:
- The data set must be eligible to be SMS-managed.
- The data set must have an SC assigned.
The categories of data sets above are eligible to be managed by SMS, but not all data sets are eligible. The following categories are not eligible:
Data sets not cataloged in an ICF catalog
ISAM data sets
SYSOUT data sets
In-stream data sets
Unmovable data sets
Data sets allocated with absolute track allocation
Data sets on a mass storage (MSS) volume
GDG Model DSCBs
VSAM spaces
VSAM catalogs or CVOLS
Tape
36 data sets are sequential. Thus, they are eligible to be SMS-managed.

Data Management

An SC is assigned to a data set to identify its performance and availability requirements.


An MC is assigned to a data set to control how the data set will be managed once it has been placed on a volume. The
DFSMShsm Hierarchical Storage Manager (DFHSM) will automatically manage the data set.
The SC provides performance characteristics such as cache and response time. It can also provide availability information
such as dual copy. The MC provides backup and migration information.
37

Data Management

The SC provides criteria that SMS uses in determining where to place a data set. Desired response time, the read/write bias
(whether reads or writes are predominant), the importance of the data set (with respect to availability), and attributes relating
to optical volumes and multivolume data sets are specified via the SC.
The SC can be assigned by the SC ACS routine, the STORCLAS DD parameter, and by other methods (ISPF/PDF, TSO/E,
and IDCAMS are examples). The assignment of the storage class, S1, can be requested via the STORCLAS parameter:
//N DD DSN=X.Y,DISP=(,CATLG),STORCLAS=S1
However, the storage administrator can choose to honor this request or allow the SC ACS routine to override the request and
make the assignment.
An MC can be requested by coding:
MGMTCLAS=M1
The only way to assign an SG is to allow the SG ACS routine to assign it. SMS then selects the volume from this group.
38

Data Management

The DASD volumes in your installation are divided into SGs based on the category of data that will reside on the volumes
within an SG.
SMS will assign an SG to your new data set through an ACS routine. The new data set will be allocated on a volume from its
assigned SG.
All volumes in an SG must have the same device geometry (same bytes per track and tracks per cylinder), but all volumes
within an SG might not have the exact same characteristics (cache versus non-cached). You may also group by same or like
data for the same department or same group.

39

Data Management

An SG has been chosen for the data set via an ACS routine, but SMS must still choose which volume within the SG will be
used for allocation.
These are the steps SMS follows to choose a volume for allocation from within the SG:
1. SMS selects the volumes from the chosen SG which have enough space to satisfy the primary allocation. (The
DC for this data set supplies the space requirements.)
2. SMS selects the volumes from the chosen SG which are able to satisfy the data set's performance and
availability requirements. (The SC for this data set supplies the performance and availability requirements.)
The first place that will provide allocation information will be the JCL. It can have all allocation information or some part of
allocation information or class specifications. Next, SMS will execute the ACS routines if not everything was passed in the JCL.
The final list of candidate allocation volumes is passed to SRM which selects the optimum volume. DFSMS also supports more
40
than 32 systems.

Data Management

z/OS works with two completely different ways of handling data on disk storage: traditional MVS data sets and UNIX files.
Traditional MVS data sets are accessed using one of two general methods, either the VSAM or non-VSAM access method.
Access methods are used to define the organization of the data in a data set and the technique by which the data is stored and
retrieved. MVS data sets are collections of records. Each record has a logical data structure defined in the processing program.
Record formats can be fixed (F), fix-blocked (FB), variable (V), variable-blocked (VB), or undefined (U). Each data set is given a
data set name, which can be a maximum of 44 characters long. The data set's location and some of its attributes are stored in a
catalog structure. Security is related to the data set name.
The introduction of UNIX services to the OS/390 operating system required support for UNIX files. UNIX files are byte-oriented.
They are stored in a hierarchical form, that is, directory, subdirectory, and filename. Filenames as such have no relation to
security.
41

Data Management

VSAM data sets are different from non-VSAM data sets in that records are stored in control intervals. The control intervals in a
VSAM data set are grouped into control areas.
Whenever a record is retrieved from direct access storage, the entire control interval containing the record is read into a VSAM
I/O buffer in virtual storage. The desired record is then transferred from the VSAM buffer to a user-defined buffer or work area.
VSAM data sets can be:
Entry-sequenced Data Set (ESDS): Similar to a sequential (non-VSAM) data set.
Key-sequenced Data Set (KSDS): Logical records are placed in the data set in ascending collating sequence by a field, called
the key.
Linear Data Set (LDS): A VSAM data set which does not have imbedded control information.
Fixed-length Relative Record Data Set (RRDS): Consists of several slots of equal length. A variable-length RRDS is similar to a
42
fixed-length
RRDS, except that it contains variable-length records.

Data Management
ESDS: An entry-sequenced data set is similar to a physical sequential (non-VSAM) data set. Records are
sequenced by their entry into the data set. Records are only added at the end of the data set. Existing
records cannot be deleted. If you want to delete a record, you must flag that record as inactive.
KSDS: In a key-sequenced data set, logical records are placed in the data set in ascending collating
sequence by a field, called the key. The key contains a unique value, such as an employee number or part
number, which determines the record's collating position in the data set. The key must be in the same
position in each record, the key data must be contiguous, and each record's key must be unique. The value
of the key cannot be altered, but the entire record can be erased or deleted. When a new record is added to
the data set, it is inserted in collating sequence by key. KSDSs consist of an index and a data part.
LDS: A linear data set is a VSAM data set having a control interval size from 4096 to 32768 bytes, in 4096byte increments. An LDS does not have imbedded control information. It is processed as an ESDS, with
certain restrictions. Since an LDS does not contain control information, there are no individual records to
access.
RRDS: A fixed-length relative record data set consists of several slots of equal length. Each slot has a
unique relative record number, and are sequenced by ascending relative record number. Each record
occupies a slot, and is stored and retrieved by the relative record number of that slot. The position of a data
record is fixed; its relative record number cannot change. Because the slot can either contain data or be
empty, a data record can be inserted or deleted without affecting the position of the other data records. Each
control interval contains the same number of slots determined by the control interval size and
record length.
A variable-length RRDS is similar to a fixed-length RRDS, except that it contains variable-length records.
Each record has a unique relative record number, and is placed in ascending relative record number order.
Each record is stored and retrieved using its relative record number. Unlike a fixed-length RRDS, a variablelength RRDS does not have slots. The relative record number of a record cannot change. When that record
is erased, the relative record number can be reused for a new record.

43

Data Management

Sequential data set: In a sequential data set, also referred to as physical sequential (PS), records are arranged
sequentially in the order in which they are written.
Partitioned data set (PDS): PDS is divided into sequentially organized members, each of which can have one or more
records. Each member has a unique name, stored in a directory that is part of the data set. The directory is located at the
beginning of the data set and contains an entry for each member. Each directory entry contains the member name and the
starting location of the member within the data set.
Partitioned data set extended (PDSE): In appearance, a PDSE is very similar to a PDS. For accessing a PDS directory or
member, most PDSE interfaces are indistinguishable from PDS data set interfaces. However, PDSEs have a different
internal format, which gives them increased usability.
44

Data Management
The terms primary space allocation and secondary space allocation: when you allocate data sets, z/OS first
allocates the amount of primary space defined for a data set. Once this primary space is used, it
automatically allocates additional space, as defined by the secondary space specification:
A physical sequential data set can have 16 extents on each volume.
An extended sequential data set can have 123 extents per volume.
A PDS can have 16 extents.
A direct data set can have 16 extents on each volume.
A PDSE can have 123 extents.
For sequential data sets: New records are appended at the end of the data set. In sequential data sets, data
is organized for sequential access. For example, to retrieve the fifth record of the data set, the system first
has to read the preceding four. Sequential data sets can be stored on DASD or on tape. They are required
for the use of magnetic tape devices or printed output.
For PDS: The directory entries are arranged by name in alphanumeric collating sequence. The main
advantage of using a PDS is that, without searching the entire data set, you can retrieve any individual
member after the data set is opened. Individual members can be added or deleted as required. A deleted
member is removed from the directory. Its space cannot be reused until the data set is reorganized using a
system utility.
For PDSE: The main advantage of using a PDSE over a PDS is that PDSEs use DASD space much more
efficiently. The size of a PDS directory is fixed regardless of the number of members in it, while the size of a
PDSE directory is flexible and expands to fit the members stored in it. Also, the system reclaims space
automatically whenever a member is deleted or replaced, and returns it to the pool of space available for
allocation to other members of the same PDSE. The space can be reused without having to reorganize the
data set.
45

Data Management

DFSMShsm is an optional feature for z/OS.


DFSMShsm:
Automatically performs space management and availability management in a storage device hierarchy.
Makes sure space is available on DASD volumes so that old data sets can be extended and new data
sets can be allocated.
Makes sure that backup copies of data sets are always available if the working copies are lost or
corrupted.
46

Data Management
DFSMShsm provides functions for:
Storage management: DFSMShsm uses a hierarchy of storage devices in its automatic management of
data, relieving end-users from manual storage management tasks.
Space management: DFSMShsm improves DASD space usage by keeping only active data on fast-access
storage devices.
Tape mount management: DFSMShsm can write multiple output data sets to a single tape, making it a
useful tool for implementing the tape mount management methodology under SMS.
DFSMShsm automatic backup, migration, and dump support SMS 32-name support.
DFSMShsm backup and ABARS support VSAM data sets that are open in RLS mode.
Support includes the following features:
Data sets can be backed up directly to tape
Multitasking for the data set backup function
Concurrent copy support
Recover tape takeaway support
A single z/OS image allows multiple instances of DFSMShsm

47

Data Management

DFSMSdss is an optional feature in z/OS.


DFSMSdss is a DASD data and space management tool. DFSMSdss works on DASD volumes only in the
z/OS environment.
DFSMSdss can be used to:
Copy and move data sets between volumes of like and unlike device types
Dump and restore data sets, entire volumes, or specific tracks
Convert data sets and volumes to and from SMS management
Compress partitioned data sets
Release unused space in data sets
48
Reduce or eliminate DASD free-space fragmentation by consolidating free space on a volume

Data Management
Stand-alone restore
Stand-alone restore function is a single-purpose program designed to allow the system programmer to
restore vital system packs during disaster recovery without needing to rely on a z/OS environment. Standalone services runs independently of a system environment either as a true stand-alone system or under a
VM system. Stand-alone services can perform either a full-volume or a tracks restore from dump tapes
produced by DFSMSdss or DFDSS, and offers the following benefits when compared to the existing standalone functions:
- Provides user-friendly commands to replace the existing control statements
- Supports IBM 3494 and 3495 tape libraries
- Supports IPLing from a DASD volume, in addition to tape and card readers
- Allows you to predefine the operator console to be used during stand-alone services processing
UTILMSG=ERROR
DFDSS invokes utility programs (IDCAMS, IEBCOPY, IEBISAM, IEHMOVE, and ICKDSF) to perform
requested DFDSS functions. DFSMSdss uses the UTILMSG=keyword parameter in the parm field of the
EXEC statement to control the
disposition of the utilities' SYSPRINT data sets.
UTILMSG=ERROR parameter causes error messages from the utility programs that DFSMSdss invokes to
be copied into the SYSPRINT output only if the utility program returns an error return code to DFSMSdss.
UTILMSG=ERROR is the default.
VSAM RLS support
In support of VSAM RLS, DFSMSdss insures that data is consistent between DASD and the coupling
facility. It also is responsible for recording the time of backup of a CICS data set.

49

Data Management

The DFSMSrmm element is an optional feature for both z/OS. It provides support for managing tape media
and other removable media defined to it.
With the DFSMSrmm functional component of DFSMS, removable media can be managed as one
enterprise-wide library across systems.
DFSMSrmm manages an installation's tape volumes and the data sets on those volumes. DFSMSrmm also
manages the shelves where volumes reside in all locations except in automated tape library data servers.
50

Data Management
DFSMSrmm manages your removable media resources, including tape cartridges and reels.
It provides the following functions:
Library management
You can create tape libraries, or collections of tape media associated with tape drives, to balance the work
of your tape drives and help the operators that use them. DFSMSrmm can manage the following devices:
A removable media library, which incorporates all other libraries, such as:
- System-managed manual tape libraries.
- System-managed automated tape libraries. Examples of automated tape libraries include:
IBM TotalStorage
Enterprise Automated Tape Library (3494)
IBM TotalStorage
Virtual Tape Servers (VTS)
Non-system-managed or traditional tape libraries, including automated libraries such as a library under
Basic Tape Library Support (BTLS) control.
Shelf management
DFSMSrmm groups information about removable media by shelves into a central online inventory, and
keeps track of the volumes residing on those shelves. DFSMSrmm can manage the shelf space that you
define in your removable media library and in your storage locations.
Volume management
DFSMSrmm manages the movement and retention of tape volumes throughout their life cycle.
Data set management
DFSMSrmm records information about the data sets on tape volumes. DFSMSrmm uses the data set
information to validate volumes and to control the retention and movement of those data sets.
51

Data Management

DFSMSdfp provides the foundation for:


Storage Management - DFSMSdfp includes an interactive facility that lets installations define and maintain policies to
manage their storage resources.
Data Management - DFSMSdfp helps users store and catalog information on DASD, optical, and tape resources.
Program Management - DFSMSdfp combines programs into executable modules, prepares them to run on the operating
system, stores them in libraries, and reads them into storage for execution.
Device Management - DFSMSdfp is involved in defining input and output devices to the system.
Distributed data access - Distributed data access allows all authorized systems and users in a network to exploit the
52
powerful features of system-managed storage management provided by DFSMS.

BACKUP
Slides

53

Data Management

You can have spanned HFS data sets. Spanned


data sets must be SMS-managed. Data sets that
do not span DASD volumes may be allocated on
non-SMS-managed volumes.
The maximum size of an HFS data set is 8 TB.

A hierarchical file system (HFS) may be in a data set type called HFS or may be in a zFS data set.
An HFS data set can span many DASD volumes. If spanned, they must be SMS-managed.
However, while HFS always has a single file system per data set, zFS may have multiple file systems in a single data set.
These data sets are called aggregates and are a collection of data sets.
HFS data sets are PDSE-like data sets. They are allocated like any other MVS data set. Although an HFS data set is an MVS
data set, an MVS OPEN request will fail. Users must use UNIX System Services (USS) to access data in such a data set.
zFS is a file system that is different from HFS. The administration view of zFS is different from HFS. zFS file systems are
created in a different manner than HFS file systems are created. In general, however, the application view of zFS is the same
as the application view of HFS.
The same APIs and commands are used for zFS as are used for HFS. Once the zFS file system is mounted, it is almost
54
indistinguishable from a mounted HFS file system.

Data Management

The z/OS UNIX file system is hierarchical in structure in the same way as a UNIX file system. All files are members of a directory, and each directory is in
turn a member of another directory at a higher level in the hierarchy. The highest level in the hierarchy is called the root directory.
A file in the HFS is called an HFS file. HFS files are byte-oriented and there is no concept of a record structure.
A file system is contained in an MVS data set type called an HFS data set or zFS.
The HFS in z/OS UNIX is similar to the file system in DOS and Windows.
As with other UNIX file systems, a path name identifies a file and consists of directory names and a file name. A fully qualified file name, which consists of
the name of each directory in the path to a file plus the file name itself, can be up to 1023 bytes long.
The path name is constructed of individual directory names and a file name separated by the forward-slash character, for example:
/dir1/dir2/dir3/MyFile
Like UNIX, z/OS UNIX is case-sensitive for file and directory names.
For example, in the same directory, the file MYFILE is a different file than MyFile.
The files in the HFS are sequential files, and are accessed as bytestreams. A record concept does not exist with these files other than the structure defined
55 an application.
by

Data Management

The set of names required to specify a particular file in a hierarchy of directories is called the path to the file. The path is
specified as a pathname.
A pathname can be absolute or relative. An absolute pathname is a sequence which begins with a slash for the root, followed
by one or more directories, and ends with the file name. A relative pathname is a pathname which is relative to the working
directory. A relative pathname does not start with a slash. A pathname can be up to 1023 characters in length, including all
directory names, separating slashes, and the file name.
A file name can be up to 255 characters in length, and can consist of uppercase and lowercase letters. z/OS UNIX is case
sensitive, which means a file called ProgABC is different from progabc. File names can include extensions which identify the
56
contents of a file, for example, proga.c and name.lst.

Data Management

The HFS consists of one root file system and multiple user file systems. The root file system is the base for HFS and is at the
top of the hierarchy. HFS datasets have the following characteristics:
HFS data sets are allocated as z/OS data sets DSNTYPE=HFS.
HFS data sets can only be opened by USS kernel operations.
HFS data sets can reside on the same volumes as other z/OS data sets.
Non-SMS managed HFS data sets are often used.
The size of the data set is limited to 2 GB if single volume.
A multivolume HFS can span up to a maximum of 59 volumes.
HFS can have up to a maximum of 123 extents per volume.
A multivolume HFS can have up to a maximum of 255 total extents for all volumes.
57

Data Management
Within a file system in HFS there is no way to control the amount of space each user can use. It is possible
for one user to use the entire space in a file system. The recommendation is that each user have their own
file system.
An introduction to the HFS
MVS views an entire file hierarchy as a collection of HFS data sets. Each HFS data set is a mountable file
system. DFSMS/dfp facilities are used to manage an HFS data set, and DFSMS Hierarchical Storage
Manager (DFSMShsm) is used to back up and restore an HFS data set.
With DFSMS/dfp 1.5, the new HFS multivolume support works the same as any other multivolume nonVSAM SMS-managed data set. HFS data sets can now span up to 59 volumes, with up to 255 total extents
for all volumes. As users add files and extend existing files, an HFS data set can increase in size to a
maximum of 123 extents:
If you allocate the HFS with a secondary allocation, it should extend when the current allocation is filled.
If you do not allocate the HFS with a secondary allocation, it will not automatically extend, but you can use
the confighfs shell utility (new in Release 7) to manually extend it.

58

Data Management

A zFS aggregate is a data set that contains zFS file systems. The aggregate is a VSAM LDS and is a container that can contain one or more zFS file
systems. An aggregate can only have one VSAM LDS, but it can contain an unlimited number of file systems. The name of the aggregate is the same as
the VSAM LDS name.
A zFS aggregate that contains only a single zFS file system can be defined and is called a compatibility mode aggregate.
Compatibility mode aggregates are more like HFS. Aggregates that contain multiple file systems are called multi-file system aggregates.
zFS aggregate is a VSAM LDS that contains one or more zFS file systems. The name of the aggregate is its VSAM LDS name.
zFS file system is a logical named entity contained within a zFS aggregate that has a root directory and can be mounted. It has a logical maximum size
which is known as its quota.
Some new terminology. A zFS aggregate is a data set that contains zFS file systems. The zFS aggregate is actually a VSAM LDS.
The term zFS aggregate should not be confused with DFSMS/dfp aggregate groups (a group of data sets).
A zFS file system is a named entity that resides in a zFS aggregate. It contains a root directory and can be mounted into the USS hierarchy. It has a
maximum size known as its quota. (File system is not really a new term but a zFS file system resides in a zFS aggregate, which is different than an HFS
file system.)
The
59 data set that contains zFS file systems is called a zFS aggregate. A zFS aggregate can contain one or more zFS file systems. A zFS aggregate is a
VSAM LDS. Once the zFS aggregate is defined and formatted, one or more zFS file systems can be created in the aggregate.

Data Management

HFS data sets:


Have a partitioned organization
Have the same structure as a PDSE
Must be SMS-managed
Contain mountable file systems used by USS
To create an HFS data set, specify the usual DD statement parameters which are necessary to create any z/OS data set. In
addition, specify DSNTYPE=HFS to request that z/OS create an HFS data set.
The additional PATH parameters shown above must be specified to create an HFS file in the HFS data set:
PATH specifies the name of the HFS file to be created
PATHMODE specifies the file access attributes
PATHDISP specifies the file disposition at end of step
60
PATHOPTS specifies the file access and status

Data Management

This shows an example of how to allocate an HFS data set for the root file system.
The important JCL keyword here is:
DSNTYPE=HFS
Note that an HFS can be either SMS- or non-SMS-managed, but must be cataloged in order for the mount to succeed.
61

Data Management

The IOEAGFMT utility is used to format an existing VSAM LDS as a zFS aggregate. All zFS aggregates must be formatted
before use (including HFS compatibility mode aggregates).
IOEAGFMT does not require the zFS physical file system to be active on the system. The size of the aggregate is as many 8
KB blocks as will fit in the primary allocation of the VSAM LDS.
To extend it to its secondary allocation (assuming it has a secondary allocation), use the zfsadm grow command.
It requires two steps: the first step creates the aggregate and the second step formats it. Since it is a compatibility mode
aggregate, the second step also creates a file system (with the same name as the VSAM LDS cluster entryname) and sets the
quota to the size of the aggregate.
The user must have read and update permission for the VSAM LDS to run the ioeagfmt command.
Format is:
ioeagfmt -aggregate name [-logsize blocks] [-overwrite] [-compat] [-owner
62
{uid|name}] [-group {gid|name}] [-perms {number}] [-parmfile name] [-help]

Data Management

The DFSMStvs element is an optional feature for z/OS. It provides transactional recovery directly within VSAM. DFSMStvs is
an extension to VSAM RLS. It enables any job or application that is designed for data sharing to read-share or write-share
VSAM recoverable data sets.
VSAM RLS provides a server for sharing VSAM data sets in a sysplex. VSAM RLS uses coupling-facility-based locking and
data caching to provide sysplex-scope locking and data access integrity, while DFSMStvs adds logging, two-phase commit,
and backout protocol required for full transactional capability and sharing.
63

Data Management

The DFSMS Network File System (NFS) server is a part of the base z/OS 's Distributed Services. It enables a z/OS system to
act as a file server to workstations, personal computers, or other authorized systems in a TCP/IP network.
Using the DFSMS NFS server, users can remotely access z/OS HFS files as well as conventional data sets, bringing the
resources of a z/OS system (system-managed storage, high-performance storage access, file access security, and centralized
data access) to client platforms.
The NFS server is a TCP/IP application. (TCP/IP is an open set of protocols and applications that allow many different vendor
systems to inter-operate.) When an NFS client system requests access to data on a z/OS system, the request is transmitted
through TCP/IP to the NFS server on a z/OS system. The NFS server processes the request and returns the requested
64
information
to the client through TCP/IP.

References
Introduction to the New Mainframe: Large-Scale Commercial Computing, SG24-7175-00
ABCs of z/OS System Programming Volume 1, SG24-6981-01
ABCs of z/OS System Programming Volume 2, SG24-6982-02
ABCs of z/OS System Programming Volume 3, SG24-6983-03
ABCs of z/OS System Programming Volume 4, SG24-6984-00
ABCs of z/OS System Programming Volume 5, SG24-6985-01
ABCs of z/OS System Programming Volume 6, SG24-6986-00
ABCs of z/OS System Programming Volume 7, SG24-6987-01
ABCs of z/OS System Programming Volume 8, SG24-6988-00
ABCs of z/OS System Programming Volume 9, SG24-6989-04
ABCs of z/OS System Programming Volume 9, SG24-6989-05
ABCs of z/OS System Programming Volume 10, SG24-6990-03
ABCs of z/OS System Programming Volume 11, SG24-6327-01
ABCs of z/OS System Programming Volume 12, SG24-7621-00
ABCs of z/OS System Programming Volume 13, SG24-7717-00
IBM Mainframe Strengths and Values, SG24-7333-01
Introduction to the New Mainframe: z/OS Basics, SG24-6366-01
IBM zEnterprise System Technical Guide, SG24-7833-00
IBM zEnterprise System Technical Introduction, SG24-7832-00

(Chapter 3.2-3.3 of z/OS Basics, Chapter 3 of ABCs of z/OS Volume 1)

65