Secondary storage is usually anything outside of “primary memory”.

It does not permit direct

instruction execution or data fetch by load/store instructions

Disks are organized in much the same way as an ordinary file cabinet. A disk may be divided into
partitions, which are analogous to drawers in a file cabinet. Within each partition are directories,
which are analogous to folders in a drawer, and in fact are often called folders. Each
directory/folder contains files, which are analogous to the papers in a folder. Directories/folders
may contain other directories/folders to add another level of organization.

A Unix filesystem can be visualized as a tree, with each file and directory contained within another
directory. The only file or directory that is not contained in another is the root directory. There is
exactly one root directory on every Unix system.

http://www.cs.uwm.edu/classes/cs315/Bacon/Lecture/HTML/ch02s04.html

Standard Unix systems are traditionally limited to 8 “slices” per disk, some of which have pre-
defined uses. So standard Unix and Solaris disk layouts for many years worked around that limit
by using best practices that didn't need more. However modern Unix systems (including Solaris
10) allows for a form of logical volume management, either with ZFS pools or by using Solaris
Volume Management (SVM). So, you shouldn't be afraid to define additional filesystems if they
do make sense.

Of course, just because you can partition a disk doesn't mean you should! The more partitions you
create, the more there is to manage. If you guess wrong on the space required, you may have to
later grow a partition (not the big deal it used to be). So, you shouldn't make extra partitions unless
you consider the extra work, they create to be worth the additional protections they provide.

https://wpollock.com/AUnix1/Partitioning.htm

Disk partitioning or disk slicing [1] is the creation of one or more regions on secondary storage,
so that each region can be managed separately.[2] These regions are called partitions. It is typically
the first step of preparing a newly installed disk, before any file system is created. The disk stores
the information about the partitions' locations and sizes in an area known as the partition table that
the operating system reads before any other part of the disk. Each partition then appears to the
operating system as a distinct "logical" disk that uses part of the actual disk. System administrators
use a program called a partition editor to create, resize, delete, and manipulate the partitions.[3].
Partitioning allows the use of different filesystems to be installed for different kinds of files.
Separating user data from system data can prevent the system partition from becoming full and
rendering the system unusable. Partitioning can also make backing up easier.
On Unix-based and Unix-like operating systems such as Linux, macOS, BSD, and Solaris, it is
possible to use multiple partitions on a disk device. Each partition can be formatted with a file
system or as a swap partition.

Multiple partitions allow directories such as /boot, /tmp, /usr, /var, or /home to be allocated their
own filesystems. Such a scheme has a number of advantages:

 If one file system gets corrupted, the data outside that filesystem/partition may stay intact,
minimizing data loss.
 Specific file systems can be mounted with different parameters e.g. read-only, or with the
execution of setuid files disabled.
 A runaway program that uses up all available space on a non-system filesystem does not
fill up critical filesystems.

https://en.wikipedia.org/wiki/Disk_partitioning

In a UNIX disk structure, the topmost directory is the root directory and is denoted by a forward
slash (/). A root directory has a number of directories, and these directories have subdirectories. A
subdirectory is known as a child of the parent directory from which it is branched. When a file
system is created, a superblock is also created that keeps track of metadata including the type of
file system, device block size, maximum file volume, name, and other information related to the
file system. After the superblock, another important structure is the inode. An inode acts as an
index of the file. Each node represents a file stored on the disk. Using the -I option with the ls
command, a user can see the inode number of the file.

Computer Forensics Investigating Hard Disks, File and Operating Systems: EC-Council | Press

UNIX I/O

Two types of I/O: buffered and unbuffered.

Buffered I/O

 System Buffer Cache - A form of the disk cache concept. Three lists are maintained:

- Free list: slots in the cache available for allocation

- Device list: the list of buffers associated with each disk

- Driver I/O queue: the list of buffers waiting for or in transit of I/O

 Character Queue - this services the non-block oriented devices, the ones that are character
stream oriented. This allows input to precede the consumption, and output to continue
before delivery
Unbuffered I/O

This is the fastest form of I/O but it dedicates the I/O device to the process and prevents the process
from being swapped out.

UNIX Devices

 Disk drives - Unbuffered or buffered

 Tape drives - Unbuffered or buffered
 Terminals - Character queue
 Communication lines- Character queue
 Printers - Unbuffered for fast and character queues for slow

http://jcsites.juniata.edu/faculty/rhodes/os/ch11b.htm

Raid Levels in Unix

The different raid levels available today

Raid 0 - Stripping data across the disks.

This stripes the data across all the disks present in the array. This improves the read and write
performance. Eg. Reading a large file takes a long time in comparison to reading the same file
from a Raid 0 system.They is no data redundancy in this case.

Raid 1 - Mirroring.
In case of Raid 0 it was observed that there was no redundancy,i.e if one disk fails then the data is
lost. Raid 1 overcomes that problem by mirroring the data. So if one disk fails the data is still
accessible through the other disk.

Raid 2
RAID level that does not use one or more of the "standard" techniques of mirroring, striping and/or
parity. It is implemented by splitting data at bit level and spreading it across the data disks and
redundant disk. It uses a special algorithm called as ECC (error correction code) which is
accompanied across each data block. These are tallied when the data is read from the disk to
maintain data integrity.

Raid 3 - data is striped across multiple disks at a byte level. The data is stripped with parity and
the parity is maintained in a separate disk. So if that disk goes off , it results in a data loss.

Raid 4
Similar to Raid 3 the only difference is that the data is striped across multiple disks at block level.

Raid 5
Block-level striping with distributed parity. The data and parity is stripped across all disks thus
increasing the data redundancy. Minimum three disks are required and if any one disk goes off the
data is still secure.

Raid 6
Block-level striping with dual distributed parity. Its stripes blocks of data and parity across all
disks in the Raid except that it maintains two sets of parity information for each parcel of data thus
increasing the data redundancy. So if two disk go off the data is still intact.

Raid 7
Asynchronous, cached striping with dedicated parity. This level is not a open industry standard. It
is based on the concepts of Raid 3 and 4 and a great deal of cache is included across multiple
levels. Also there is a specialized real time processor to manage the array asynchronously.

http://msunixadm.blogspot.com/2012/12/raid-levels-in-unix.html

UNIX storage is looking more and more like a giant set of Lego blocks that you can put together
in an infinite variety of configurations.

Traditional hard disks remain the dominant medium for on-line storage, but they’re increasingly
being joined by solid state drives (SSDs) for performance sensitive applications. Running on top
of this hardware are a variety of software components that mediate between the raw storage devices
and the filesystem hierarchy seen by users. These components include device drivers, partitioning
conventions, RAID implementations, logical volume managers, systems for virtualizing disks over
a network, and the filesystem implementations themselves.

A storage device is anything that looks like a disk. It can be a hard disk, a flash drive, an SSD, an
external RAID array implemented in hardware, or even a network service that provides block-
level access to a remote device. The exact hardware doesn’t matter, as long as the device allows
random access, handles block I/O, and is represented by a device file.

Storage management layers

• A partition is a fixed-size subsection of a storage device. Each partition has its own device file
and acts much like an independent storage device. For efficiency, the same driver that handles the
underlying device usually implements partitioning. Most partitioning schemes consume a few
blocks at the start of the device to record the ranges of blocks that make up each partition.
Partitioning is becoming something of a vestigial feature. Linux and Solaris drag it along primarily
for compatibility with Windows-partitioned disks. HP-UX and AIX have largely done away with
it in favor of logical volume management, though it’s still needed on Itanium-based HP-UX
systems.
• A RAID array (a redundant array of inexpensive/independent disks) combines multiple storage
devices into one virtualized device. Depending on how you set up the array, this configuration can
increase performance (by reading or writing disks in parallel), increase reliability (by duplicating
or parity-checking data across multiple disks), or both. RAID can be implemented by the operating
system or by various types of hardware. As the name suggests, RAID is typically conceived of as
an aggregation of bare drives, but modern implementations let you use as a component of a RAID
array anything that acts like a disk.

Commonly used RAID levels for UNIX

Following are commonly used RAID levels :

Minimum
Suggested
RAID level hard Notes
application
disks

1. Video Production Provides improved performance and

and Editing additional storage but no-fault
RAID 0 – Striped 2 Hard 2. Image Editing tolerance from disk errors or disk
Set without parity disks 3. Any application failure. Any disk failure destroys the
requiring high array, which becomes more likely
bandwidth with more disks in the array.

Provides fault tolerance from disk

errors and single disk failure.
1. Office application Increased read performance occurs
RAID 1
2. Financial when using a multi-threaded
– Mirrored Set (2 2 Hard
application operating system that supports split
disks minimum) disks
3. Payroll seeks, very small performance
without parity.
application etc reduction when writing. Array
continues to operate so long as at
least one drive is functioning

Highest Read data transaction rate,

Medium Write data transaction rate,
1. File and Overall good (aggregate) transfer
Application servers rate. drive failure requires
3 Hard replacement, but the array is not
RAID 5 2. Internet Web, E-
disks destroyed by a single drive failure.
mail servers
3. Intranet servers Upon drive failure, any subsequent
reads can be calculated from the
distributed parity such that the drive
failure is masked from the end user.
The array will have data loss in the
 event of a second drive failure and is
vulnerable until the data that was on
the failed drive is rebuilt onto a
replacement drive

1. Database server
(such as Oracle /
Provides fault tolerance and
RAID 10 (nested 4 Hard MySQL / MS-SQL)
improved performance but increases
RAID 1+0) disks which requiring
complexity.
high performance
and fault tolerance

https://www.cyberciti.biz/faq/different-raid-levels/

Disk Partitioning

A physical disk can be divided into smaller blocks, called partitions. Unix disk devices operate on
partitions, where each device is a single partition. The simplest configuration is one big partition
for the entire disk. The advantage to having filesystems on separate partitions is that different parts
of your operating system are somewhat protected from each other. If your users have filled up
/home, programs writing log files in /var aren't affected if /home and /var are separate partitions.
If your disk gets corrupted, only the corrupted partition is damaged. The disadvantage is that, in
most cases, if you mistakenly allocated too little disk space for a partition, you can't steal space
from your /var to give you more room on /home once your system is set up.

docstore.mik.ua/orelly/unix3/upt/ch44_04.htm

Disk Management

The kernel is also responsible for maintaining free and used disk space and for the orderly and fair
allocation and deallocation of disk space. It decides where and how much space to allocate to a
newly created file. The kernel code that performs this task is known as the disk storage manager.
Also, the kernel performs disk scheduling, deciding which request to serve next when multiple
requests for file read, write, and so on, arrive for the same disk.
Simple/ordinary files are used to store information and data on a secondary storage device,

typically, a disk.

How files are stored on the secondary storage (usually, a hard disk).

A typical UNIX system has several disk drives that contain user and system files, but, as a user,
you do not have to worry about which disk drive contains the file that you need to access. In UNIX,
multiple disk drives and/or disk partitions can be mounted on the same file system structure,
allowing their access as directories and not as named drives A:, B:, C:, and so on, as in MS-DOS
and Microsoft Windows. You can access files and directories on these disks and/or partitions by
specifying their pathnames as if they are part of the file structure on one disk/partition. Doing so
gives a unified view of all the files and directories in the system, and you do not have to worry
about remembering the names of drives and the files and directories they contain.

As stated earlier, the attributes of a file are stored in a data structure on the disk, called an
inode. At the time of its creation, every file is allocated a unique inode from a list (array)
of inodes on the disk, called the i-list. The index value of the inode in the i-list is called the
inode number for the inode allocated to the file, and is known as the file’s inode number.

The UNIX kernel also maintains a table of inodes, called the inode table, in the main memory
for all open files. When an application opens a file, an inode is allocated from the inode
table and the contents of the file’s inode on the disk are copied into it. The inode number is
used to index the inode table, allowing quick access to the attributes of an open file. When
a file’s attributes (e.g., file size) change, the inode in main memory is updated; disk copies
of inodes are updated at fixed intervals. For files that are not open, their inodes reside on
the disk.

The “link count” field specifies the number of different names the file has within the
system. This count is also known as the hard link count.
The “file mode” field specifies what the file was opened for (read, write, etc.). The “user ID”
is the ID of the owner of the file. The “access permissions” field specifies who can access
the file for what type of operation (discussed in more detail in Chapter 5). The file’s location
on disk is specified by a number of direct and indirect pointers to disk blocks containing
file data.
A typical computer system has several disk drives. Each drive consists of a number of
platters with two surfaces (top and bottom). Each surface is logically divided into concentric
circles called tracks, and each track is subdivided into fixed size portions called sectors.
Tracks at the same position on both surfaces of all platters comprise a cylinder. Disk input/
output (I/O) takes place in terms of one sector, also called a disk block. For this reason,
disks are known as block devices. Traditionally, the sector size for hard disks has been 512
bytes. Newer hard disks use 4K-byte (i.e., 4096-byte) sector sizes. CD-ROMs and DVDROMs
use 2K-byte (i.e., 2048-byte) sector sizes.

A sector may be addressed by using a four-dimensional address comprising <disk #,

cylinder #, surface #, and sector #>. This four-dimensional address is translated to a linear
(one-dimensional) block number, and most of the software in UNIX deals with block
addresses because they are relatively easy to deal with. These blocks start with the sector
numbered as 0 on the outermost cylinder on the topmost surface (i.e., the topmost track
of the outermost cylinder), which is assigned block number 0. The block numbers increase
through the rest of the tracks in that cylinder, through the rest of the cylinders on the disk,
and then through the rest of the disks

UNIX: The Textbook, Third Edition

Disk Allocation

Unix uses indexed disk space allocation. with a disk block size of 4 K bytes. Each file has a file
allocation table analogous to FMT. which is maintained in its inode. 'Ihe allocation table contains
15 entries. Twelve of these entries directly point to data blocks of the file. The next entry in the
allocation table points to an indirect block, i.e.. a block that contains pointers to data blocks. The
next two entries point to double and triple indirect blocks. In this manner. the total file size can as
large as 242 bytes. However, the file size information is stored in a 32-bit word of the inode, so file
size is limited to 232 1 bytes, for which the direct, Single and indirect blocks of allocation table
are adequate.

Operating Systems: A Concept Based Approach