File-System Implementation

Sources: Operating System Concepts by Silberschatz, Galvin and Gagne.

CSC 4103: Operating Systems

12.1

B.B. Karki, LSU

Topics

Implementation issues Allocation Methods Free-Space Management Efficiency and Performance Recovery NFS

CSC 4103: Operating Systems

12.2

B.B. Karki, LSU

File-System Structure

File definition Logical storage unit Named collection of related information File system resides on secondary storage disks are convenient medium rewritable Direct access. Two design problems How the file system should look to the user. How to map the logical file system onto the secondary storage devices.

CSC 4103: Operating Systems

12.3

B.B. Karki, LSU

Layered File System

I/O control: acts as a translator using device drivers and interrupt handlers. Basic file system: commands device driver to read and write physical blocks File-organization module: translates logical block addresses to physical block addresses Logical file system: manages file-system structure, directory structure, protection and security.

CSC 4103: Operating Systems

12.4

B.B. Karki, LSU

Virtual File Systems

Virtual File

Systems (VFS) provide an objectoriented way of implementing multiple file systems.

VFS allows the

same system call interface (the API) to be used for different types of file systems.

CSC 4103: Operating Systems

12.5

B.B. Karki, LSU

In-Memory Structure
The in-memory structure provides file-system management and improved performance via caching:
In-memory partition table: information about each mounted partition. In-memory directory structure: information about recently accessed directories. System-wide open-file table: copy of the FCB of each open file. Per-process open-file table: pointer to the entry in the system-wide open-file table.
12.6

CSC 4103: Operating Systems

B.B. Karki, LSU

In-Memory File System Structures

File open:
-

Pass the file name Search the file in the directory structure Copy FCB into a systemwide open-file table.

File read:
-

Make an entry in a perprocess open-file table. Pointers to other fields (current location in the file, access mode)

File close:
Remove per-process entry and decrement the systemwide entrys open count.
12.7 B.B. Karki, LSU

CSC 4103: Operating Systems

On-Disk Structure
The on-disk structure for implementation of a file system includes:
Boot control block Information about how to boot an OS. e.g., boot block in UFS. Partition control block Partition details such as information about blocks, block and file pointers. e.g., superblock in UFS. Directory structure Organization of files. File control block File details. e.g., inode in UFS.

CSC 4103: Operating Systems

12.8

B.B. Karki, LSU

Directory Implementation

Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute

Hash Table linear list with hash data structure. decreases directory search time collisions situations where two file names hash to the same location fixed size

CSC 4103: Operating Systems

12.9

B.B. Karki, LSU

File Control Block (FCB)

Storage structure consisting of information about a file:

CSC 4103: Operating Systems

12.10

B.B. Karki, LSU

Allocation Methods

Allocation method refers to how disk blocks are allocated to files so that disk space is used effectively and files can be accessed quickly:

Three methods: Contiguous allocation

Linked allocation Indexed allocation

CSC 4103: Operating Systems

12.11

B.B. Karki, LSU

Contiguous Allocation

Each file occupies a set of contiguous blocks on the disk. Simple only starting location (block #) and length (number of

blocks) are required.

Both sequential and random accesses supported. Wasteful of space (dynamic storage-allocation problem) First fit and best fit strategies can be used. Suffers from external fragmentation Files cannot grow.

CSC 4103: Operating Systems

12.12

B.B. Karki, LSU

Contiguous Allocation of Disk Space

CSC 4103: Operating Systems

12.13

B.B. Karki, LSU

Extent-Based Systems

Many file systems use a modified contiguous allocation

scheme.
Initial continuous chunk of space Apply extra chunk of contiguous space, if needed.

Extent-based file systems allocate disk blocks in extents. An extent is a contiguous block of disks. Extents are

allocated for file allocation. A file consists of one or more extents.

CSC 4103: Operating Systems

12.14

B.B. Karki, LSU

Linked Allocation
Each file is a linked list of disk blocks: blocks may be scattered

anywhere on the disk.

Simple need only starting address Any isolated free block can be used - no external fragmentation. A file can grow as long as free blocks are available. Useful for sequential-access files Follow pointers from one block to next starting from the first one. Memory cost for pointers Cluster of multiple blocks to reduce memory overhead and disk head seeks.

CSC 4103: Operating Systems

12.15

B.B. Karki, LSU

Linked Allocation (Contd.)

The directory contains a

pointer to the first and last blocks of the file.

A file of five blocks might

start at block 9, continue at block 16, then block 1, block 10, and finally block 25.
Each block contains a

pointer to the next block

CSC 4103: Operating Systems

12.16

B.B. Karki, LSU

File-Allocation Table
The table has one entry

for each disk block, and is indexed by the block number.

The entry contains the

block number of the next block in the file.

Unused blocks are

indicated by a 0 table value.

Example: file consisting of

disk blocks 271, 618 and 339.

CSC 4103: Operating Systems

12.17

B.B. Karki, LSU

Indexed Allocation
Brings all pointers (block

addresses) together into the index block.

Each file has its own

index box The ith entry points to the ith block of the file The directory contains the address of the index block. Logical view: Similar to a paging scheme.

CSC 4103: Operating Systems

12.18

B.B. Karki, LSU

Indexed Allocation (Cont.)

Supports random access No external fragmentation Mapping from logical to physical in a file of maximum size of 256K

words and block size of 512 words. We need only 1 block for index table.
Mapping from logical to physical in a file of unbounded length (block

size of 512 words).

One block is not enough for index table.

Linked scheme Link blocks of index table (no limit on size). Multilevel index Combined scheme
CSC 4103: Operating Systems 12.19 B.B. Karki, LSU

Combined Scheme: UNIX

CSC 4103: Operating Systems

12.20

B.B. Karki, LSU

Free-Space Management Bit Vector

Maintains a free-space list to keep track of free disk space. Records all free disk blocks and allocate that space to new files Bit vector or bit map (n blocks) Simple and efficient to locate first free block Block number calculation:

0 1

n-1

bit[i] = 1 block[i] free 0 block[i] occupied

(number of bits per word) * (number of 0-value words) + offset of first 1 bit

Bit map requires extra space in main memory.

block size = 212 bytes disk size = 230 bytes (1 gigabyte) n = 230/212 = 218 bits (or 32K bytes)

CSC 4103: Operating Systems

12.21

B.B. Karki, LSU

Linked Free Space List

Link together all the free disk

blocks.
Keep a pointer to the first free

block on the disk and cache it in memory.

The first block contains a pointer

to the next free disk block and so on.

CSC 4103: Operating Systems

12.22

B.B. Karki, LSU

Grouping and Counting Free Blocks

Grouping A linked list of index blocks is kept Each index block contains addresses of n free blocks and a pointer to the next index block Provides address of a large number of free blocks quickly. Counting Keeps the address of the first free block and the number n of free contiguous blocks that follow the first block Each entry in the list contains a disk address and a count. Makes overall list shorter.

CSC 4103: Operating Systems

12.23

B.B. Karki, LSU

Efficiency

Efficiency is dependent on disk allocation and directory

algorithms.

Some examples: UNIX inodes Clustering blocks Files directory entries Pointer sizes Dynamic allocation

CSC 4103: Operating Systems

12.24

B.B. Karki, LSU

Performance
Performance can still improved once basic file-system algorithms are

selected:
Caching approaches: Track cache Disk cache Page cache Virtual disk or RAM disk

Various disk caching locations

12.25

CSC 4103: Operating Systems

B.B. Karki, LSU

Recovery

Files and directories are kept both in main memory and on disk. System failure can result in loss of data or in data inconsistency. Consistency checking Compares data in directory structure with data blocks on disk, and tries to fix inconsistencies. Back up and restore Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape). Recover lost file or disk by restoring data from backup. Back-up cycle: Full back up followed by incremental backups at later times or days.

CSC 4103: Operating Systems

12.26

B.B. Karki, LSU

Log Structured File Systems

Log structured file systems record each update to the file

system as a transaction.
All transactions are written to a log. A transaction is

considered committed once it is written to the log.

The transactions in the log are asynchronously written to

the file system. When the file system is modified, the transaction is removed from the log.
If the file system crashes, all remaining transactions in the

log must still be performed.

CSC 4103: Operating Systems

12.27

B.B. Karki, LSU

Network File System (NFS)

An implementation and specification of a software system

for accessing remote files across LANs (or WANs).

A client-server-network file system

Uses TCP or UDP/IP protocol for networking

CSC 4103: Operating Systems

12.28

B.B. Karki, LSU

NFS (Cont.)
A set of interconnected workstations is viewed as a set of

independent machines with independent file systems

allow sharing among these file systems in a transparent

manner. A remote directory is mounted over a local file system

directory.
The mounted directory looks like an integral subtree of the

local file system, replacing the subtree descending from the local directory. Files in the remote directory can then be accessed in a transparent manner. Subject to access-rights accreditation, potentially any file system (or directory within a file system), can be mounted remotely on top of any local directory.

CSC 4103: Operating Systems

12.29

B.B. Karki, LSU

NFS (Cont.)

NFS specifications are independent of the media: heterogeneous environment of different machines, operating systems, and network architectures. use of RPC primitives built on top of an External Data Representation (XDR) protocol. NFS specification distinguishes between the services

provided by a mount mechanism and the actual remotefile-access services.

Uses two protocols Mount protocol NFS protocol.

CSC 4103: Operating Systems

12.30

B.B. Karki, LSU

Three Independent File Systems

Three independent file systems and machines. Only local files

can be accessed at each machine.

CSC 4103: Operating Systems 12.31 B.B. Karki, LSU

Mounting in NFS
Mounting S1 over U:

Users can use /usr/local/dir1 on U. /usr/local is no longer visible.

Cascading: A file

system can be mounted over another file system that is remotely mounted, not local. Mounts Cascading mounts

CSC 4103: Operating Systems

12.32

B.B. Karki, LSU

NFS Mount Protocol

Establishes initial logical connection between server and client. Mount operation includes name of remote directory to be mounted

and name of server machine storing it.

Mount request is mapped to corresponding RPC and forwarded to

mount server running on server machine. Export list specifies local file systems that server exports for mounting, along with names of machines that are permitted to mount them. Following a mount request that conforms to its export list, the server

returns a file handlea key for further accesses.

File handle a file-system identifier, and an inode number to

identify the mounted directory within the exported file system. The mount operation changes only the users view and does not affect the server side.

CSC 4103: Operating Systems

12.33

B.B. Karki, LSU

NFS Protocol
Provides a set of remote procedure calls for remote file operations. The

procedures support the following operations:

searching for a file within a directory reading a set of directory entries manipulating links and directories accessing file attributes reading and writing files

NFS servers are stateless; each request has to provide a full set of

arguments.
Modified data must be committed to the servers disk before results are

returned to the client.

The NFS protocol does not provide concurrency-control mechanisms.

CSC 4103: Operating Systems

12.34

B.B. Karki, LSU

Three Major Layers of NFS Architecture

UNIX file-system interface (based on the open, read, write, and

close calls, and file descriptors).

Virtual File System (VFS) layer distinguishes local files from

remote ones, and local files are further distinguished according to their file-system types.
The VFS activates file-system-specific operations to handle local

requests according to their file-system types. Calls the NFS protocol procedures for remote requests. NFS service layer bottom layer of the architecture; implements

the NFS protocol.

CSC 4103: Operating Systems

12.35

B.B. Karki, LSU

Schematic View of NFS Architecture

CSC 4103: Operating Systems

12.36

B.B. Karki, LSU

Summary
File system resides on secondary storage Disk partitions to support multiple file systems which can be mounted onto a logical file system Layered file system structure Integrate different files systems Disk space can be allocated to files using contiguous, linked or

indexed allocation methods.

Free space can be managed using bit vectors, linked lists, grouping

and counting.
NFS (network file systems) based on client-server paradigm allows

access remotely located file systems

The VFS activates file-system-specific operations to handle local requests

according to their file-system types. Calls the NFS protocol procedures for remote requests.
CSC 4103: Operating Systems 12.37 B.B. Karki, LSU

Example

Given a file of 100 blocks, what is the minimum number of disk

I/O operations needed to insert a block in the middle of the file for the contiguous, linked and indexed allocation strategies. Assume that the FCB information and the block to be inserted are already in memory.

CSC 4103: Operating Systems

12.38

B.B. Karki, LSU

Example

Consider a file system on a disk that has both logical and physical

block sizes of 512 bytes. Assume that the information about each file is already in memory. For each of the three allocation strategies (contiguous, linked and indexed), answer these questions:
How is the logical-to-physical mapping accomplished in the system? (For

the indexed allocation, assume that a file is always less than 512 blocks long.) If we are currently at logical block 10 (the last block access was block 10) and want to access logical block 4, how many physical blocks must be read from the disk?

CSC 4103: Operating Systems

12.39

B.B. Karki, LSU