Operating System File System: Session 10 (Chapter 9)

OPERATING SYSTEM FILE SYSTEM
Session 10(Chapter 9)
Concept and Structure of a File

2
Contiguous logical address space Types:
None - sequence of words, bytes Simple record structure

Data

numeric character binary
Lines Fixed length Variable length

Formatted document Re-locatable load file
Complex Structures

Program
Can simulate last two with first method by inserting appropriate control characters Who decides:
Operating system Program
OperatingSystems-FileSystem
Attributes of a File
3
Name only information kept in human-readable form Identifier unique tag (number) identifies file within file system Type needed for systems that support different types Location pointer to file location on device Size current file size Protection controls who can do reading, writing, executing Time, date, and user identification data for protection, security, and usage monitoring Information about files are kept in the directory structure, which is maintained on the disk
File Operation
4
File is an abstract data type Create Write Read Reposition within file Delete Truncate Open(Fi) search the directory structure on disk for entry Fi, and move the content of entry to memory Close (Fi) move the content of entry Fi in memory to directory structure on disk
What is needed to Open Files

5
Several pieces of data are needed to manage open files:

File pointer: pointer to last read/write location, per process that has the file open File-open count: counter of number of times a file is open to allow removal of data from open-file table when last processes closes it Disk location of the file: cache of data access information Access rights: per-process access mode information
Open File Locking

6
Provided by some operating systems and file systems Mediates access to a file Mandatory or advisory:

Mandatory access is denied depending on locks held and requested Advisory processes can find status of locks and decide what to do
File type Name and Extension

7
What is File System?

8

The file system is a mechanism for storing data and programs. Main characteristics of Secondary storage

Large storage capacity Non-volatile nature. Give raise to the convenience of storing data and programs on secondary storage.
Consequence?
What do you Require of File Systems?

Naming

Should be flexible, e.g., allow multiple names for same files Support hierarchy for easy of use Want to be sure data has been written to disk in case crash occurs Want to restrict who has access to files Want to share files with other users Sequential access Random access Sequential is most common & Random next Other pattern is Keyed access (not usually provided by OS
Persistence
Sharing/Protection

Speed & Efficiency for different access patterns

10
What do you Require of File Systems?

Minimum Space Overhead
Disk space needed to store metadata is lost for user data
Twist: all metadata that is required to do translation must be stored on disk

Translation scheme should minimize number of additional accesses for a given access pattern Harder than, say page tables where we assumed page tables themselves are not subject to paging!
File System Organization

11
The file system on the disk of an operating system allows data to be stored, searched and retrieved. One part of the file system is the user interface. This interface defines the definition of a file, file attributes, operations allowed on files and a directory structure for organizing files. The other part of the file system has algorithms and data structures to map the logical file system onto the physical storage devices. A file system has a layered design
Each layer uses the features provided by the layer below it to provide features to the layer above it.
The lowest layer is the I/O control.

It consists of device drivers and interrupt handlers for information transfer between the memory and the disk. A device driver is a translator between the file system and the actual disk hardware. The device driver takes high level instructions as input.

12
The next layer, the basic file system issues generic commands to the appropriate device driver to access (read / write) physical blocks on the disk.
The layer above the basic file system is the file organization module.
This module has information about files, the logical records and the physical blocks.
The file organization module translates / maps logical addresses to physical block addresses for the basic file system.
This module also keeps track of free blocks. The logical file system makes use of the directory structure of the file system and provides information to the file organization module given a file name. Protection and security are also part of the logical file system.

13
Over-View
14
15
16
Steps in Opening and Reading a File
Look-up (via-directory)
Find on disks file descriptors block number

Create one if none, else increment ref count By reading on-disk file descriptor
Find entry in open file table.
Find where file data is located
Read data & return to user
Open File Table

17
inode represents file
at most 1 in-memory instance per unique file #number of openers & other properties
With separate offsets for byte-stream
filerepresents one or more processes using an file
dirrepresents an open directory file Generally:

None of data in OFT is persistent Reflects how processes are currently using files Lifetime of objects determined by open/close
Reference counting is used
File Descriptors (inodes)

18
Term inode can refer to 3 things:
1.in-memory inode
Store information about an open file, such as how many openers, corresponds to on-disk file descriptor Region on disk, entry in file descriptor table, that stores persistent information about a file who owns it, where to find its data blocks, etc. A bytewise copy of 2. in memory
2.on-disk inode
3.on-disk inode, when cached in buffer cache
Q.: Should in-memory inode store a pointer to cached on-disk inode?
19
What information does File System contain?

Contains super blockstores information such assize of entire file system, etc.
Location of file descriptor table & free map
Free Block Map

Bitmap used to find free blocks Typically cached in memory
Superblock & free map often replicated in different positions on disk
Allocation of Disk Space

20
These mechanisms looks at how efficiently files are stored and accessed.

Contiguous Linked Indexed
Commonly referred as file allocation strategies
Contiguous Allocation
21
Idea: allocate files in contiguous blocks File Descriptor = (first block, length) Good sequential & random access Problems:
hard to extend files may require expensive compaction external fragmentation analogous to segmentation-based VM
Linked Files
22
Idea: implement linked list

either with variable sized blocks or fixed sized blocks (clusters)
Solves fragmentation problem, but now
need lots of seeks for sequential accesses and random accesses
lose first block, may lose file Solution: keep linked list in memory
unreliable:
DOS: FAT File Allocation Table
23
Indexed Allocation
24
Single-index: specify maximum file-size, create index array, then note blocks in index
access ok one translation step Sequential access requires more seeks depending on contiguous allocation
Random
Drawback: hard to grow beyond maximum
Directory Structure
25
Directory is collection of nodes containing information about files.

Both the directory structure and the files reside on disk Backups of these two structures are kept on tapes
Disk Structure
26
Disk can be subdivided into partitions Disks or partitions can be RAID protected against failure Disk or partition can be used raw without a file system, or formatted with a file system Partitions also known as minidisks, slices Entity containing file system known as a volume Each volume containing file system also tracks that file systems info in device directory or volume table of contents As well as general-purpose file systems there are many special-purpose file systems, frequently all within the same operating system or computer.
Disk Schematics
27
Tracks Sectors and Cylinders

28
From O.S Perspective

29
Disks are big & slow -compared to RAM Access to disk requires

Seek (move arm to track) to cross all tracks anywhere from 20-50ms, on average takes 1/3. Rotational delay (wait for sector to appear under track) 7,200rpm is 8.3ms per rotation, on average takes : 4.15ms rot delay Transfer time (fast: 512 bytes at 998 Mbit/s is about 3.91us)
Seek+Rot Delay dominates Random Access is expensive
And unlikely to get better
Consequence:

Avoid seeks Seek to short distances Amortize seeks by doing bulk transfers
Disk Scheduling
30
Can use priority scheme Can reduce avg access time by sending requests to disk controller in certain order
Or, more commonly, have disk itself reorder requests

Like SJF in CPU scheduling, guarantees minimum avg seek time, but
SSTF: shortest seek time first
can lead to starvation
SCAN: elevator algorithm
Process requests with increasing track numbers until highest reached, then decreasing etc. repeat LOOK dont go all the way to the top without passengers C-SCAN: -only take passengers when going up
Variations:

31
Typical File System Organization
Operations Performed on a Directory

32
Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system
Organize the Directory (Logically) to Obtain

33
Efficiency locating a file quickly Naming convenient to users

Two users can have same name for different files The same file can have several different names
Grouping logical grouping of files by properties, (e.g., all Java programs, all games, )
Single Directory
34
A single directory for all users
Naming and grouping problem
Single Directory
35
Separate directory for each user.

Path name Can have the same file name for different user Efficient searching No grouping capability
Tree Structured Directories

36
Efficient searching Grouping Capability Current directory (working directory)

cd /spell/mail/prog type list
Tree Structured Directories

37
Absolute or relative path name Creating a new file is done in current directory Delete a file
rm <file-name>
Creating a new subdirectory is done in current directory

mkdir <dir-name>
Example: if in current directory /mail

mkdir count
Deleting mail deleting the entire subtree rooted by mail

Acyclic Graph Directories

38
They have shared Sub-directories and files
Acyclic Graph Directories

39
Two different names (aliasing) If dict deletes list dangling pointer Solutions:

Back-pointers, so we can delete all pointers Variable size records a problem Back-pointers using a daisy chain organization
Entry-hold-count solution New directory entry type

Link another name (pointer) to an existing file Resolve the link follow pointer to locate the file
General Graph Directory

40
How do we guarantee no cycles?

Allow only links to file not subdirectories Garbage collection Every time a new link is added use a cycle detection algorithm to determine whether it is OK.
Directory Implementation
41
Linear List
list of file names with pointers to the data blocks is one way to implement a directory. linear search is necessary to find for a particular file The method is simple but the search is time consuming. To create a file a linear search is made to look for the existence of a file with the same file name and if no such file is found the new file created is added to the directory at the end. To delete a file a linear search for the file name is made and if found allocated space is released. Every time making a linear search consumes time and increases access time that is not desirable since a directory information is frequently used. A sorted list allows for a binary search that is time efficient compared to the linear search.
But maintaining a sorted list is an overhead especially because of file creations and OperatingSystems-FileSystem deletions.
Directory Implementation
42
Hash Table

A linear list is used to store directory entries. A hash table takes a value computed from the file name and returns a pointer to the file name in the linear list. Thus search time is greatly reduced. Insertions are prone to collisions that are resolved. The main problem is the hash function that is dependent on the hash table size. A solution to the problem is to allow for chained overflow with each hash entry being a linked list.
Directory lookups in a hash table are faster than in a linear list.
How is File System Mounted?

43
A file system must be mounted before it can be accessed A un-mounted file system is mounted at a mount point
(a) Existing.
(b) Unmounted Partition
Mount point
File Sharing
44
Sharing of files on multi-user systems is desirable Sharing may be done through a protection scheme On distributed systems, files may be shared across a network Network File System (NFS) is a common distributed filesharing method. Multiple Users
User IDs identify users, allowing permissions and protections to be per-user Group IDs allow users to be in groups, permitting group access rights
45
File Sharing Remote File Systems

Uses networking to allow file system access between systems

Manually via programs like FTP Automatically, seamlessly using distributed file systems Semi automatically via the world wide web
Client-server model allows clients to mount remote file systems from servers

Server can serve multiple clients Client and user-on-client identification is insecure or complicated NFS is standard UNIX client-server file sharing protocol CIFS is standard Windows protocol Standard operating system file calls are translated into remote calls
Distributed Information Systems (distributed naming services) such as LDAP, DNS, NIS, Active Directory implement unified access to information needed for remote computing
File Sharing Failure Modes

46
Remote file systems add new failure modes, due to network failure, server failure Recovery from failure can involve state information about status of each remote request Stateless protocols such as NFS include all information in each request, allowing easy recovery but less security
47
File Sharing Consistency Semantics
Consistency semantics specify how multiple users are to access a shared file simultaneously
Similar to process synchronization algorithms
Tend to be less complex due to disk I/O and network latency (for remote file systems
Andrew File System (AFS) implemented complex remote file sharing semantics Unix file system (UFS) implements:

Writes to an open file visible immediately to other users of the same open file Sharing file pointer to allow multiple users to read and write concurrently Writes only visible to sessions starting after the file is closed
AFS has session semantics
Protection
48
File owner/creator should be able to control:

what can be done by whom

Read Write Execute Append Delete List
Types of access

Access Lists and Groups

49
Mode of access: read, write, execute Three classes of users
Ask manager to create a group (unique name), say G, and add some users to the group. For a particular file (say game) or subdirectory, define an appropriate access.
Attach a group to a file
50
A Sample UNIX Directory Listing

Operating System File System: Session 10 (Chapter 9)

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Operating System File System: Session 10 (Chapter 9)

Hochgeladen von

Copyright:

Verfügbare Formate

OPERATING SYSTEM FILE SYSTEM

Concept and Structure of a File

Contiguous logical address space Types:

None - sequence of words, bytes Simple record structure

numeric character binary

Lines Fixed length Variable length

Operating system Program

What is needed to Open Files

Several pieces of data are needed to manage open files:

Open File Locking

File type Name and Extension

What is File System?

What do you Require of File Systems?

Speed & Efficiency for different access patterns

What do you Require of File Systems?

Disk space needed to store metadata is lost for user data

Twist: all metadata that is required to do translation must be stored on disk

File System Organization

The lowest layer is the I/O control.

File System Organization

File System Organization

Steps in Opening and Reading a File

Find on disks file descriptors block number

Find entry in open file table.

Find where file data is located

Read data & return to user

Open File Table

inode represents file

filerepresents one or more processes using an file

dirrepresents an open directory file Generally:

Reference counting is used

File Descriptors (inodes)

Term inode can refer to 3 things:

3.on-disk inode, when cached in buffer cache

Q.: Should in-memory inode store a pointer to cached on-disk inode?

What information does File System contain?

Location of file descriptor table & free map

Free Block Map

Superblock & free map often replicated in different positions on disk

Allocation of Disk Space

Contiguous Linked Indexed

Commonly referred as file allocation strategies

Idea: implement linked list

Solves fragmentation problem, but now

need lots of seeks for sequential accesses and random accesses

DOS: FAT File Allocation Table

Drawback: hard to grow beyond maximum

Directory is collection of nodes containing information about files.

Tracks Sectors and Cylinders

From O.S Perspective

Seek+Rot Delay dominates Random Access is expensive

And unlikely to get better

Or, more commonly, have disk itself reorder requests

SSTF: shortest seek time first

can lead to starvation

SCAN: elevator algorithm

Typical File System Organization

Operations Performed on a Directory

Organize the Directory (Logically) to Obtain

Efficiency locating a file quickly Naming convenient to users

A single directory for all users

Naming and grouping problem