CS2028 - Unix Internals Chapter 1


General Overview of the Unix History System structure User Perspective Operating System Services Assumptions about hardware

What is Unix?

What is an operating system?
The low­level (first layer loaded) software which 

handles the interface to system hardware (input/output  devices, memory, file system, etc), schedules tasks,  and provides common core services such as a basic  user interface.
all of the software that comes with a system before 

applications are installed.


 LynxOS. Windows  4 . UnixWare. RTAI. TRUSIX. etc. VxWorks. Embedded:  BeOS.  IRIX. Tru64.  Net. Triance.  Windows CE. QNX.  System V. *BSD (Free. SE­Linux. Solaris. Linux.  z/OS. JUNOS. FreeRTOS. VSTa.  IOS.What Is Unix? Examples of Operating Systems: Unix and Unix­like:  A/UX.1/95/98/NT/XP/2000/2003. MorphOS. etc. Chorus OS. Primos. AIX. RTLinux. VRTX. Micro. Open. Symbian. QNX. etc. 3. LynxOS. HP­UX. Hurd. JavaOS. Others:  AOS. Darwin (Mac OS X). etc). Minix. eCos.

Motorola 680x0. interrupts. multi-tasking and multi-user operating system  What is Unix? Portable: runs on many different hardware architectures (Intel x86 and IA-64. and task switching). 5   . IBM S/390. PowerPC. SPARC. Multi-user: many users can share the computer system at the same time. MIPS. etc.). HP PA-RISC. Preemptive multi-tasking: several programs can run at the same time (time slices. Alpha.A portable.

The command prompt is a simple user process. Includes support for regular expressions which are convenient for complex searching. uniform file model which includes devices and access to other services in a flexible. change and port. which is also a convenient job programming language. the Unix shell. understand. Written in a high-level language (“ C” ) making it easy to read. 6 . hierarchical file system.What is Unix? Other Features     Uses a simple.

   7 . By understanding the Unix/Linux community and its culture. HP.). SGI. Source code availability provides the ability to “ get under the hood” of operating system design and function (and TCP/IP). etc. Linux is the fastest growing operating system in the market.Why Do I Need to Know This?  Ubiquitous: Most big computers and much of the Internet infrastructure runs on some variant of Unix (SUN. it is easy to collect critical information.


           

Multics: 1965 Bell with General Electronic Company and Massachusetts Institute of Technology.GE645. Multiuser, Large data storage & power, data sharing. UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File system – Thompson (Space travel program) – PDP 7 UNIX moved to PDP – 11 in 1971 UNIX in C (Ritchie) 1977 – UNIX ported into non PDP machine , Interdata 8/32 BSD: 1978 Berkeley Software Distribution- UNIX system III – V(1983) Commercial Vendors: Sun, HP, IBM, SGI, DEC GNU: 1984 Richard Stallman, FSF POSIX: 1986 IEEE Portable Operating System unIX Minix: 1987 Andy Tannenbaum SVR4: 1989 AT&T and Sun Linux: 1991 Linus Torvalds Intel 386 (i386) Open Source: GPL, LGPL, Cathedral and the Bazaar

Unix History

1965 joint project between AT&T Bell Labs, GE(General Electric company), and MIT(Massachusetts Institute of Technology) to develop a new OS. Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users. Result: Multiplexed Information & Computer Service - MULTICS.


Unix History

1969 Bell Labs withdraws from group. Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “ Convenient interactive computing service” *.

* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal, Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.


He was told to get the game off his work computer.Unix History  At the same time Ken Thompson wrote a game “ space travel” in Fortran to run on GECOS OS (Honeywell 635).  11 . The spaceship was hard to control and it was expensive to run.

 12 . Unix History Thompson ported the game to a little used PDP-7 computer. Unics (later Unix) was born as a pun on Multics.

Unix History PDP-7 13 .

Unix History  Dennis Ritchie developed “ B” .  14 . Then wrote “ C” a compiled language. In 1973 entire OS ported to “ C” .

 SGI. Introduced many new  features. IBM. NCR.Unix History Because of a 1956 Consent Decree AT&T  could not market Unix so it provided it to  academia. HP. ➲ AT&T Bell Labs realized the commercial  potential and began distributing System V. ➲ Commercialization of Unix (70s / 80s)     AT&T. 15 . DEC. ➲ Late 70s : Thompson took a sabbatical to  teach Unix at UC Berkley –  ➲         Birth of BSD Unix. Sun.

0 is released.Linus Torvalds   1991 Linux 0. 1994 Linux 1. 16 .02 is first released to the public.

MIPS. configure. IBM S/390. SPARC. Alpha.).Three Definitions of Linux Kernel: The very low-level software that manages your computer hardware and provides a library (POSIX) interface for user-level software. Linux GNU/Linux Linux . and use (at least for the target audience). the GNU/Linux OS and lots of other software to make Linux easy to install. The Linux kernel runs on many platforms (Intel x86 and IA-64. OS: The Linux kernel plus utility software to provide a useful working environment. HP PARISC. etc. Motorola 680x0. PowerPC. Distributions: The packaging of the Linux Kernel.

the Linux Mascot .Tux.

mail. Now Unix is maintained by Open Group. text processing packages and source code control systems Second parts-system calls and hardware for supports to first parts Example is UNIX system V produced by AT & T.What is UNIX?       Unix is a Operating System that popular since 1969. System divided into two parts First parts-programs and services such as shell. .

prolog. . java with their Compiler or interpreter. multi process system Hides machine architecture – portable Even though Unix is in ‘ C’ we can run other languages like cobol. basic. lisp. Multi user. fortran. pascal.Unix Features          High level language ‘ C’ Simple user interface Complex programs built from simpler programs Hierarchical file system Consistent format for files-byte streams Consistent interface to peripheral devices. ada.

dynamic kernel modules shared copy-on-write executables TCP/IP networking SMP support.Linux Features    “ UNIX-like” operating system “ aims at” standards compliance “ all the features you would expect in a modern UNIX”       preemptive multitasking virtual memory (protected memory. applications  other features:      . documentation. large memory. support. paging) shared libraries demand loading. highly portable. large files advanced networking. stable. open source GUIs. supports most device hardware active development community. advanced filesystems efficient.

Architecture of Unix System emacs sh cpp cc as ld who date kernel ed wc nroff grep • OS interacts directly with    the hardware  • Such OS is called    system kernel hardware Other apps .

 Network • Kernel: the memory resident portion of Unix system • File system and process control system are two major   components of Unix Kernel. Memory.Unix: Structure • Operating System: a system that manages the resources    of a computer. I/O devices. • Resources: CPUs. .

Unix System Kernel • Three major tasks of kernel:  Process Management  Device Management  File Management • Three additional Services for Kernel: B Virtual Memory B Networking B Network File Systems • Experimental Kernel Features: D Multiprocessor support D Lightweight process (thread) support .

Block Diagram of System Kernel User Level Kernel   Level User Programs Libraries System Call Interface Process control Inter­process communication Scheduler Memory management File Subsystem Device drivers subsystem hardware control Hardware Level hardware  .

Process Control Subsystem • Process Synchronization • Interprocess communication • Memory management:  • Scheduler: process scheduling  (allocate CPU to Processes) .

File subsystem • A file system is a collection of files and directories on   a disk or tape in standard UNIX file system format.  • Kernel’s file sybsystem regulates data flow between   the kernel and secondary storage devices.  .

Hardware Control • Hardware control is responsible for handling interrupts    and for communicating with the machine. . • The kernel may resume execution of the interrupted     process after servicing the interrupt. • Devices such as disks or terminals may interrupt the     CPU while a process is executing.

 mount.      exec: invoke a new program. unlink  .System Call • A process accesses system resources through system call. unmount. close     inode: chdir.      exit: terminate process execution a File system:     File: open. link. stat fstat     others: pipe dup. • System call for a Process Control:       fork: create a new process      wait: allow a parent process to synchronize its  execution with the exit of a child process. chown chmod. read. lseek. write.

User Perspective-File system .

Architecture of Unix Unix file system root (/) bin dev home tmp spool (users’ accounts) mail (users’ mail) 31 usr etc var log bin X11R6 .

  ➲ home: contains the home directories of users  (/export/home on sun computers). sbin. Where it starts. 32 .  ➲ dev: short for devices. holds the files necessary  to operate peripherals such as printers and  terminals. usr/bin: software for the shells and  most common Unix commands.Typical Directories ➲ ➲  / :  Root of the tree. bin.

) etc: administrative files such as lists of user names and passwords. etc. var: contains files that vary in size.Typical Directories tmp: holds temporary files. 33 . logs. printer spool files. (Mail directories.

Typical Directories  usr: Contains application programs lib: Contains libraries for programs proc: a pseudo-filesystem used as an interface to kernel data structures.   34 .

Unix File System File system is organised as a heirarchy of directories  It starts from a single directory called root(represented by a /). Types      Ordinary files Directories Special files Pipes .

.a filename(14 bytes) .Directory   Directory is file containing list of files and subdirectories It has fixed size records of 16 bytes each which contains .an inode number (2 bytes) which acts as a pointer to where the system can find info about the file.

Special Files Special files are contained in the directory /dev..  Ex: Special device . Pipes    UNIX allows us to link commands together using a pipe. The pipe acts as a temporary file which only exists to hold data from one command until its read by another./dev/null(unwanted output can be redirected). ex: command1 | command2 | command3.  They are used to represent a real physical device such as a printer... tape device etc.  .

• A process is an instance of the program in execution. • For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ .Processes • A program is an executable file.

• A process is an instance of the program in execution.Processes • A program is an executable file. • For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ .

• Process information are stored in two places: p Process table  p User table .Processes • A process has   text: machine instructions   (may be shared by other processes)  data  stack • Process may execute either in user mode and in kernel    mode.

 newpid = child process id • In child process. . • The process that invokes fork is called parent process    and the newly created process is called child process. • On return from fork system call.  • In parent process. the two processes have     identical copies of their user­level context except for the    return value pid.System call: fork() • fork: the only way for a user to create a process in Unix    operating system. • The syntax of fork system call: newpid = fork(). newpid = 0.

c ­o forkEx1 $ forkEx1 Before forking .. fpid).    } else {       printf("Parent Process fpid=%d\n".h> main() {    int fpid.    if (fpid == 0) {       printf("Child Process fpid=%d\n"./* forkEx1..    fpid = fork(). fpid).\n"). Child Process fpid=0 After forking fpid=0 Parent Process fpid=14707 After forking fpid=14707 $ .    } $ cc forkEx1.     printf("Before forking .c */ #include <stdio.     }    printf("After forking fpid=%d\n"... fpid).

c ­o forkEx1 $ forkEx1 Before forking ./* forkEx1.    if (fpid == 0) {       printf("Child Process fpid=%d\n".     printf("Before forking .\n").h> main() {    int fpid.    } $ cc forkEx1.     }    printf("After forking fpid=%d\n"..    } else {       printf("Parent Process fpid=%d\n". Child Process fpid=0 After forking fpid=0 Parent Process fpid=14707 After forking fpid=14707 $ .. fpid). fpid).c */ #include <stdio..    fpid = fork(). fpid)..

/* forkEx2...    system("ps").    printf("After forking  fpid=%d\n".    system("ps").     printf("Before forking ..c */ #include <stdio.    fpid = fork().\n"). } $  ps    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh $ $ forkEx2 Before forking .h> main() {    int fpid. fpid)..    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh  14778 pts/9    0:00 sh  14777 pts/9    0:00 forkEx2    PID TTY      TIME CMD  14781 pts/9    0:00 sh  14759 pts/9    0:00 tcsh  14782 pts/9    0:00 sh  14780 pts/9    0:00 forkEx2  14777 pts/9    0:00 forkEx2 After forking fpid=14780 $    PID TTY      TIME CMD  14781 pts/9    0:00 sh  14759 pts/9    0:00 tcsh  14780 pts/9    0:00 forkEx2 After forking fpid=0  .

• Processes can obtain their PID by calling getpid(). . • Kernel assigns the PID when a new process is created. • Processes can obtain their parent’s PID by calling     getppid(). typically in the range 0 through 30000. • Each process has a parent process and a corresponding     parent process ID.System Call: getpid() getppid() • Each process has a unique process id (PID).  • PID is an integer.

h> #include <unistd. getppid())./* pid.c */ #include <stdio.getpid(). } $ cc pid.h> main() {    printf("pid=%d ppid=%d\n".h> #include <sys/types.c ­o pid $ pid pid=14935 ppid=14759 $ .

     printf("Before forking . } .h> #include <sys/types.    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n".    fpid = fork()./* forkEx3. getppid())..h> main() {    int fpid.         fpid.         fpid.c */ #include <stdio.         fpid.    if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n". getpid().    }    printf("After forking fpid=%d pid=%d ppid=%d\n".h> #include <unistd.\n"). getpid(). getppid()).. getpid(). getppid()).

.$ cc forkEx3.c ­o forkEx3 $ forkEx3 Before forking .. Parent Process fpid=14942 pid=14941 ppid=14759 After forking fpid=14942 pid=14941 ppid=14759 $ Child Process fpid=0 pid=14942 ppid=1 After forking fpid=0 pid=14942 ppid=1 $ ps     PID TTY      TIME CMD  14759 pts/9    0:00 tcsh .

  .System Call: wait() • wait system call allows a parent process to wait    for the demise of a child process.

 status.  fpid.h> main() {    int fpid.. getppid()). } .    if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n". getppid()).\n"). getpid()..#include <stdio.h> #include <sys/types.    fpid = fork().     printf("Before forking .    }    wait(&status).  fpid. getpid().h> #include <unistd.  fpid. getpid().    printf("After forking fpid=%d pid=%d ppid=%d\n".    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n". getppid()).

$ cc forkEx4... Parent Process fpid=14980 pid=14979 ppid=14759 Child Process fpid=0 pid=14980 ppid=14979 After forking fpid=0 pid=14980 ppid=14979 After forking fpid=14980 pid=14979 ppid=14759 $  .c ­o forkEx4 $ forkEx4 Before forking .

   Thus. execl. execle. execv. execve.) • exec system call allows a process to choose its successor.System Call: exec() • exec() system call invokes another program by replacing   the current process • No new process table entry is created for exec() program. the total number of processes in the system isn’t     changed. execvp. • Six different exec functions:    execlp. .    (see man page for more detail.

Sun May  9 16:39:17 CST 1999 $ .c */ #include <stdio.. "date".    } $ execEx1 Before execing .    execl("/bin/date". 0)./* execEx1.    printf("After exec\n").\n")..h> main() {    printf("Before execing .h> #include <unistd...

   execl("/bin/date".. Sun May  9 16:39:17 CST 1999 $ . 0). "date".c */ #include <stdio.    } $ execEx1 Before execing .../* execEx1.h> main() {    printf("Before execing .    printf("After exec\n").\n").h> #include <unistd..

   if (fpid == 0) {      execl("/bin/date"...h> $ execEx2 Before execing .    fpid = fork().\n").    printf("Before execing . After exec and fpid=14903 main() $ Sun May  9 16:47:08 CST 1999 { $    int fpid.h> #include <unistd. 0)../* execEx2.    }    printf("After exec and fpid=%d\n".    } .. "date".fpid).h>  #include <stdio.c */ #include <sys/types.

   fpid = fork().    if (fpid == 0) {      execl("/bin/date". 0)./* execEx2.\n")..h> $ execEx2 Before execing . After exec and fpid=14903 main() $ Sun May  9 16:47:08 CST 1999 { $    int fpid.c */ #include <sys/types..h> #include <unistd.h>  #include <stdio. "date"...    printf("Before execing .    }    printf("After exec and fpid=%d\n".fpid).    } .

c file2. Users capability to (1)redirect I/O.Building Block Primitives    User to create complex program effectively.  Example: grep ssn file1. ls > ssn.  User much access three files  Std input file  Std output file  Std error file  Example: ls.c file4.txt. 2>(err)  (2)Pipe: mechanisms allows a stream of data to be passed between reader and writer processes.c | wc -l .c file3. mail x < letter.

> file . FD 2 -STDERR . << token . Eg. This is how we append to a file using a redirect. FD 1 – STDOUT.means open a file for writing and truncate it and associate it with STDOUT. >> file .means open a file for reading and associate with STDIN.means open a file for writing and seek to the end and associate it with STDOUT. FD 0 – STDIN .      < file . n>&m means redirect FD n to the same places as FD m.Means use the current input stream as STDIN for the program until token is seen. 2>&1 means send STDERR to the same place that STDOUT is going to.

.Operating System Services    Controlling Scheduling Memory Management   Allocating to main memory Allocating to secondary memory   Process management – controlled access to devices Transparent Service – Kernel hides the type of file from user.

ASSUMPTIONS ABOUT H/W:User mode and Kernel mode • At any given instant a computer running the Unix system    is either executing a process or the kernel itself is running • The computer is in user mode when it is executing     instructions in a user process and it is in kernel mode    when it is executing instructions in the kernel. • Executing System call ==> User mode to Kernel mode    perform I/O operations    system clock interrupt     .

 Some machine instructions are privileged. Part of each user process.  Example: virtual address space access only in kernal mode.Difference between two modes  Process in user mode can access their own instruction not kernal instructions and data of other processes but in kernal mode can access both. .   Kernel runs on behalf of the user processes. Example: the process status register is not capable to access from the user mode.

.Interrupts and Exceptions  Interrupts: for multitasking CPU is shared by many processes  Blocks low priority interrupts but services high priority interrupts. executing privileged instructions and dividing by zero.  Exceptions in middle of an instruction whereas interrupts in between 2 instructions.  Exception refers to unexpected events caused by process.   Example: addressing illegal memory. Restart the instruction after handling exception.

 Manipulating linked list pointers Machine Errors Clock Disk Network Devices Terminals Software Interrupts  High priority interrupts     Lower Priority interrupts     Masking a level will block below and allow above.Processor Execution Levels  Kernal prevent the occurrence of interrupts during critical activity. .

Memory Management    Kernal resides in main memory permanently Compiler generates virtual address space Kernal Mapping virtual address to physical address .