CS2028 - Unix Internals Chapter 1

Agenda

General Overview of the Unix History System structure User Perspective Operating System Services Assumptions about hardware

What is Unix?

What is an operating system?
The low­level (first layer loaded) software which 

handles the interface to system hardware (input/output  devices, memory, file system, etc), schedules tasks,  and provides common core services such as a basic  user interface.
all of the software that comes with a system before 

applications are installed.

3

 etc. Embedded:  BeOS. etc). 3. QNX.  IRIX. FreeRTOS. etc.1/95/98/NT/XP/2000/2003. VSTa.  Windows CE. LynxOS.  System V. TRUSIX. Linux.What Is Unix? Examples of Operating Systems: Unix and Unix­like:  A/UX.  z/OS. LynxOS.  IOS.  Net. RTAI. eCos. HP­UX. etc. Symbian. QNX. *BSD (Free. Solaris. VRTX. VxWorks. RTLinux. Triance. Micro. MorphOS. Minix. UnixWare. AIX. SE­Linux. Chorus OS. Others:  AOS. Hurd. JavaOS. Windows  4 . Darwin (Mac OS X). JUNOS. Primos. Tru64. Open.

MIPS. PowerPC. Motorola 680x0. 5   . Alpha. Multi-user: many users can share the computer system at the same time. Preemptive multi-tasking: several programs can run at the same time (time slices.).A portable. IBM S/390. HP PA-RISC. multi-tasking and multi-user operating system  What is Unix? Portable: runs on many different hardware architectures (Intel x86 and IA-64. etc. and task switching). SPARC. interrupts.

the Unix shell. 6 . which is also a convenient job programming language. understand. Written in a high-level language (“ C” ) making it easy to read. hierarchical file system. change and port. uniform file model which includes devices and access to other services in a flexible.What is Unix? Other Features     Uses a simple. The command prompt is a simple user process. Includes support for regular expressions which are convenient for complex searching.

it is easy to collect critical information. SGI. Linux is the fastest growing operating system in the market.Why Do I Need to Know This?  Ubiquitous: Most big computers and much of the Internet infrastructure runs on some variant of Unix (SUN. HP. By understanding the Unix/Linux community and its culture. Source code availability provides the ability to “ get under the hood” of operating system design and function (and TCP/IP).    7 . etc.).

HISTORY

           

Multics: 1965 Bell with General Electronic Company and Massachusetts Institute of Technology.GE645. Multiuser, Large data storage & power, data sharing. UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File system – Thompson (Space travel program) – PDP 7 UNIX moved to PDP – 11 in 1971 UNIX in C (Ritchie) 1977 – UNIX ported into non PDP machine , Interdata 8/32 BSD: 1978 Berkeley Software Distribution- UNIX system III – V(1983) Commercial Vendors: Sun, HP, IBM, SGI, DEC GNU: 1984 Richard Stallman, FSF POSIX: 1986 IEEE Portable Operating System unIX Minix: 1987 Andy Tannenbaum SVR4: 1989 AT&T and Sun Linux: 1991 Linus Torvalds Intel 386 (i386) Open Source: GPL, LGPL, Cathedral and the Bazaar

Unix History

1965 joint project between AT&T Bell Labs, GE(General Electric company), and MIT(Massachusetts Institute of Technology) to develop a new OS. Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users. Result: Multiplexed Information & Computer Service - MULTICS.

9

Unix History

1969 Bell Labs withdraws from group. Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “ Convenient interactive computing service” *.

* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal, Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.

10

Unix History  At the same time Ken Thompson wrote a game “ space travel” in Fortran to run on GECOS OS (Honeywell 635).  11 . The spaceship was hard to control and it was expensive to run. He was told to get the game off his work computer.

Unics (later Unix) was born as a pun on Multics.  12 . Unix History Thompson ported the game to a little used PDP-7 computer.

Unix History PDP-7 13 .

Then wrote “ C” a compiled language. In 1973 entire OS ported to “ C” .Unix History  Dennis Ritchie developed “ B” .  14 .

 SGI.Unix History Because of a 1956 Consent Decree AT&T  could not market Unix so it provided it to  academia. ➲ AT&T Bell Labs realized the commercial  potential and began distributing System V. DEC. 15 . ➲ Late 70s : Thompson took a sabbatical to  teach Unix at UC Berkley –  ➲         Birth of BSD Unix. ➲ Commercialization of Unix (70s / 80s)     AT&T. Sun. HP. NCR. IBM. Introduced many new  features.

1994 Linux 1.02 is first released to the public. 16 .0 is released.Linus Torvalds   1991 Linux 0.

the GNU/Linux OS and lots of other software to make Linux easy to install. OS: The Linux kernel plus utility software to provide a useful working environment. and use (at least for the target audience). The Linux kernel runs on many platforms (Intel x86 and IA-64. IBM S/390. configure.Three Definitions of Linux Kernel: The very low-level software that manages your computer hardware and provides a library (POSIX) interface for user-level software. MIPS. HP PARISC. Distributions: The packaging of the Linux Kernel. Motorola 680x0. Alpha. Linux GNU/Linux Linux . SPARC. etc.). PowerPC.

Tux. the Linux Mascot .

mail. System divided into two parts First parts-programs and services such as shell. text processing packages and source code control systems Second parts-system calls and hardware for supports to first parts Example is UNIX system V produced by AT & T. . Now Unix is maintained by Open Group.What is UNIX?       Unix is a Operating System that popular since 1969.

basic. prolog. multi process system Hides machine architecture – portable Even though Unix is in ‘ C’ we can run other languages like cobol. java with their Compiler or interpreter. Multi user. lisp. ada. fortran. pascal. .Unix Features          High level language ‘ C’ Simple user interface Complex programs built from simpler programs Hierarchical file system Consistent format for files-byte streams Consistent interface to peripheral devices.

stable. open source GUIs. highly portable.Linux Features    “ UNIX-like” operating system “ aims at” standards compliance “ all the features you would expect in a modern UNIX”       preemptive multitasking virtual memory (protected memory. large memory. supports most device hardware active development community. dynamic kernel modules shared copy-on-write executables TCP/IP networking SMP support. support. applications  other features:      . advanced filesystems efficient. documentation. paging) shared libraries demand loading. large files advanced networking.

Architecture of Unix System emacs sh cpp cc as ld who date kernel ed wc nroff grep • OS interacts directly with    the hardware  • Such OS is called    system kernel hardware Other apps .

. Network • Kernel: the memory resident portion of Unix system • File system and process control system are two major   components of Unix Kernel. • Resources: CPUs. Memory. I/O devices.Unix: Structure • Operating System: a system that manages the resources    of a computer.

Unix System Kernel • Three major tasks of kernel:  Process Management  Device Management  File Management • Three additional Services for Kernel: B Virtual Memory B Networking B Network File Systems • Experimental Kernel Features: D Multiprocessor support D Lightweight process (thread) support .

Block Diagram of System Kernel User Level Kernel   Level User Programs Libraries System Call Interface Process control Inter­process communication Scheduler Memory management File Subsystem Device drivers subsystem hardware control Hardware Level hardware  .

Process Control Subsystem • Process Synchronization • Interprocess communication • Memory management:  • Scheduler: process scheduling  (allocate CPU to Processes) .

  • Kernel’s file sybsystem regulates data flow between   the kernel and secondary storage devices.  .File subsystem • A file system is a collection of files and directories on   a disk or tape in standard UNIX file system format.

Hardware Control • Hardware control is responsible for handling interrupts    and for communicating with the machine. • The kernel may resume execution of the interrupted     process after servicing the interrupt. • Devices such as disks or terminals may interrupt the     CPU while a process is executing. .

 close     inode: chdir. chown chmod. read. lseek. link. unlink  . mount.      exit: terminate process execution a File system:     File: open. stat fstat     others: pipe dup. write.      exec: invoke a new program. • System call for a Process Control:       fork: create a new process      wait: allow a parent process to synchronize its  execution with the exit of a child process. unmount.System Call • A process accesses system resources through system call.

User Perspective-File system .

Architecture of Unix Unix file system root (/) bin dev home tmp spool (users’ accounts) mail (users’ mail) 31 usr etc var log bin X11R6 .

  ➲ home: contains the home directories of users  (/export/home on sun computers). bin. holds the files necessary  to operate peripherals such as printers and  terminals. usr/bin: software for the shells and  most common Unix commands.Typical Directories ➲ ➲  / :  Root of the tree. 32 . sbin.  ➲ dev: short for devices. Where it starts.

33 .Typical Directories tmp: holds temporary files. printer spool files. (Mail directories.) etc: administrative files such as lists of user names and passwords. logs. etc. var: contains files that vary in size.

  34 .Typical Directories  usr: Contains application programs lib: Contains libraries for programs proc: a pseudo-filesystem used as an interface to kernel data structures.

Unix File System File system is organised as a heirarchy of directories  It starts from a single directory called root(represented by a /). Types      Ordinary files Directories Special files Pipes .

a filename(14 bytes) .an inode number (2 bytes) which acts as a pointer to where the system can find info about the file.Directory   Directory is file containing list of files and subdirectories It has fixed size records of 16 bytes each which contains . .

The pipe acts as a temporary file which only exists to hold data from one command until its read by another.  They are used to represent a real physical device such as a printer. tape device etc. Pipes    UNIX allows us to link commands together using a pipe..  Ex: Special device ..Special Files Special files are contained in the directory /dev.  . ex: command1 | command2 | command3../dev/null(unwanted output can be redirected).

Processes • A program is an executable file. • A process is an instance of the program in execution. • For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ .

Processes • A program is an executable file. • A process is an instance of the program in execution. • For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ .

• Process information are stored in two places: p Process table  p User table .Processes • A process has   text: machine instructions   (may be shared by other processes)  data  stack • Process may execute either in user mode and in kernel    mode.

• The process that invokes fork is called parent process    and the newly created process is called child process. newpid = child process id • In child process. newpid = 0. • The syntax of fork system call: newpid = fork(). .  • In parent process. the two processes have     identical copies of their user­level context except for the    return value pid.System call: fork() • fork: the only way for a user to create a process in Unix    operating system. • On return from fork system call.

     }    printf("After forking fpid=%d\n".    fpid = fork()..c ­o forkEx1 $ forkEx1 Before forking . fpid).     printf("Before forking .h> main() {    int fpid...c */ #include <stdio.    } else {       printf("Parent Process fpid=%d\n".    } $ cc forkEx1.    if (fpid == 0) {       printf("Child Process fpid=%d\n".\n")./* forkEx1. Child Process fpid=0 After forking fpid=0 Parent Process fpid=14707 After forking fpid=14707 $ .. fpid). fpid).

 fpid).h> main() {    int fpid.. fpid).\n"). fpid). Child Process fpid=0 After forking fpid=0 Parent Process fpid=14707 After forking fpid=14707 $ .    } else {       printf("Parent Process fpid=%d\n".     }    printf("After forking fpid=%d\n".    fpid = fork().    if (fpid == 0) {       printf("Child Process fpid=%d\n".    } $ cc forkEx1.     printf("Before forking ....c */ #include <stdio.c ­o forkEx1 $ forkEx1 Before forking ./* forkEx1.

h> main() {    int fpid. } $  ps    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh $ $ forkEx2 Before forking . fpid).c */ #include <stdio....\n").     printf("Before forking .    system("ps")../* forkEx2.    system("ps").    printf("After forking  fpid=%d\n".    fpid = fork().    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh  14778 pts/9    0:00 sh  14777 pts/9    0:00 forkEx2    PID TTY      TIME CMD  14781 pts/9    0:00 sh  14759 pts/9    0:00 tcsh  14782 pts/9    0:00 sh  14780 pts/9    0:00 forkEx2  14777 pts/9    0:00 forkEx2 After forking fpid=14780 $    PID TTY      TIME CMD  14781 pts/9    0:00 sh  14759 pts/9    0:00 tcsh  14780 pts/9    0:00 forkEx2 After forking fpid=0  .

  • PID is an integer.System Call: getpid() getppid() • Each process has a unique process id (PID). • Processes can obtain their parent’s PID by calling     getppid(). typically in the range 0 through 30000. • Kernel assigns the PID when a new process is created. • Each process has a parent process and a corresponding     parent process ID. • Processes can obtain their PID by calling getpid(). .

c */ #include <stdio.getpid().h> #include <sys/types. } $ cc pid.h> #include <unistd.c ­o pid $ pid pid=14935 ppid=14759 $ .h> main() {    printf("pid=%d ppid=%d\n". getppid())./* pid.

 getpid().         fpid./* forkEx3.h> #include <sys/types.    }    printf("After forking fpid=%d pid=%d ppid=%d\n".    if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n".. getppid()). getppid()). } .     printf("Before forking .\n").    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n"..h> main() {    int fpid.         fpid.         fpid. getppid()).h> #include <unistd. getpid().    fpid = fork().c */ #include <stdio. getpid().

.$ cc forkEx3.c ­o forkEx3 $ forkEx3 Before forking . Parent Process fpid=14942 pid=14941 ppid=14759 After forking fpid=14942 pid=14941 ppid=14759 $ Child Process fpid=0 pid=14942 ppid=1 After forking fpid=0 pid=14942 ppid=1 $ ps     PID TTY      TIME CMD  14759 pts/9    0:00 tcsh ..

  .System Call: wait() • wait system call allows a parent process to wait    for the demise of a child process.

h> #include <sys/types. getpid().    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n"..    fpid = fork().     printf("Before forking .\n").    printf("After forking fpid=%d pid=%d ppid=%d\n".#include <stdio. getppid()).  fpid. status. getpid().    if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n".  fpid. getppid()).  fpid.. getppid()). } .h> #include <unistd.    }    wait(&status).h> main() {    int fpid. getpid().

$ cc forkEx4.. Parent Process fpid=14980 pid=14979 ppid=14759 Child Process fpid=0 pid=14980 ppid=14979 After forking fpid=0 pid=14980 ppid=14979 After forking fpid=14980 pid=14979 ppid=14759 $  .c ­o forkEx4 $ forkEx4 Before forking ..

 execle.    Thus. execl. execve. the total number of processes in the system isn’t     changed. execv. • Six different exec functions:    execlp. . execvp.) • exec system call allows a process to choose its successor.    (see man page for more detail.System Call: exec() • exec() system call invokes another program by replacing   the current process • No new process table entry is created for exec() program.

   execl("/bin/date".h> main() {    printf("Before execing ./* execEx1..c */ #include <stdio.. 0). Sun May  9 16:39:17 CST 1999 $ .    } $ execEx1 Before execing . "date".h> #include <unistd..    printf("After exec\n").\n")..

   execl("/bin/date"..\n").    } $ execEx1 Before execing .. "date".../* execEx1.h> #include <unistd.    printf("After exec\n"). Sun May  9 16:39:17 CST 1999 $ .c */ #include <stdio.h> main() {    printf("Before execing . 0).

\n").h> #include <unistd.    }    printf("After exec and fpid=%d\n"..h> $ execEx2 Before execing . 0).c */ #include <sys/types..    if (fpid == 0) {      execl("/bin/date".    fpid = fork().fpid).    printf("Before execing .. After exec and fpid=14903 main() $ Sun May  9 16:47:08 CST 1999 { $    int fpid. "date".h>  #include <stdio../* execEx2.    } .

.    }    printf("After exec and fpid=%d\n".h>  #include <stdio./* execEx2..fpid).. After exec and fpid=14903 main() $ Sun May  9 16:47:08 CST 1999 { $    int fpid. "date".h> #include <unistd.h> $ execEx2 Before execing .    } . 0).    if (fpid == 0) {      execl("/bin/date"..    printf("Before execing .c */ #include <sys/types.    fpid = fork().\n").

mail x < letter.  User much access three files  Std input file  Std output file  Std error file  Example: ls.Building Block Primitives    User to create complex program effectively.c file3. Users capability to (1)redirect I/O.c file2. ls > ssn.c file4.  Example: grep ssn file1.c | wc -l . 2>(err)  (2)Pipe: mechanisms allows a stream of data to be passed between reader and writer processes.txt.

FD 1 – STDOUT. n>&m means redirect FD n to the same places as FD m. Eg. > file . << token . FD 0 – STDIN .      < file .means open a file for writing and truncate it and associate it with STDOUT.Means use the current input stream as STDIN for the program until token is seen. >> file . This is how we append to a file using a redirect. 2>&1 means send STDERR to the same place that STDOUT is going to.means open a file for writing and seek to the end and associate it with STDOUT. FD 2 -STDERR .means open a file for reading and associate with STDIN.

.Operating System Services    Controlling Scheduling Memory Management   Allocating to main memory Allocating to secondary memory   Process management – controlled access to devices Transparent Service – Kernel hides the type of file from user.

• Executing System call ==> User mode to Kernel mode    perform I/O operations    system clock interrupt     .ASSUMPTIONS ABOUT H/W:User mode and Kernel mode • At any given instant a computer running the Unix system    is either executing a process or the kernel itself is running • The computer is in user mode when it is executing     instructions in a user process and it is in kernel mode    when it is executing instructions in the kernel.

Example: the process status register is not capable to access from the user mode. Part of each user process.  Some machine instructions are privileged. .   Kernel runs on behalf of the user processes.  Example: virtual address space access only in kernal mode.Difference between two modes  Process in user mode can access their own instruction not kernal instructions and data of other processes but in kernal mode can access both.

Interrupts and Exceptions  Interrupts: for multitasking CPU is shared by many processes  Blocks low priority interrupts but services high priority interrupts.   Example: addressing illegal memory. executing privileged instructions and dividing by zero. .  Exception refers to unexpected events caused by process. Restart the instruction after handling exception.  Exceptions in middle of an instruction whereas interrupts in between 2 instructions.

Processor Execution Levels  Kernal prevent the occurrence of interrupts during critical activity.  Manipulating linked list pointers Machine Errors Clock Disk Network Devices Terminals Software Interrupts  High priority interrupts     Lower Priority interrupts     Masking a level will block below and allow above. .

Memory Management    Kernal resides in main memory permanently Compiler generates virtual address space Kernal Mapping virtual address to physical address .