You are on page 1of 64

CS2028 - Unix Internals

Chapter 1

Agenda

 General Overview of the Unix

 History

 System structure

 User Perspective

 Operating System Services

 Assumptions about hardware

What is Unix?

What is an operating system?

The low­level (first layer loaded) software which 
handles the interface to system hardware (input/output 
devices, memory, file system, etc), schedules tasks, 
and provides common core services such as a basic 
user interface.
all of the software that comes with a system before 
applications are installed.

3

 LynxOS. Primos. *BSD (Free. Minix. Symbian. Tru64.1/95/98/NT/XP/2000/2003.  IRIX. Windows  3. etc. QNX. UnixWare. SE­Linux. HP­UX. VxWorks. Embedded:  BeOS. AIX. eCos. TRUSIX. etc). JavaOS. RTLinux. JUNOS. What Is Unix? Examples of Operating Systems: Unix and Unix­like:  A/UX. 4 . Micro. Chorus OS. Open. VSTa.  Windows CE. VRTX. Others:  AOS.  IOS.  System V. Solaris. LynxOS. Darwin (Mac OS X). Linux. etc. QNX. Triance.  Net.  z/OS. etc. MorphOS. RTAI. Hurd. FreeRTOS.

Alpha. 5 . SPARC. IBM S/390. etc. What is Unix? A portable. Motorola 680x0. interrupts.). MIPS. and task switching). multi-tasking and multi-user operating system  Portable: runs on many different hardware architectures (Intel x86 and IA-64.  Multi-user: many users can share the computer system at the same time.  Preemptive multi-tasking: several programs can run at the same time (time slices. HP PA-RISC. PowerPC.

What is Unix? Other Features  Uses a simple. which is also a convenient job programming language. 6 .  The command prompt is a simple user process. change and port. hierarchical file system. the Unix shell. uniform file model which includes devices and access to other services in a flexible.  Written in a high-level language (“ C” ) making it easy to read.  Includes support for regular expressions which are convenient for complex searching. understand.

Why Do I Need to Know This?  Ubiquitous: Most big computers and much of the Internet infrastructure runs on some variant of Unix (SUN. etc.). SGI.  Linux is the fastest growing operating system in the market. HP.  By understanding the Unix/Linux community and its culture.  Source code availability provides the ability to “ get under the hood” of operating system design and function (and TCP/IP). 7 . it is easy to collect critical information.

HISTORY
 Multics: 1965 Bell with General Electronic Company and
Massachusetts Institute of Technology.GE645. Multiuser, Large
data storage & power, data sharing.
 UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File
system – Thompson (Space travel program) – PDP 7
 UNIX moved to PDP – 11 in 1971
 UNIX in C (Ritchie)
 1977 – UNIX ported into non PDP machine , Interdata 8/32
 BSD: 1978 Berkeley Software Distribution- UNIX system III –
V(1983)
 Commercial Vendors: Sun, HP, IBM, SGI, DEC
 GNU: 1984 Richard Stallman, FSF
 POSIX: 1986 IEEE Portable Operating System unIX
 Minix: 1987 Andy Tannenbaum
 SVR4: 1989 AT&T and Sun
 Linux: 1991 Linus Torvalds Intel 386 (i386)
 Open Source: GPL, LGPL, Cathedral and the Bazaar

Unix History
 1965 joint project between AT&T Bell Labs,
GE(General Electric company), and
MIT(Massachusetts Institute of Technology) to
develop a new OS.

 Goal : develop an OS that could provide
computational power, data storage and the ability to
share data among multiple users.

 Result: Multiplexed Information & Computer Service
- MULTICS.

9

Unix History

 1969 Bell Labs withdraws from group.

 Two Bell Lab scientists, Ken Thompson and Dennis
Ritchie, continue research. They were still left
without a “ Convenient interactive computing
service” *.

* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal,
Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.

10

Unix History  At the same time Ken Thompson wrote a game “ space travel” in Fortran to run on GECOS OS (Honeywell 635). He was told to get the game off his work computer. 11 .  The spaceship was hard to control and it was expensive to run.

 Unics (later Unix) was born as a pun on Multics. Unix History  Thompson ported the game to a little used PDP-7 computer. 12 .

Unix History PDP-7 13 .

 In 1973 entire OS ported to “ C” . 14 . Unix History  Dennis Ritchie developed “ B” . Then wrote “ C” a compiled language.

➲ Late 70s : Thompson took a sabbatical to  teach Unix at UC Berkley –          Birth of BSD Unix. 15 . Unix History ➲ Because of a 1956 Consent Decree AT&T  could not market Unix so it provided it to  academia. IBM. Sun. Introduced many new  features. ➲ AT&T Bell Labs realized the commercial  potential and began distributing System V. DEC. NCR. SGI. ➲ Commercialization of Unix (70s / 80s)     AT&T. HP.

 1994 Linux 1.02 is first released to the public. 16 . Linus Torvalds  1991 Linux 0.0 is released.

GNU/Linux OS: The Linux kernel plus utility software to provide a useful working environment. the GNU/Linux OS and lots of other software to make Linux easy to install. MIPS. The Linux kernel runs on many platforms (Intel x86 and IA-64. etc. Motorola 680x0.). IBM S/390. configure. and use (at least for the target audience). SPARC. Linux Distributions: The packaging of the Linux Kernel. Three Definitions of Linux Linux Kernel: The very low-level software that manages your computer hardware and provides a library (POSIX) interface for user-level software. Alpha. PowerPC. . HP PA- RISC.

the Linux Mascot .Tux.

What is UNIX?  Unix is a Operating System that popular since 1969. . mail.  Now Unix is maintained by Open Group. text processing packages and source code control systems  Second parts-system calls and hardware for supports to first parts  Example is UNIX system V produced by AT & T.  System divided into two parts  First parts-programs and services such as shell.

Unix Features  High level language ‘ C’  Simple user interface  Complex programs built from simpler programs  Hierarchical file system  Consistent format for files-byte streams  Consistent interface to peripheral devices. multi process system  Hides machine architecture – portable  Even though Unix is in ‘ C’ we can run other languages like cobol. java with their Compiler or interpreter. . ada. lisp. basic. fortran. prolog. pascal.  Multi user.

paging)  shared libraries  demand loading. Linux Features  “ UNIX-like” operating system  “ aims at” standards compliance  “ all the features you would expect in a modern UNIX”  preemptive multitasking  virtual memory (protected memory. applications . support. highly portable. large files  advanced networking. supports most device hardware  active development community. dynamic kernel modules  shared copy-on-write executables  TCP/IP networking  other features:  SMP support. open source  GUIs. stable. large memory. advanced filesystems  efficient. documentation.

Architecture of Unix System emacs • OS interacts directly with  sh who   the hardware  kernel date • Such OS is called  cpp   system kernel ed cc as hardware ld wc grep nroff Other apps .

• Resources: CPUs. . Network • Kernel: the memory resident portion of Unix system • File system and process control system are two major   components of Unix Kernel. Unix: Structure • Operating System: a system that manages the resources    of a computer. I/O devices. Memory.

Unix System Kernel • Three major tasks of kernel:  Process Management  Device Management  File Management • Three additional Services for Kernel: ✯ Virtual Memory ✯ Networking ✯ Network File Systems • Experimental Kernel Features: ✱ Multiprocessor support ✱ Lightweight process (thread) support .

Block Diagram of System Kernel User Programs Libraries User Level Kernel   System Call Interface Level Inter­process File Subsystem Process communication control Scheduler Device drivers Memory subsystem management hardware control hardware  Hardware Level .

Process Control Subsystem • Process Synchronization • Interprocess communication • Memory management:  • Scheduler: process scheduling  (allocate CPU to Processes) .

  .  • Kernel’s file sybsystem regulates data flow between   the kernel and secondary storage devices. File subsystem • A file system is a collection of files and directories on   a disk or tape in standard UNIX file system format.

• The kernel may resume execution of the interrupted     process after servicing the interrupt. Hardware Control • Hardware control is responsible for handling interrupts    and for communicating with the machine. . • Devices such as disks or terminals may interrupt the     CPU while a process is executing.

     exit: terminate process execution ❂ File system:     File: open. chown chmod. link. System Call • A process accesses system resources through system call. write. read. unmount. mount. close     inode: chdir. lseek. • System call for ❂ Process Control:       fork: create a new process      wait: allow a parent process to synchronize its  execution with the exit of a child process. unlink  . stat fstat     others: pipe dup.      exec: invoke a new program.

User Perspective-File system .

Architecture of Unix Unix file system root (/) bin usr dev etc var home tmp bin spool log X11R6 (users’ accounts) mail (users’ mail) 31 .

 Where it starts. holds the files necessary  to operate peripherals such as printers and  terminals. ➲ bin.  ➲ dev: short for devices. 32 .  ➲ home: contains the home directories of users  (/export/home on sun computers). usr/bin: software for the shells and  most common Unix commands. Typical Directories ➲  / :  Root of the tree. sbin.

etc. 33 . printer spool files. var: contains files that vary in size. Typical Directories tmp: holds temporary files.) etc: administrative files such as lists of user names and passwords. logs. (Mail directories.

34 . Typical Directories  usr: Contains application programs  lib: Contains libraries for programs  proc: a pseudo-filesystem used as an interface to kernel data structures.

Types  Ordinary files  Directories  Special files  Pipes . Unix File System  File system is organised as a heirarchy of directories  It starts from a single directory called root(represented by a /).

a filename(14 bytes) .an inode number (2 bytes) which acts as a pointer to where the system can find info about the file. . Directory  Directory is file containing list of files and subdirectories  It has fixed size records of 16 bytes each which contains .

 The pipe acts as a temporary file which only exists to hold data from one command until its read by another.  Ex: Special device .  ex: command1 | command2 | command3.  They are used to represent a real physical device such as a printer./dev/null(unwanted output can be redirected). Pipes  UNIX allows us to link commands together using a pipe.... tape device etc. Special Files  Special files are contained in the directory /dev. .

• A process is an instance of the program in execution. Processes • A program is an executable file. • For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ .

• For example: create two active processes $ emacs & $ emacs &  $ ps PID   TTY   TIME CMD 12893  pts/4 0:00 tcsh 12581  pts/4 0:01 emacs 12582  pts/4 0:01 emacs $ . • A process is an instance of the program in execution. Processes • A program is an executable file.

• Process information are stored in two places: ❋ Process table  ❋ User table . Processes • A process has   text: machine instructions   (may be shared by other processes)  data  stack • Process may execute either in user mode and in kernel    mode.

  • In parent process. the two processes have     identical copies of their user­level context except for the    return value pid. • The process that invokes fork is called parent process    and the newly created process is called child process. . • The syntax of fork system call: newpid = fork(). newpid = child process id • In child process. System call: fork() • fork: the only way for a user to create a process in Unix    operating system. newpid = 0. • On return from fork system call.

   if (fpid == 0) {       printf("Child Process fpid=%d\n". $ cc forkEx1..    fpid = fork(). fpid)... fpid). fpid).    } else {       printf("Parent Process fpid=%d\n".h> Child Process fpid=0 After forking fpid=0 main() Parent Process fpid=14707 { After forking fpid=14707    int fpid..  $    printf("Before forking . #include <stdio.c */ Before forking .c ­o forkEx1 $ forkEx1 /* forkEx1.     }    printf("After forking fpid=%d\n".    } .\n").

  $    printf("Before forking ..    if (fpid == 0) {       printf("Child Process fpid=%d\n".    } .h> Child Process fpid=0 After forking fpid=0 main() Parent Process fpid=14707 { After forking fpid=14707    int fpid. fpid). fpid).c */ Before forking . fpid)..     }    printf("After forking fpid=%d\n".    fpid = fork()..c ­o forkEx1 $ forkEx1 /* forkEx1.\n").. $ cc forkEx1. #include <stdio.    } else {       printf("Parent Process fpid=%d\n".

c */ Before forking .     PID TTY      TIME CMD    printf("Before forking . $ forkEx2 /* forkEx2.  14759 pts/9    0:00 tcsh    fpid = fork(). After forking fpid=14780 } $    PID TTY      TIME CMD $  ps  14781 pts/9    0:00 sh    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh  14759 pts/9    0:00 tcsh  14780 pts/9    0:00 forkEx2 $ After forking fpid=0  ....  14780 pts/9    0:00 forkEx2    printf("After forking   14777 pts/9    0:00 forkEx2 fpid=%d\n".  14781 pts/9    0:00 sh    system("ps").\n").. fpid).h>    PID TTY      TIME CMD  14759 pts/9    0:00 tcsh main()  14778 pts/9    0:00 sh {  14777 pts/9    0:00 forkEx2    int fpid. #include <stdio.  14782 pts/9    0:00 sh    system("ps").

• Kernel assigns the PID when a new process is created. • Processes can obtain their parent’s PID by calling     getppid(). typically in the range 0 through 30000. System Call: getpid() getppid() • Each process has a unique process id (PID). .  • PID is an integer. • Each process has a parent process and a corresponding     parent process ID. • Processes can obtain their PID by calling getpid().

/* pid.getpid(). } $ cc pid.c */ #include <stdio.h> #include <unistd.h> main() {    printf("pid=%d ppid=%d\n".c ­o pid $ pid pid=14935 ppid=14759 $ .h> #include <sys/types. getppid()).

 getppid()).c */ #include <stdio. getppid()).    fpid = fork(). getppid()). getpid().. } . getpid()..h> #include <sys/types.         fpid.h> main() {    int fpid.\n").         fpid.    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n"./* forkEx3.    }    printf("After forking fpid=%d pid=%d ppid=%d\n".     printf("Before forking .    if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n".h> #include <unistd.         fpid. getpid().

c ­o forkEx3 $ forkEx3 Before forking .. Parent Process fpid=14942 pid=14941 ppid=14759 After forking fpid=14942 pid=14941 ppid=14759 $ Child Process fpid=0 pid=14942 ppid=1 After forking fpid=0 pid=14942 ppid=1 $ ps     PID TTY      TIME CMD  14759 pts/9    0:00 tcsh ..$ cc forkEx3.

  . System Call: wait() • wait system call allows a parent process to wait    for the demise of a child process.

   if (fpid == 0) {       printf("Child Process fpid=%d pid=%d ppid=%d\n"..    } else {       printf("Parent Process fpid=%d pid=%d ppid=%d\n". getppid()).  fpid. status.h> #include <sys/types.    printf("After forking fpid=%d pid=%d ppid=%d\n". } . getppid()).    fpid = fork(). getpid().\n").  fpid. getppid()).h> main() {    int fpid..    }    wait(&status).  fpid.     printf("Before forking . getpid(). getpid().h> #include <unistd.#include <stdio.

$ cc forkEx4.. Parent Process fpid=14980 pid=14979 ppid=14759 Child Process fpid=0 pid=14980 ppid=14979 After forking fpid=0 pid=14980 ppid=14979 After forking fpid=14980 pid=14979 ppid=14759 $  ..c ­o forkEx4 $ forkEx4 Before forking .

 execle.    (see man page for more detail. the total number of processes in the system isn’t     changed.    Thus.) • exec system call allows a process to choose its successor. execve. . execl. • Six different exec functions:    execlp. execv. execvp. System Call: exec() • exec() system call invokes another program by replacing   the current process • No new process table entry is created for exec() program.

.h> main() {    printf("Before execing .    printf("After exec\n").c */ #include <stdio. "date"..\n").    execl("/bin/date". 0).h> #include <unistd.. Sun May  9 16:39:17 CST 1999 $ ../* execEx1.    } $ execEx1 Before execing .

/* execEx1..c */ #include <stdio.\n"). 0)... Sun May  9 16:39:17 CST 1999 $ ..h> #include <unistd. "date".    } $ execEx1 Before execing .    printf("After exec\n").h> main() {    printf("Before execing .    execl("/bin/date".

. "date". $    printf("Before execing .c */ #include <sys/types..fpid).\n").    } .    if (fpid == 0) {      execl("/bin/date".    }    printf("After exec and fpid=%d\n".h> $ execEx2 Before execing .    fpid = fork().../* execEx2. main() After exec and fpid=14903 { $ Sun May  9 16:47:08 CST 1999    int fpid.h> #include <unistd.h>  #include <stdio. 0).

c */ #include <sys/types.fpid).h>  #include <stdio. "date". 0). main() After exec and fpid=14903 { $ Sun May  9 16:47:08 CST 1999    int fpid.    if (fpid == 0) {      execl("/bin/date".    }    printf("After exec and fpid=%d\n".    } .\n").    fpid = fork()./* execEx2.. $    printf("Before execing ..h> $ execEx2 Before execing .h> #include <unistd...

 Example: grep ssn file1.c | wc -l . ls > ssn.  Users capability to  (1)redirect I/O. mail x < letter. 2>(err)  (2)Pipe: mechanisms allows a stream of data to be passed between reader and writer processes.txt. Building Block Primitives  User to create complex program effectively.  User much access three files  Std input file  Std output file  Std error file  Example: ls.c file3.c file4.c file2.

 FD 0 – STDIN . FD 1 – STDOUT.means open a file for writing and seek to the end and associate it with STDOUT. This is how we append to a file using a redirect.  << token .means open a file for reading and associate with STDIN.means open a file for writing and truncate it and associate it with STDOUT.  > file .Means use the current input stream as STDIN for the program until token is seen. FD 2 -STDERR .  n>&m means redirect FD n to the same places as FD m. < file .  >> file . 2>&1 means send STDERR to the same place that STDOUT is going to. Eg.

Operating System Services  Controlling  Scheduling  Memory Management  Allocating to main memory  Allocating to secondary memory  Process management – controlled access to devices  Transparent Service – Kernel hides the type of file from user. .

• Executing System call ==> User mode to Kernel mode    perform I/O operations    system clock interrupt     . ASSUMPTIONS ABOUT H/W:User mode and Kernel mode • At any given instant a computer running the Unix system    is either executing a process or the kernel itself is running • The computer is in user mode when it is executing     instructions in a user process and it is in kernel mode    when it is executing instructions in the kernel.

 Kernel runs on behalf of the user processes.  Example: the process status register is not capable to access from the user mode. Difference between two modes  Process in user mode can access their own instruction not kernal instructions and data of other processes but in kernal mode can access both. .  Example: virtual address space access only in kernal mode. Part of each user process.  Some machine instructions are privileged.

executing privileged instructions and dividing by zero.  Exception refers to unexpected events caused by process. Interrupts and Exceptions  Interrupts: for multitasking CPU is shared by many processes  Blocks low priority interrupts but services high priority interrupts.  Example: addressing illegal memory.  Exceptions in middle of an instruction whereas interrupts in between 2 instructions. .  Restart the instruction after handling exception.

 Manipulating linked list pointers  High priority interrupts  Machine Errors  Clock  Disk  Lower Priority interrupts  Network Devices  Terminals  Software Interrupts  Masking a level will block below and allow above. . Processor Execution Levels  Kernal prevent the occurrence of interrupts during critical activity.

Memory Management  Kernal resides in main memory permanently  Compiler generates virtual address space  Kernal Mapping virtual address to physical address .