The UNIX Operating System

TCS Confidential

UNIX Operating System

Objectives :
Learn about the Unix Operating System and develop the
skills
required to build software applications in the Unix Environment.

TCS Confidential

Contents :

1.

Introduction & History of UNIX

2.

File System

3.

Basic Utilities

4.

Shell Features

5.

Advanced Utilities

6.

Communication Features

7.

System Calls

TCS Confidential

Chapter 1
Introduction
of UNIX

and

History

TCS Confidential

Introduction and
of UNIX
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.

Evolution
Development
Standard release- AT&T versions
BSD UNIX
5
Other implementations
of UNIX
Features of UNIX
UNIX Philosophy
UNIX operating system
UNIX Kernel
Programming Environment
Design Principles
Limitations

History

TCS Confidential

Evolution

Primarily influenced by MULTICS

MULTICS
- Developed for GENERAL ELECTRIC
mainframe computer
6 system
- Interactive operating
- Batch - mode characteristics

645

UNIX Operating System

- First Version developed in 1969 by
Ken Thompson on DEC PDP Series ( PDP - 7 )
- Business, Scientific and Industrial Users
TCS Confidential

Development
*

Originally written in assembly

language for
PDP-7

Transportability facilitated by Thompson

7
who developed the
B language

B language modified by Ritchie and

renamed as C language

Thompson (1980) and others wrote

UNIX in C which could be ported to any
computer

TCS Confidential

Standard release - AT&T Versions

Version 3

Version 6

Version 7

Version 8

Written in C
Moved to PDP - 11/45 and
PDP - 11/70
Released in 1975
8 use by universities only
For
Released in 1978
First commercial version but
primarly used in universities
32V - VAX version
UNIX Support Group(USG)
takes responsibility
Research Group still busy
TCS Confidential
8

Standard release - AT&T Versions

Contd...
System III

System V
Release 1
System V
Release 2
System V
Release 3
System V
Release 4.0

Released in 1981
Commercial orientation
Over 100 manufacturers of UNIX-based
micro , mini and mainframes
UNIX9 users > 100,000

Released in 1983
Enhancement of System III

Released in 1984
Enhancements of System V , Release 1

Released in 1988

TCS Confidential

BSD UNIX
(from Berkeley Software
Distributions)
* UNIX's entry into University of California, Berkeley Campus during
1976-77
* Inputs of UCB Faculty/researchers incorporated in System V of
10
AT&T UNIX
UNIX 3BSD

Released in 1978
Developed by Bill Joy and Ozalp Baboglu
Added Virtual memory, Demand paging
to the VAX version 32V

UNIX 4BSD

UNIX 4.1 BSD

DARPA - funded Development

Provided support for DARPA TCP/IP
C-Shell , ex/vi , new compilers for PASCAL
and LISP were added
Released in 1981
TCS Confidential

UNIX 4.2 BSD

Released in 1983
10

Other Implementations of UNIX

(based on Version 7, System III, System V of AT&T UNIX or UNIX 4.2
BSD)

Xenix
IBM PC/IX
Zeus
DEC Ultrix
HP-UX
UNIX
Elxsi UNIX
Pyramid UNIX
Data General UNIX
Perkin Elmer UNIX
MassComp UNIX
NBI UNIX
Amdahl UNIX

Microsofts UNIX operating system

to run on microcomputers
IBM PC
Zilog Computers
11
DEC
HP
SUN
ELXSI
Pyramid
Data General
Perkin Elmer
Mass Computers
NBI
TCS Confidential
Amdahl
11

UNIX Features
Multitasking
Performing tasks simultaneously
rather than
sequentially
e.g., While printing a document , start
editing
another document
Each task is considered to be a process
Multi-user

Several users can use the same

computer

12

simultaneously
i.e , more than one keyboard and
terminal
can be connected to one computer
Highly cost-effective

Portability

Easier to modify the UNIX system

code for installation on a new
TCS Confidential
computer rather
1
than write from scratch a new operating
12

UNIX Features

UNIX System supplied tools

Integral utilities
Operating
Command

Basic Operation of
13
system like
interpreter or Shell

Tools

Text print formatting like nroff

and typesetting troff

TCS Confidential

13

*
*

*
*
*
*

Available on micros, minis and mainframes

UNIX Communication and electronic mail
Communication between different
terminals
hooked to same computer
Communication between computers of
different
14
sizes and types in different locations as
far away
as other countries
Incorporation of third party application
programs
Hierarchical File System
On-Line Help facility
TCS Confidential
Example : man <Command>
1
Library of application software
14

The UNIX Philosophy

Make each program do one thing well

Expect the output of one program to

become the input to another

Suited for prototype development

i.e., design and build
15 easily

Modular approach
i.e., be prepared to throw and rebuild

Usage of tools

Building essential tools

TCS Confidential

15

The UNIX Operating System

Kernel( Heart of Unix Operating System)

Shell Interpreter
Tools and Applications

Kernel

Schedules 16
tasks and manages data storage

Shell

A program that
interprets the user commands/requests
calls programs from memory and
executes them one at a time or in a

series
(called a pipe)
Tools &
OS

TCS Confidential

Specific capabilities added on to the

16

The Users
Shells and Commands
Compilers and Interpreters
System Libraries
System Call Interface to Kernel
Signal
File System
Scheduling
Terminal
Handling

Swapping

CPU
Page

17

Replacement

Character
I/O System

Block
I/O System

Demand Paging

Terminal
Drivers

Disk & Tape

Drivers

Virtual
Memory

Kernel Interface to the Hardware

Terminal
Controller

Device
Controller

Terminals

Disks & Tapes

Memory
Controller
Physical TCS Confidential
Memory
17

The UNIX Kernel

Networking
Services

File Management
And Security

Input / Output
Services

Date and Time

Services

18

UNIX
Signal
Handling

SYSTEM
KERNEL

Process
Scheduling

System
Memory
Administration
Management
and Accounting

TCS Confidential

18

The Programming Environment

*

A GOOD operating system !!

Simple yet powerful

Allows building of tools, writing of small

programs and putting them together

A rich & productive environment

19

A PORTABLE operating system !!

Written in C
Runs on a range of computers
Source code is available
TCS Confidential

19

The Design Principles

*
*
*

Designed for Programmers

Interactive
Multiple Processes can be initiated
Foreground Process
Background Process

*
*
*
*
*
*
*

20
Methods to Co-ordinate Process
Signal to communicate with processes
Files, Directories, Devices treated as files
Tree structured directories to hold files
Emphasis on program development facilities
Sources available on-line
Simplicity before efficiency
TCS Confidential

20

Drawbacks of UNIX

Designed for computer professionals

Not a real-time operating system

21

File Security depends on owner

Defective programs may keep running

TCS Confidential

21

Chapter 2
22

File System

TCS Confidential

22

The UNIX File System

1.
2.
3.
4.
5.
6.
7.
8.
9.

File System Layout

Concepts of file
Features of File system
Starting and Ending
a session
23
File Management utilities
Directory Management
File operation
Mountable File System
File security
TCS Confidential

23

File Systems in UNIX

Disks are partitioned into File Systems.
Logical blocks of 512 bytes or multiples
Arranged to form a hierarchical structure
Convenient to manage data
File System Layout
Boot
block

Super
block

Inode
list

Data
block

24

Boot block
-

generally occupies the first sector

may contain bootstrap code

Information about the file system

How large, where free and other details

Contains a list of inodes

One inode is the root inode

Contains file and administrative data

Allocated data can belong to that block
only

Super block
Inode list
Data block

TCS Confidential

24

Classification of files in UNIX

Ordinary Files

Directory Files

Special Files25

Standard Files

TCS Confidential

25

Concept of Ordinary Files

*

A File in UNIX is a Stream of bytes ( 8-bits)

Kernel does not impose a structure on files

File may contain text, data and even machine

language code

Examples
Text Files

Lines of ASCII characters

separated by a new-line

26

Commands

Sequence of commands
interpreted by UNIX text

Data

File containing data as

stream of bytes

Executable

File containing machine

language instructions

The files format remains unchanged but only the way

the program interprets it is different

TCS Confidential

26

Concept of Directory Files

*

Gathering together related files in a common place

Every file is assigned to a Directory

Directories have names (maximum of 14 characters)

A file within a Directory can itself be another Directory (A

Sub-directory)
Internally a Directory is just a file that contains
a list of file names
27
their Inode Numbers and
a pointer to the actual file on the disk

An upper limit of around 35 sub -directories

Examples
/ ( Forward slash )
Prime or Root Directory
Note - in DOS it is \ ( Backward slash )
/usr
Directory for the user
/usr/trg1
Directory usr

Sub-Directory for trg1 under

TCS Confidential

27

Concept of Special Files

* Used to reference Physical devices such as terminals, disks, floppy
drives and tape drives
*

Read and written like ordinary files

Requests cause activation of the associated Physical Device

Device Drivers associated with each file

* Types :
Character file
by
Block file
or 1024

28
Terminal (tty0, tty1 )
Disk Drives (hd0,fd0)

Transmits data Character

Character
Transmits data in 512
byte chunks

* Major and Minor numbers identify the hardware link

Major No.s : Indicates type of device

TCS Confidential

Minor No.s : Different instances of the device

28

Concept of Standard Files

*
*
*

Helps display information on the screen

Special Names for Communication channels
Keyboard input channel is called Standard
input
(stdin) file id29 is 0
Terminal Screen output is called Standard
output
(stdout) file id is 1
Diagnostic error messages (generated by a
program) are sent to Standard error
(stderr) file id is 2 ( shown on terminal
TCS Confidential
screen )
2
All three files are open by default at the time
29

File Names
*

A means for the system to identify a file

Note : Unix is case sensitive
* User accesses a file by a user-assigned
file
name
* Can be a sequence of characters (a
30 of 14 )
maximum
*
May be identified by two or more names
(Multiple links to a file)
*
Internally assigned a unique inode
number
(for a File System)
* Different files can be grouped under a
directory
TCS Confidential

30

Path Names
Absolute Path name
*
e.g.,

A file is identified by the path name from the root

/usr/trg/c/test.c
where

test.c is an ordinary file

usr, trg, c are directories
trg is a sub-directory under usr

31

Relative path name

*
*
interpretation

UNIX keeps track of the user's current directory

If a "/" does not precede a file name then the name
begins with the current directory

e.g.,
If current directory is
/usr/trg
then the file could be just referenced as c/test.c

TCS Confidential

31

Features
* Hierarchical
* Security on each file
- Owner
- Group
- All others
*

Separate security for

- read
32
- write and
- execute

Removable

File Independence -

Time stamp on each file

- Modification time
- Access time

TCS Confidential

32

Unix
dev

bin

lib
/

bin

console
lp0
tty0

src

sh
csh
--Libc.a
---

uch

33

tmp

man
---

bin

local

lib
---

usr

include

spool
etc

spell
troff
---

*.h
tmac

passwd
group
init
---

lib

troff
---

tmp
---

TCS Confidential

33

Standard File Hierarchy

/bin

Basic Unix utilities

/dev

Special I/O device files

fd0, lp0

/etc

Administrative Programs

passwd, shutdown

/lib

Libraries used 34
by Unix

libc.a

/usr/bin

Unix utilities

cal, bc,spell

/usr/adm
/tmp

Administrative commands
and files

cp, mv, ln

adduser

Temporary files created

on error conditions
TCS Confidential

34

/usr/games

Game Programs

/usr/include
procedure

Include files for language

/usr/lib

35

Examples : C-header files

stdio.h, math.h
Archive libraries
Example : trof

/usr/mail

Mail files
Example : mailbox

/usr/news

News files

/usr/spool

Spool files

/usr/tmp

Temporary files

Program Source Files

/usr/src

TCS Confidential

35

Security and Access Methods

*
*

Three levels of access

User/Owner, group, others
Three types of access on Files and Directories
Read, Write, Execute

Access Mode
Directory

Ordinary

Read

Examination36
of
File Contents

Listing of
files within
Directory

Write

Allows changing
of file contents

Creating new
files within
Directory

Execute

Executing file as
a command
allowed

Searching the
Directory

allowed
TCS Confidential

36

Starting and Ending a Session

login :
User can type his name and password to
identify himself
login command can be used as
$ exec login
to log-on onto another user account after identifying yourself
in response to prompts for user name and password
su

setuser

37

This is used to become another user or super-user provided

the password is known.
e.g.,
$su
Prompt the user for the superuser password $su - trg2
Prompt the user for the password of user trg2
"ls -l"

$su - trg2 -c

TCSls
Confidential
Temporarily changes to trg2 and executes the command
-l
and
3
comes back to the original user
37

Starting And Ending A Session

Contd.
passwd
Change the password for the user
e.g., $ passwd
Prompt you for old password and new password
38
logout
user

This command exits or logs-out from the current

and executes the file .logout before coming out
e.g.,

$ logout
or
$ exit
or
$ <ctrl-d>

exits from the current login

TCS Confidential

38

File Management Utilities

Directory
Security
Management

Operation

cd
pwd
mkdir
rmdir
mvdir

File Comp.

cmp
comm

passwd
crypt
chown
chgrp
umask
chmod

39

File
contents

File
compression

Mountable
file

cat
ls
wc
file

pack
unpack

mount
umount

Copy, Move
Remove & Time
cp
ln
mv
rm
touch

TCS Confidential

39

mkdir

creates a new directory

rm

removes a file

rmdir

removes a directory

du

displays disk usage

df

displays number of free block

touch

updates the time of last modification

find

locates files that match certain area

file

displays the type of file

pwd

displays full pathname of current

directory

40

TCS Confidential

40

Command Structure
General Structure:
Command Name

[Options]

[Arguments]

E.g.,
Command Name
Tells the shell what to do
(in this case list files)
Options
Control how the
41command will work
(in this case ask for a long listing)
Arguments
on which the command works
(in this case the directory
usr/trg )

ls

-l

/usr/trg

TCS Confidential

41

Directory Management
cd
cd..
cd.
e.g.,
$ cd /usr/trg/c
$ cd ..
$ cd ./c
or $ cd c
$ cd
mkdir
$ mkdir pathname

Change working Directory

Parent Directory
Current Directory
( current Directory is c )
( current Directory is trg )
( current Directory is again c )

42

( home directory - in this case /usr/trg)

Make a Directory

Makes Directory in 777 mode

Write permission should at least be permitted for owner in parent Directory
e.g.,

TCS Confidential

$ mkdir /usr/trg2 ( makes directory trg2 )

42

rmdir
*
rm -r

Remove a Directory
$ rmdir pathname

Directory should be empty, or else

(recursively remove)
43

e.g.,
$ rmdir /usr/trg2

(removes directory

trg2)
pwd
Directory

Print Working

TCS Confidential

43

File Contents
cat

Concatenate & Print on screen or printer

$cat [Options] [Arguments]

Options -

take input from stdin

-n
no. of output lines
-s
squeeze adj. blank lines
-v
enable display of non-printing

characters
-b

44 with -n to avoid numbering blank

used

lines
e.g.,
$ cat try.c
$ cat
displays on

Display the contents of try.c on the screen

Takes input from stdin i.e. keyboard and
screen
TCS Confidential

44

$ cat f1 > f2
$ cat f2 > f3
$ cat f4 >> f3
$ cat try[0-3] > final
$ cat test* > report

Takes input from file f1 & puts it

on file f2
f3 contains the contents of f1
Appends
the contents of f4 to
45
file f3
The file final contains contents
of try0, try1, try2 try3
The file report contains all files
beginning with test
TCS Confidential

45

Is[Options]

List the Directory Contents

Options

-1

number one single column output

-l

long format (II also used)

-a
-s

all entries including dot files

46
gives no. of disk blocks

-i

inode no.

-t

ordered by modification time recent

first recursively display all directories,
starting specified or current directory

TCS Confidential

46

$ Is -I

List the files along with the protection bits and the user

$ Is -a

List the files starting with .and..also

$ Is -1

symtab.c
symtab.o
treegen
test

$ Is -I

-rw-rr 1 smj proj1 30766 Mar 3 15:08 symtab.c

-rw-rr 1 smj proj1 8759 Mar 3 15:12 symtab.o
47
-rwxr-xr-x 4 smj proj1 40743
Mar 3 15:23 treegen
drwxrwxr-x 1 smj proj1 53 Mar 1 09:15 test

$ Is -a

$ Is -iI

..
.profile
.cshrc
symtab.c
...

10936-rw-rrI smj proj1 3076 Mar 3 15:08 test.c

10936 - inode number of file test.c
TCS Confidential

47

wc

Word Count
$wc [Options] filename

Options

e.g.,

nl
the file

Display
-I
Display
-w
Display
-c
Display
48
$ wc test.c
20 200 5678
20 - lines
200 - words
5678- characters

no.
no.
no.
no.

of
of
of
of

lines, words, characters

lines
words
characters

no. of lines in the file and temply lists out

Similar to wc -l < filename >

TCS Confidential

48

file

Determine file types

$file [Options] [Arguments]

Options

-f
filelist
Normal File Types
C49program text
assembler program text
commands text
ASCII text
English text
e.g.,

$ file test.c
C Program test
TCS Confidential

49

cp

copy a file
-i
e.g.,

In

- user interactive mode

$ cp test.c test.c.bak
test.c and test.c.bak contain the same contents
Extra disk storage
50
Create link

e.g.,
$ ln first.c second.c
The file is referenced by two different names
No Extra disk storage

TCS Confidential

50

mv

Moves or renames files and directories

-i

interactive mode

e.g.,
rm

$ mv old.c new.c
Renames the file old.c as new.c
Deletes the indicated file(s) files
rm
-i
-f
-r

touch

e.g.,

removes files and directories

remove interactively
51
forcible remove
remove recursively
Dangerous
used in conjunction with -i

Updates access, modification or change times of a file

-a
update access time
-m
update modification time
-c
prevents creating the file
$ touch f1
* The current system date & time stamp is put on the file f1
* If f1 does not exist then it is created with 0 bytes

TCS Confidential

51

File Comparison
cmp

Compare two files

If files are same no output is sent to the terminal, or else The

line number and
the byte at which the first difference occurs
is reported
-s

Outputs nothing Registers return code

Return code 52

0
1
2
e.g.,

if files are identical

if files are different
on error

$ cmp test1 test2

test1 and test2 differ in char 36 line 3
$ cmp -s test1 test2
$ echo $status
outputs 1 indicating that the files are different

TCS Confidential

52

diff

Reports more than one differences

-b
-e

convert

$diff [Options] file1 file2

Ignores trailing blanks
Gives a list of ed commands so as to
file1 into file2.

53

e.g.,

$ diff test1 test2

Outputs:

n1 a n3,n4
n1,n2 d n3
n1,n1 c n3,n4

where

*
*

n1 ,n2, n3 ,n4 are line numbers

a ,d, c means append, delete ,change

respectively
TCS Confidential

53

comm

Display common lines

$comm -[123] f1 f2
Prints a three column output:
- lines that occur only in f1
- lines that occur only in f2
- lines that occur in both
comm -12
- prints
lines common to the two files
54
comm -23
- prints only lines in the first file but not in
the
second
comm -123
- prints nothing
e.g.,
$ comm test1 test2

Reports the common lines between

files test1, test2 and reports

the lines differing

$ comm -12 test1 test2 Prints line common to both
TCS Confidential

54

File Compression
pack

Compress the file

$ pack <filename>
e.g.,

$ pack try
- Creates a file try.z which is packed
- Normally the executables are packed
- The size is reduced by 25 - 40 %

55

unpack
or
pcat
e.g.,

Uncompress packed file

$ unpack try.z
or
$ pcat try.z
unpacks the file try.z
TCS Confidential

55

Mountable File System

mount
e.g.,

umount

Associates a directory with a device

Mounting a floppy on the root file system

Dissociates directory from the device

56

e.g.,

$ mount /dev/fd096 /mnt

$ umount /mnt

Mounts the floppy on

the directory /mnt
Dissociates /mnt from
the floppy

TCS Confidential

56

File Security
passwd

chown
file

To change the password

To change
the ownership of the
57
$ chown owner filename

e.g., $ chown trg2 test.c

* Initially the owner is trg1
* Only the owner or the superuser
TCS Confidential
can
change the ownership
of the file
5
57

chmod

change the permissions of the file

$ chmod who op permission <filelist>

who
a, u, g, o
op
+, -, =
permission
r,w,x
execute

all, user, group, others

+ add, - remove, = set
r read, w write, x

58

e.g.,
$ chmod a=rw test.c
* users, group, others have read
and write permissions
$ chmod u+r, g+w, o+x test.c
* read for users write for groups execute for others
$ chmod 777 test.c
* Sets read, write, execute Permissions
TCS Confidential

58

umask

Set file creation mode mask

$ umask nnn
(nnn set file creation mode)
umask can also be set as a shell variable
59

e.g.,
umask 022
- Files normally created with 777 mode is assigned
755 permission
The value of each digit is subtracted from the corresponding "digit"
specified by the system for the creation of a file.

TCS Confidential

59

File Operation

tail

Displays the last lines of file

options : -n (n= no. of lines)
e.g., $ tail -30 test.c
Displays the last 30 lines of file test. c

head

Displays the top lines of file

e.g., $ head -10 test.c Displays the first 10 lines of test.c

split
lines

Splits the file into different files as

60

specified by the number of

e.g., $ split -20 test.c

Splits the file test.c in blocks of 20 lines and creates files xaa, xab,
xac
and so on, such that
xaa
has first 20 lines of test.c
xab
has the next 20 lines of test.c
...
The file test.c is unaffected
$ split-20 test.c try Generates files as tryaa , tryab , tryac
paste

Joins the two or more files horizontally

e.g., $ paste xaa xab
File xaa and xab are joined horizontally and output to theTCS Confidential

terminal
60

Chapter 3
61

BASIC
UTILITIES

TCS Confidential

61

Basic Utilities

1.

Line Editor

Ed

2.

Visual Editor Vi

3.

62
Debuggers

4.

Profiling Tools

5.

C-Beautification

TCS Confidential

62

The Line Editor - ed

Invoking ed:
Prompt :
Append:
Write :

$ ed filename

Ed has no formal prompt

P - Prompts with a *
a

w
Exiting ed :
q

This is a toggle function

Appends given text after current line

Writing to the file

Quit
63from ed

Example :

$ ed test.c
? test.c
P
*a
<>
<>
<>
*w
*q
The edit buffer can be written to some other file also as *w filename
TCS Confidential

63

Pattern Matching
abc

abc

a*bc

abc,

aabc,

aaa...bc

aac,

acc,

axc,

abc,

axyc,

a$+pmc,

a[b2m]c

abc,

a2c,

amc

a[0-2b-d]c
adc

a0c,

a1c,

a2c,

a[^0-2]c

a3c,

a5c,

axc, ....

a[xy]*c

ac,

axc,

axxc,

a\{3,\}

aaa,

aaaa -

3 or more as

a\{3,5\}

aaa,

aaaa,
aaaaa
between 3 & 5 as

a.c
a.*c
abcdefgc

abc,

64

a$c

abc,

acc,

ayyyc

TCS Confidential

64

Adding and Replacing Text..

Command
Usage
p
l
n

Explanation
print current line (s)
list current line(s)
(display invisible
65
print line(s) with

1,3p
2,4l
characters like tabs etc.)
number before it

1,5n
a

append text after current line

i

4a

insert text before current line

3i
TCS Confidential

65

Traversing within a file

line

Command

Explanation

Pointer to current line

.=

Gives line number

Current line is n

+n

Examples
*-

*-

Previous

Current line

Move up by n lines

66

-n

Move down by n lines

points to the last line

short for 1,$

short for .,$

/.../

forward search for string

of characters enclosed between
slashes

*$= Prints
total no. lines
but does not
changecurrent
line

TCS Confidential

66

Modifying Text
Command

Explanation

Usage
d
2,5d
c

delete line(s) in text

move line(s) in text

copy line(s) in text

change line(s) in text

2,4c
2,3m5
2,3t5
j
u

67

join lines in text

undo last command

2,3j

Note : The above commands can be easily remembered by associating

them with the
first characters of their action.

Substitute Commands
(n,n)s/oldstr/newstr
replaces oldstr with newstr (n,n) range of
lines - optional
s/oldstr/newstr/g all oldstrs' on the current line
are replaced with newstr
TCS Confidential
s/oldstr/newstr/p the replacement is only effected in print but not
executed

67

Finding Text

/^ pattern/

searches for patterns at

beginning of line

/pattern $/
line

searches for pattern at end of

/pattern/

the pattern is searched forward

//
??

68

forward search
backward search

TCS Confidential

68

Command Execution

Command

Explanation

(n,n)g/findstr/commandlist Executes given commandlist for

every occurrence of findstr
(n,n)G/findstr/

Inputs one command to execute for

every occurrence of findstr

69

(n,n)v/findstr/commandlist Executes given commandlist for

every
non-occurrence of findstr
(n,n)V/findstr/

Inputs one command to execute for

every non-occurrence of findstr

TCS Confidential

69

Special ed Commands
Command

Explanation

print current filename

write contents of buffer into file

w file2

write contents of buffer to file 2

read current
70 file

r file2

read contents of file2 after current line

e file2
!shellcmd

after current line

edit file2 > invoke ed on file2

Execute shell command by preceding with !
! ls l gives a listing of current directory

TCS Confidential

70

The ex editor

*
an improved version of ed
*
less terse
*
provides display
options like
71
numbered
line
*
allows shorthand versions of
commands
*
clear response for error messages
TCS Confidential

71

The vi - Visual Editor

The

editor
*
*
*
*
*
*

Invoking

Powerful full screen editor

vi v/s ed, ex
Mostly single key stroke commands
Interface with ex
Macro facility
Ability to process text
72
$ vi filename
e.g., $ vi pgm.c

Modes :
*
*
*

Command mode
Insert mode
From Insert mode Pressing
<ESC> remitts Command mode
TCS Confidential

72

Types of commands
* vi-commands (invisible)
Command mode
Commands can be categorised as :
* Cursor movement
* Text manipulations
73
- insert, delete, copy, change
* Marking/Selecting, Positioning
* Search
Objects of interest recognized by vi:
* characters
* words
* lines
* block
TCS Confidential

73

Format of commands
[count] command [operand]
Use . to repeat last command
Use u to undo last command
Cursor Movement
Line Oriented :
74
^ or (zero)
$
Character oriented
h
l
j
k

beginning of line
end of line
move
move
move
move

left
right
down
up
TCS Confidential

74

Format of commands (contd.)

Word Oriented :
e

move to end of next word

move to beginning of word

75 to beginning of previous word

move

move to end of next word

ignoring punctuation

move to beginning of word ignoring

punctuation

move to previous word ignoring

punctuation
TCS Confidential

75

Block Oriented :
)

move forward one sentence

move backward one sentence

move forward one para

move backward one para

move to find matching parenthesis

Rightly used
76 in C- Programming for matching (

mark a particular line with a label.

e.g., ma (marks the line with label a)

return cursor to position (m)

e.g., "a moves the cursor to the line
which was marked with label a

^g

file information, line number

& )

TCS Confidential

76

Text Manipulation
Insert mode :
a
A
i
I
o
O
esc
Delete mode :
u
nx
ndw
ndd

append after cursor

at end of line
insert at cursor
at beginning of line
enter in new line after current line
77 in before current line
enter
to exit insert mode

undo last command

delete n- next character
deleting n next words
deleting n lines current line and n-1 below current
line are deleted.
delete till end of line
TCS Confidential

77

Changing Text :

mode

nr
s

replace char
delete current char, enter insert

S
cw

delete line, enter insert mode

change word

Cut & Paste :

78

nyy

yank n line
Named Buffers : a to z ( 26 )
Unnamed Buffers : 1

print buffer

join next line to current EOL

Join :

TCS Confidential

78

Search
Searching for a character :
fc

search forward for character c

tc

similar to79
f, cursor placed 1 char left of c

Fc

search backward for character c

Tc

similar to F, cursor place 1 char right of c

continue search in same direction

continue

search

in

reverse direction
TCS Confidential

79

Search
File Related :
G

go to the end line

nG

go to the nth line

go to the middle line

go to the top line

go to the last line

80

TCS Confidential

80

Text Manipulation

Screen Adjustments :
^d

Scroll down half a screen

^u
^f

Scroll up half a screen

81
Page forward

^b

Page backward

^e

exposes one more line at bottom of

screen

^y

exposes one more line at the top of

screen
TCS Confidential

81

Text Manipulation
Searching for a string :
82

/string
?string

for searching forward

for searching backward

Use n to continue search in the same direction

Use N to continue search in the opposite direction
s/oldstr/newstr

commands can be used in vi

TCS Confidential

82

Common Set commands

set ai

Causes automatic indentation

set noai
set nu

Nullifies the effect of auto-indent

83
Causes line numbers to be displayed

set wrapmargin = n

Sets n column right margin

set bf

The beautify option Removes all

unimportant control characters

set

Causes a displays of current set options

TCS Confidential

83

Text Manipulation
:w!

file force write

:q
:q!

quit normally
84
quit, no write

:n

next file, for $ vi f1 f2 f3 f4

:n!

edit next file, discarding changes

to current file

:n args

specify new argument list

TCS Confidential

84

Debuggers :
Why ?
*
for running a Program under the control of the programmer
*
for examining the values of various variables and stack
contents
Types
fsdb

File System Debugger

Fixes damaged
85 file system

kdb

Kernel Debugger
Resides in kernel,
Allows memory examination
Disassembles instructions
Executes programs

sdb

- Debugging source code written in C, Fortran 77

- Available on System V

Role

The functionality of any debugger is essentially the same

TCS Confidential
85

Debugger - sdb
*

Invoked as
sdb <executable file>

e.g.,

$ cc -g test.c
$ sdb a.out
86

sdb checks for the status of files

sdb gives warning messages if source files

are not present or source files
modified after the object files

sdb gives a prompt * after invocation

some commands can be used to avail the

important features of sdb
TCS Confidential

were

86

SDB Commands
Command

Explanation

*r <arguments>

run the program

(arguments optional )
stepping
through the instructions
87
setting a break point at line number 20

*s
*20b

(Normally break points are set at function calls)

*/pattern
*<function-name> b
name
\*B
*d <line-no>
line no.
*D

searches for the pattern

setting the break point at the function
gives a list of break points\
deletes the break point at the specified
TCS Confidential

deletes all the break points

87

sdb Commands (contd.)

Command

Explanation

*S

stepping over the function call

*c

continue upto the break point

*variable name

prints the value of variable active at that

time

*variable name/x

prints the value of variable in hex

*variable name/c

prints the value of variable in character

*variable name/o

prints the value of variable in octal

*variable name/s

prints the value of variable in null

terminated string

*variable name/a
*function:variable

character strings at the specified address

prints the value of the variable in the
specified function

88

TCS Confidential

88

sdb Commands (contd.)

Command

Explanation

*p

print the current line of the source

*<line-no>p

print the corresponding line-no of the source

*<ctr-d>

prints next 10 lines of source or data or instructions

89
depending on the previous command

*w

prints a window of 10 lines of source or data or

instructions around the current line

*<Enter-key>

executes the previous command

*!<command>

executes the shell command

*k

kills the debugger

*q

quits the debugger

TCS Confidential

89

CTRACE - Debugger

Different from the conventional debuggers like sdb

Works in the absence of a debugger

Ctrace simulates the insertion of printf statements

Programmers way of debugging is to insert printf statements at the required places

Results in more printf statements than the program itself

Ctrace also traces infinite loops 90

TCS Confidential

90

CTRACE - Debugger
*

(Contd .)

Invoked as
ctrace [options] < source-file > traced-file
options :
-o
-x

print the variable in octal

print the variable
in hex
91

-u

print the variable in unsigned format

-e

print the variable in floating point format

-f <function-names>

trace only the selected function names

-v <function-names>
except

complement of option -f trace all functions

those specified in the list
TCS Confidential

91

e.g.,
$ ctrace < test.c > ctest.c
test.c ctest.c -

source file
traced file

Compile the traced file as

92
$ cc ctest.c
creates in a default file a.out
Run the executable
$ a.out <arguments>
arguments are optional
Outputs the values of variables and statements as
desired by the user

TCS Confidential

92

PROFILING TOOLS :
PROF

and

GPROF

Improves efficiency and debugging

gprof is available in BSD versions

*
The program should
93have been compiled with -p
option
for prof and -g option for gprof
*

prof - refers default input file mon.out

gprof - refers default input file gmon.out

*
Gives an idea as to which function used up the
TCS Confidential
maximum
time
93

Example of prof
$ cc -p test.c
$ a.out
$ prof a.out
Generates an output like....
%time
msec/call
91.2
7.2
1.0

cumsecs

name
9.34
9.34
10.08
0.74
10.18
0.10

0.0

10.24

%time

Percentage of time consumed by the function

cumsecs

Running sum or cumulative sum of the functions called

seconds

Seconds consumed by the function

0.06

seconds
20100

94

#calls

_func2
1
100
1

calls

Number of times the function was invoked

name

Function name

acnt
_func1
_main

TCS Confidential

94

CB - C Beautification
* Beautifies the C-program code
* Indents all the statements
* Improves readability
* Invoked as
95
$ cb < source-file > target-file
e.g.,
$ cb < test.c > test1.c
test.c - source file
test1.c - Beautified version of test.c
TCS Confidential

95

Chapter 4
96

Shell Features

TCS Confidential

96

Shell Features

Shell Features
1.
2.
3.
4.
5.
6.
7.
8.

Various kind of shells

Shell commands
Shell Startup Dot files
97
Shell variables
C Shell
Test conditions
Shell programming - Bourne Shell
Shell programming - C Shell
TCS Confidential

97

Various types of Shells :

1.

Bourne Shell

sh

2.

C Shell

csh

98

3.

Korn Shell

4.

Restricted Shell

5.

Visual Shell

ksh
rsh
vsh
TCS Confidential

98

Shell Commands
Simple commands using shell Metacharacters
Exit status

Redirection

file
file

Pipeline

return value 0 - Successful

return value > 0 - Failure

Default
Default
<
>
>>
n>&m

input from Keyboard

output on the Screen
- Read from a file
- 99
Redirect it to a file
- Append to a file
- Merge output from file descriptor n with
descriptor m
n<&m - Merge input from file descriptor n with
descriptor m
where n and m are file descriptors for the
stdin, stdout and stderr files
sequence of commands separated by |
TCS Confidential

99

sequential execution

&

asynchronous execution

&&
returns

following list executed if preceding pipeline

zero value

100

||
returns

following list executed if preceding pipeline

nonzero value

matches 0 or more characters

matches any single character in filenames

$var

value of shell variable var

var=variable assign variable to var

TCS Confidential

1
100

rest of the line is a comment

...

run command within backward quote

...

treats ... literally

...
... and \

treats ... literally except for $ and

101

e.g.,
$ (sleep 5 ; echo The date is `date` ) & date
The Output of the above command will be
6345
Wed Sep 23 10:20:45 EDT 1991
$The date is Wed Sep 23 10:20:50 EDT 1991
TCS Confidential

1
101

PATH
*

If full path is given while executing the command, the command is

executed from the path specified

Else it is searched in the Shell variable $PATH

102
Search order is defined by value of Shell variable Path

*
e.g.,

PATH = :.:/usr/bin:/bin:

TCS Confidential

1
102

Shell Startup Dot files

Helps in customizing UNIX System

Executes .profile for Bourne Shell when user logs on

Executes .login and .cshrc for C Shell when user logs on

Executes .logout for C Shell when user logs out from C-Shell
103
Keeps track of the history mechanism In C Shell through the .history

TCS Confidential

1
103

Shell Variables
User defined
assignment : name = value
e.g., (on Bourne shell)
$
$

x=Hello how are you

echo $x
104
Hello how are you

$
$
$

PATH=$PATH:/usr/games
export PATH
echo $PATH
:.:/usr/bin:/bin:/usr/games:

TCS Confidential

1
104

Positional Parameters
e.g.,

105

shellscript
arg2
$0

arg1
$1

$2

TCS Confidential

1
105

Evaluation of Shell Variables

* $var

value of var ; nothing if undefined

* ${var}
characters

same as $var ; useful if alphanumeric

follow variable name
- derefrencing variable

e.g.,
$
$

var=hello 106
varx=goodbye

echo $var
hello

echo $varx
goodbye

echo ${var}x
hellox
TCS Confidential

1
106

$ {var?message} If defined , $var

Otherwise print message
if message not defined print
default message
e.g.,
$
echo ${var?}
hello

107
echo ${junk?}
junk : parameter not set

echo ${junk?my message error}

junk : my message error

${var-thing}

Value of var if defined ,

else use thing
$var unaffected
TCS Confidential

1
107

* ${var=thing}
thing
e.g.,

$
$

Value of var if defined ; otherwise

$var is set to thing
echo $ {junk-Hi there}
Hi there
108
echo ${junk?}
junk : parameter not set

echo $ {junk=Hi there}

Hi there

echo ${junk?}
Hi there

* $ {var+thing}
nothing

TCS Confidential
If var defined then thing Otherwise

1
108

Command Environment
*

Variables & associated values

109

Shell 1
Shell 2

Export A

e.g.,
export A
*

Export from parent to child shell

Vice-versa not true

TCS Confidential

1
109

Predefined Special Parameters

$#

number of positional parameters

$?

110of last command

exit value

$$

process number of a process

$*

all the parameters

$!

process id of last command started with

asynchronous execution i.e., &
TCS Confidential

1
110

Variables Used by Bourne Shell in .profile

CDPATH

search path for cd

HOME

login dir

PATH

search path for commands

PS1

primary system prompt

PS2

secondary
111
system prompt

IFS

internal field separator

MAIL

files containing mail messages

TERM

terminal type

EXINIT

list of set commands for vi

TCS Confidential

1
111

The C Shell

Developed by Bill Joy ( Berkeley University

More helpful112interaction

Permits shorthand repetition

Permits aliasing
TCS Confidential

1
112

The variables can be defined using a set for

C-Shell variables
e.g., (on - C shell)
113

% set x=date
% echo $x
Wed Sep 23 12:15:20 EDT 1991

TCS Confidential

1
113

History
execute previous

Records previous commands ! is used to

command

Command Format :
! <event> [ <word> <modifier> ]
<event>
- absolute
- relative
- name
expr
114
<word>-

!1
!-3
!c
!?su?

absolute
0, 1, ...n
range
a-e
last
$
All arguments *

<modifier>

- Substitute
:s/trg1/trg
- Print only
:s/trg1/trg/p
TCS Confidential

1
114

History Mechanism
Keeps track of commands
% history

Lists previous commands

% ^old^new
e.g.,

Modify last command

115

% cd /usr/trg/SRC
% ^SRC^src
% cd /usr/trg/src
% !15
% !c
beginning with c

Run command 15
Run previous command
TCS Confidential

1
115

Changing

Commands

:S/pattern/replacement
e.g.,
% !cd:s/trg/trg1

116

cd /usr/trg1/src
% !ls : p
Prints the previous command
beginning with ls
but does not execute it
TCS Confidential

1
116

Using Arguments

!$

last argument

!^

Begin argument

!*

All
arguments
117

!10:5-9

Arguments 5 to 9 for

command 10
e.g.,
%ls !15:2-4
TCS Confidential

1
117

Alias mechanism
Short names for frequently-used long commands
e.g.,
%
%
%

alias cdms cd /usr/man/man\!$

118
cdms 2
pwd
/usr/man/man2

% alias cd cd \!*;set prompt=`pwd`>

sets the prompt to the present working directory as and
when cd is used to change the working directory.
TCS Confidential

1
118

Filename Grouping
{

%cp

Attach filenames to common root

/usr/trg4/{ac,docs,test} .
%cp/usr/trg4/ac .
%cp /usr/trg4/docs .
%cp /usr/trg4/test .
119

%ls

{/usr/bin,/bin}/{ls,more}
/usr/bin/ls : not found
/bin/ls
/usr/bin/more
/bin/more

: not found
TCS Confidential

1
119

Using C - Shell
e.g.,
% cc test.c
% vi test.c
% !c
% a.out
% !v
120
% !c - o test
% test
% car /usr/trg2/document/report
% !! : s/car/cat
or
% ^car ^cat
TCS Confidential

1
120

Variables in .cshrc of C SHELL

% set variable=value
histchars :
% set histchars=";,"
will thereafter use
; as execution character
121
, as substitution
character
history :
% set history=40
path:
% set path=:.:/bin:/usr/bin:
prompt :
% set prompt=\!>
TCS Confidential

1
121

Variables in .cshrc of C SHELL

that

setenv

used to make a variable as an environment variable so

whenever changes are made it is also reflected in
the environment like TERM , DISPLAY is used
% setenv TERM=AT386-M

filec
% set filec
Its usage :

:
file name completion
:
Predefined
% set variable
122
% unset variable

ignoreeof
% set ignoreeof
noclobber
:
redirection
% set noclobber
appends to

To ignore all eof character ^D

Abandons all command using output

which destroys an existing file and
non-existing file
TCS Confidential

1
122

Test Conditions
TESTing Strings
test - z $string String length equal to zero
test - n $string String length not equal to zero
123

test $str1 = $str2

str1 is equal to str2

test $str1 != $str2

str1 is not equal to str2

test $string

string is not a null string

TCS Confidential

1
123

To algebraically compare variable values

test $num1 -eq $num2
Other algebraic operators

-ne, -gt -ge -lt, -le

File related
$ test -f file

124
True if file exists and is an ordinary file

$ test -r file

True if file exists and is readable file

$ test -w file True if file exists and is writable file

$ test -s file

True if file exists and has size greater than zero

TCS Confidential

1
124

Shell Programming
- The Bourne Shell
Control flows

if command
then
commands
[ elif commands ] ...
[ else commands ]
125
fi
case word in,
pattern) commands ;;
pattern) commands ;;
...
esac
TCS Confidential

1
125

for name [ in word ...]

do
commands
done
while command
do
126
commands
done
until command
do
commands
done
TCS Confidential

1
126

break n
continue n

exit from for / while / until from level n

next iteration of for / while / until from level n

exit n

exit with value

export [name..]

export

127

test expr

evaluate conditions

read [arg]

read variables from stdin or Keyboard

shift n

shift positioned parameters w.r.t $1

TCS Confidential

1
127

Examples
$
>
>
>
>
>
>
>

for i in *.dat
do
size=wc -c $i
if test $size -eq 0
then
rm $i
fi
done
128

- The complete for loop can also be edited in a file and, by

granting the
execute permission on the file , the Shell program can be
invoked
- By default any shell program is parsed to Bourne shell
However, by mentioning #! /bin/sh or #! /bin/csh in the first
line of the
Confidential
shell program, it can be parsed to either Bourne shell orTCS
C-shell
resply

1
128

Example

echo If you have a TTY vt100 enter vt100

echo If you have a TTY AT386-M enter AT386-M
echo If you have a TTY sma84 enter sma84
read term
case $term in
vt1001) TERM=vt100
;;
AT386-M) TERM=AT386-M
;;
129
sma84) TERM=sma84
;;
*)
echo Invalid TERM
;;
esac
# Making it is an environment variable
export TERM
echo end of program
This can be a part of your .profile
TCS Confidential

1
129

IF

Control Flows
if (expr) then
command;
[ else if ]
command;
[ else ]
command;
endif

SWITCH

switch (string)
130

command;
breaksw

case string1:

case string2:
command;
breaksw
default :
command;
breaksw

TCS Confidential

1
130

FOREACH

foreach name (word)

commands
end

WHILE

while
131( expr )
commands
end

UNTIL

until ( expr )
commands
end
TCS Confidential

1
131

Chapter 5
132

Advanced Utilities

TCS Confidential

1
132

Advanced Utilities
1.
2.
3.
4.
5.
6.
7.
join, csplit
8.
9.
10.
11.
12.
13.
14.
15.
16.

Redirection
Pipes
Pipe fitting with tee utility
Filters
Find
Sort
Utilities that process tabular data- cut,
Ps and Kill133
Timely Execution- nohup, at, time
User backup utilities- tar, cpio, dd , doscp
Output related commands- pr, lpr,lp
Other utilities- sleep, sync, wait, clear, tr
Introduction to Make
Lint
Lex
Yacc

TCS Confidential

1
133

Redirection

e.g.,

Metacharacter

Performs

<

input

>

output write

>>

output append

$ who
trg1
trg2

tty00
tty02

$ who > who_out

$ cat who_out
trg1
tty00
trg2
tty02

Apr 8
Apr 8

09:33
11:10

134
Apr 8
Apr 8

09:33
11:10

$ cat date_out
Fri
Apr 8

14 : 30 :

10 est

$ who >> date_out

$ cat date_out
Fri
Apr 8
trg1
tty00
trg2
tty02

14 : 30 : 10 est
Apr 8
09:33
Apr 8
11:10

$ date > date_out

1983

1983
TCS Confidential

1
134

Pipes :

Metacharacter

Performs

e.g.,

piping

135
Without Pipe

$ cat /etc/passwd > temp

$ sort < temp
$ rm temp

With Pipe

$ cat /etc/passwd | sort

* No need for creation/deletion of a file
* Sorts the file passwd as per the first
entry in passwd
TCS Confidential

1
135

A UNIX Pipeline
Standard Input

>Prgm #1

Prgm #2

Prgm #3

Prgm #4

136

>Prgm #5

Prgm #6

Standard Output<
TCS Confidential

1
136

The TEE Utility :

Helps in creating the intermediate file during the

pipe
operation
e.g.,
137

$ ls *.c | tee Cflst | sort

$ cat cflst
*
*

Creates the intermediate file Cflst

Cflst contains the list of c files
TCS Confidential

1
137

FILTERS :

grep

get regular expressions only

138

fgrep

fast, several simple strings at one

egrep
powerful

extended grep, can handle more

expressions like | - or operators

time

TCS Confidential

1
138

$ grep pattern filenames

Options :

-c

number of lines matched

-i

ignore case

-n

print line with line-number

-v

139

print lines which do not match

TCS Confidential

1
139

Searching Files with grep

e.g.,
$ grep -ni func1 *.c
that match
ignoring the case

Prints all the lines and line

numbers in files *.c
140
pattern func1

$ grep * *

Search for the pattern * in all the

files

$ ls -l | grep ^d

Searches for all subdirectories

TCS Confidential

1
140

e.g.,
$ egrep int|long test.c
Searches for all those lines
containing
either int or long in test.c
141
$ egrep (^[A-Z]) testfile Searches for all the lines which
start with a capital letter
$ cat exprfile
commas
,.*,.*,

Searches for lines having at least 3

in file testfile using the

exprfile
$ egrep -n -f exprfile testfile

TCS Confidential

1
141

Find
To locate one or more files
find path-list cond-list action-list
path-list
cond-list
-type x

one or more directory names

x can be d, f, or c
d directory
f ordinary file
c character special file

142

-user name
-group name
-size n
-links n
-atime n
-mtime n
-ctime n
-perm nnn
-name fname

for a specificed user name

for a specified group name
File size n. blocks
If n follows c then so many characters
locate for file with n links
files accessed during n days ago
modified n days ago
created n days ago
permission flags match nnn
with specified filenames

TCS Confidential

1
142

actionlist
- print

display path names

- exec command

execute the unix command

- cpio device

copy the files on the tape in specified format

143

- ok command
e.g.,
find / -atime 7 -print
find / -atime +7 -print
find / -atime -7 -print

like exec , but executed interactively

will print files accessed in
exactly 7 days
will print files accessed
earlier than 7 days
will print files accessed
within 7 days
TCS Confidential

1
143

Examples
$
$
$
$
$

find
find
find
find
find

/ -atime +30 -print

/ -atime +30 -size +10 -print
/ -atime +30
144 -size +10 -ok rm -f {} \;
. -perm 0777 -print
. -perm 0777 -print -exec rm {} \;

find / -name passwd -print

Will find the passwd file

TCS Confidential

1
144

Sort

*
*
line.

Sort keys can be fields or lines

A field is a string of characters separated by a field separator or new

$ sort {-options} {+post1} {-post2} {files}

The sort key begins at post1 ands ends just before post2. There can
be several keys.
Options
b

145

ignore leading blanks and tabs

only check if input file is already sorted

dictionary order letters, digits, blanks

ignore case

ignore non-printable characters

n
r
tp
u
o filenameout

numeric sorting
reverse order
use p as separator
output only lines with unique keys
save output in filenameout
TCS Confidential

1
145

e.g.,
$ sort -t: +2n -3 /etc/passwd
Print the passwd file sorted by numeric user id
146

$ who | sort +4n

Sorts according to the login time stamp

of the user

TCS Confidential

1
146

uniq

Finds and eliminates duplicate lines in

a file and is often used with sort
$ sort <file> | uniq -c
Sorts and ouputs the
147 number of count of lines
containing unique fields
$ sort <file> | uniq -d
Gives only the duplicated lines

tsort

Accepts as input a partial ordering and produces a

fully ordered list of the items.
$ tsort psortfile
TCS Confidential

1
147

Processing Tabular Data

Cut
*

Deletes columns from a file producing a new file

with shorter lines

Cuts out selected fields of each line in a file.

*
which

Cuts columns from a table or148

fields from a file
can be of type
-

Fixed length fields

or
Delimited by some character

TCS Confidential

1
148

Cut (contd.)
cut -c list { file1 file2 ...}
e.g.,
cut -c 5-70 file1
cut would pass 5-70 characters from file1
cut -flist {-d char } {file1 file2....}

149

e.g.,
who | cut -d -f1
gives a list of user login names
cut -d: -f 1,5 /etc/passwd
gives a list of user ID and names

TCS Confidential

1
149

JOIN
*

Combines corresponding lines in two files by relating the

contents of one or more columns.

*
files

Implements a relational data base join on two tabular

-jn m

150
join
on the mth field of file n

-on.m

output mth field of nth file

n - file no.
m - field no.

-tc

use char c as separator

TCS Confidential

1
150

e.g.,
$ join -j1 4 -j2 3 -o 1.1 2.1 1.6 -t: etc/passwd
etc/group
-

joins field group id

outputs the following parameters

151

login
name

group
name

login
dir

TCS Confidential

1
151

csplit

context split

$ csplit [-k] [-f prefix] [-s] file name arg1 [..argn]

Reads file and separates it into n+1 section defined by arg1...argn
Options :
-s

152 the character counts for each file ,

Normally csplit prints
-s is to suppress this

-k

csplit removes previously created files in case of error ,

-k is to avoid this
csplit normally creates file as xx00 , xx01 ... , xx99

xx

-f prefix creates file with that prefix instead of default

TCS Confidential

1
152

csplit(contd.)
e.g.,
$ csplit -f cobol inpfile /Procedure division/
/ Para5./ /Para6./ /Para 7./
*
*
*

Creates file as cobol 00 .. cobol 03

Edit these Cobol files
Can be recombined as
$ cat cobol 0[0-3] 153
> file

csplit -k prog.c %main(% /^}/+1 {20}

*

Breaks file prog.c containing C routine

upto a maximum of 21

% expr % > no. file is created for this section

No csplit for the main routine %main%

TCS Confidential

1
153

PS, kill
$ somecommand &
5511 - pid
$ps
pid

tty

3432
5765
5511

2
2
2

$ kill 5511
$ ps
pid
3432
5985

time
0 : 24
154 0 : 03
0 : 51

tty

time

2
2

0 : 24
0 : 03

command
-sh
ps
somecommand

command
-sh
ps
TCS Confidential

1
154

$ stubborn-cmd &
pid
3432
6004
5995
$ kill 5995

tty
2
2
2

$ ps

pid
tty
3432 2
6004 2
5995 2
$ kill -9 5995
$ ps

pid
3432
6103

tty
2
2

155

time
0 : 24
0 : 03
0 : 44

command
-sh
ps
stubborn-cmd

time
0 : 24
0 : 03
0 : 44

command
-sh
ps
stubborn-cmd

time
0 : 24
0 : 03

command
-sh
ps
TCS Confidential

1
155

Timely Execution :

time

Time a command

$ time wc test.c > wc.test

real
2.0
user
0.4
sys
0.3
156
nohup
Protecting a process from hanging and quit
signals
or interrupts
The standard output is sent to nohup.out
e.g.,
$ nohup du / &
820
Sending output to nohup.out
$ logout

TCS Confidential

1
156

nice

*
*

Executes at specified priority

Default priority is 24

$ nice +n

Raise priority

$ nice -n

Lower priority

$ nice n

set
priority
157

at

Executes process at specified time

$ at 5 pm
echo ^G ^G Time to logoff >/dev/tty04
^D
$
TCS Confidential

1
157

User Backup Utilities :

tar

tape archiver
Copies files on backup medium such as floppy
or tape in tar format

tar [options] devicename filelist

Options :

158

c
r

create a new tape backup tape old files are overwritten

append files to the tape

t
x

list the names of files from backup tape

extract files from the backup medium

update a tape, if the named files are not present or have

been modified later on
verbose ; provides informational messages, such as the
name of each file as and when it is encountered

v
f

devicename use device for backup medium

TCS Confidential

1
158

tar (example)
$ tar cvf /dev/fd096 *
directory
medium /dev/fd096

copies all files from the current

onto the backup

$ tar xvf /dev/fd096 Extracts all the files from the backup
159
medium
/dev/fd096 onto the
current directory
Creates necessary directories
tar xvf /dev/fd096 try.c
/dev/fd096
tar tvf /dev/fd096

Extracts the file try.c from

Generates a file list of /dev/fd096

TCS Confidential

find / -mtime -7 -exec tar uf /dev/fd096 {} \;

All the files modified last week is backed up onto /dev/fd096

1
159

cpio

*
Copy files archives in and out
*
Bundles all the files into one package
cpio [options] file list devicename
Options
-o

Copy these files onto tap

-i
-p

Extract files from tape

160
Read a list of file or path names from the standard input

-v

display a verbose set of cpio action

-c
character information in ASCII for portability
considerations
-t

For listing files from the backup medium

-O

Append files to the end of tape to be used with -A

-d

Creates necessary directories

-k

In case of error , ignore the error and continue

TCS Confidential

1
160

cpio (Examples)

$ ls *.c | cpio -ocBv > /dev/rmt/0n

Lists the files *.c and copy onto the device /dev/rmt/0n

161
$ cpio -ocBv < filelst > /dev/rct/c0s0
Copies all the files mentioned in filelst onto the catridge tape
$ cpio -icBdv < /dev/rct/c0s0
Extracts all the files from device c0s0 and creates necessary required
directories
$ cpio -itv < /dev/rmt/0m
List the files from the tape /dev/rmt/0m

TCS Confidential

1
161

dd

convert, reblock, translate, copy a tape file.

dd {option = value }
Options

Values

if
of
ibs
obs
cbs
skip
seek
count
conv

input file name

output file name
input block size (Default 512)
162 block size(Default 512)
output
conversion block size
n records before copying from I/P file
write after n output records in O/P file
n records
To ASCII, EBCDIC, lcase, ucase (separated by
a comma)

e.g.,
$ dd if=/dev/rmt/0n of=x ibs=800 cbs=80 conv=ascii,lcase
TCS Confidential

1
162

doscp

Copy a UNIX file in DOS format

or
Copy a DOS file in UNIX format

doscp source file target file

e.g.,

163

$ doscp /usr/trg/test.c a: copies the file test.c onto the

floppy
$ doscp a:try.c .
floppy onto

copies the file try.c from the

the current working directory
TCS Confidential

1
163

Output Related Commands :

pr

prepares a file for printing

Options
-k

K col. Output

+k

from page k

-lk

set length of page to k lines

-p

pause after164
each page

-h

take next argument as header

-wk

set width to k characters

-d

double space

e.g.,
$ pr -3 d h file list f1 f2
Generates a header as file list
3 Column output
Double spacing
$ pr -5 wordlist
Generates 5 column output

TCS Confidential

1
164

pr (contd.)
$ pr myfile
prepare myfile
165 1987 myfile page 1
nov 24 : 11 : 31
$ pr -t myfile
suppresses the header
This file can be printed using
$ pr myfile | lpr
TCS Confidential

1
165

lpr

Print a file on the line printer

$ lpr myfile

prints myfile on the printer

$ lpr -c myfile

make a copy to the spool directory

166

$ lpr -r myfile myfile is removed from the spool

lp

Combines the actions of pr and lpr

$ lp myfile

prints myfile with header, date, time and

page numbering.
TCS Confidential

1
166

Other Utilities

sleep

Suspend execution

$ sleep 5

prompt appears after 5 seconds

$ sync

* updates super block

167
* writes all
disc buffers
* calls sync before stopping system to
system integrity
* saves all modifications

ensure file
$ wait
$du

waits till all background jobs are over

determines disk usage
-s total blocks
-a size of each file
TCS Confidential

1
167

df
clear
goes to
tr
e.g.,

displays no. of free blocks

df [option] file system

clears the screen and the prompt

top of screen
168
replaces specified
characters with other
characters
$ tr [a-z] [A-Z] < lfile > ufile
Replaces all small case letters to upper case letters
$ tr -d < tstfl > tstfl2
Deletes all the blank characters
$ tr -s \012 < try > try2
Removes adjacent blank lines in file try

TCS Confidential

1
168

make - An Introduction
Making & Maintaining large programs
What is make?
of programs

Program for maintaining large number

Need for make

dependencies

* Difficulties in remembering the

169

* Simple mechanism for maintaining an upversion of Programs

to-date

Characteristics

* Helps maintain large systems

system and t

* Specifies dependencies of files in the

the actions to make them

actions to

* Uses creation date to determine the

TCS Confidential
be taken

1
169

make : AN INTRODUCTION Contd...

make program takes the file named makefile or Makefile as

its input.
makefile details:
the names of the files that make up the program
170
system
their interdependencies
how to regenerate the program system
Example :
final

TCS Confidential

mod1

mod2

mod3

1
170

make : AN INTRODUCTION Contd...

Example (contd):
Makefile or makefile
final
mod1.o
mod2.o
mod3.o

: mod1.o mod2.o mod3.o

cc -o final mod1.o mod2.o mod3.o
: mod1.c
cc -c mod1.c
171
: mod1.c
cc -c mod2.c
: mod3.c
cc -c mod3.c

Run the command as ....

$ make final
use -f urflname option

If the name of the file is not

Makefile or makefile
TCS Confidential

1
171

Using the Macro facility and Inference rules

QSRC = qp1.c qp2.c qp3.c
INCLUDES = lim.h com.h
OBJECTIF = qp.o qex.o libs.a
query : $(OBJECTIF)
cc -o query $ (OBJECTIF)
172
qp.o : $ (QSRC)
cc -o qp.o $ (QSRC)
libs.a : libs(lib1.o) libs(lib2.o)
qex.o lib1.o lib2.o : $ (INCLUDES)
LIBES
= libs(libs1.o) libs(libs2.o)
prog
: $(OBJECTIF)
cc $(OBJECTIF) $(LIBES) -o prog
TCS Confidential

1
172

Lint - C Program Checker

the

Reports the various inconsistencies that can exist in a program in

following areas
. Syntax errors
. Unused variables
. Unused arguments
. Unused functions
173
. Unused return values
. Unused external variables
. Unintialized variables
. Type checking features
. Portability considerations
. Statement not reached
. Excludes goto statements
* Invoked as
lint [options] filename

TCS Confidential

1
173

Lint - C Program Checker

Contd...

Example :
test.c
#include <stdio.h>
main(argc,argv)
int argc;
char **argv;
{
int i,j,k,l,number,num2;
int int1, int2;
int arr[20];
174
scanf( %d,num2);
printf(The square root of %d is %f \n,
number , sqrt(number) );
i=arr[j++];
add(int1,int2);
}
add (int1,int2,res)
int int1,int2, *res;
{
printf("The value %d", int 1 + int 2);
}
TCS Confidential

1
174

Lint - C Program Checker

Contd...

Generate an output after invoking lint as lint test.c -lm

test.c
warning

k , l unused in function main

warning

j may be used before set

main() returns random to invocation environment

warning

175

Function argument (number) type inconsistency

scanf (arg2) int
:: format int * : test.c
Value type declared inconsistently
sqrt llib-lm : test.c
Function argument ( number ) used inconsistently
sqrt ( arg1 ) : test.c
Function called with variable number of arguments
add
: test.c
Function returns value which is always ignored
printf
scanf

TCS Confidential

1
175

Lex

source * General tool for separating character strings

patterns in an input
* String patterns can be specified using rules
176
syntax

definitions
% %
rules
% %
user subroutines

TCS Confidential

1
176

Lex Contd...
Example (on rule)
Expression > integer * integer
Expression > integer + integer
Expression > integer - integer
Expression > integer / integer
The lex specification file
%%
[0-9]+
[-*+/]
[\t ] ;

177

{ return (INT); }
{ return (OPR); }
{
printf(Lexical analyser error\n);
exit(-1);
}
TCS Confidential

1
177

Lex Contd...
%%
yywrap()
{
return(1);
}
yywrap() indicates no further
178input available
Invoked as
$ lex filename
$ lex lex.l
Generates
lex.yy.c
TCS Confidential

1
178

Yacc :

What is it?
Features
program

Yet Another Compiler Compiler

* General tool describing input to a
179

* Specify input structure

* Code for each structure
* Outputs a subroutine which handles the
input
TCS Confidential
* User supplied routine to supply next

basic item

1
179

Yacc Contd...
Specifications

Example

Declarations
%%
rules
%%

%token INT OPR

%start expr
%%
180
expr : INT OPR INT
{
printf(The input expression is correct\n);
}
| error
{
printf (The input expression is wrong \n);
}
%%
Invoked as
Generates

$ yacc filename
$ yacc yac.y
y.tab.c
TCS Confidential

1
180

A Program using both lex & yacc

Consider this program...

#include <stdio.h>
main()
{
yyparse();
}
181
#include y.tab.c
#include lex.yy.c
The whole program can be compiled as ...
$ cc main.c -ll -ly
-ll and -ly to link lex and yacc library
$ a.out
TCS Confidential

1
181

Chapter 6
182

Communication Features

TCS Confidential

1
182

Communication Features

1.

2.

User communication commands

mail
write
talk
post
news
mesg
183
Networking commands
uucp
uuto
rlogin
telnet
ftp
rcp
uux
ct
cu
TCS Confidential

1
183

User Communication Commands :

*

Mail

Write

Talk

Post

News

Mesg

184

TCS Confidential

1
184

Mail

Sends and receives mail messages between

users
Sending mail on local system
At the end of the message press ^D to exit
$ mail <loginnames>

185

Example : (Assume the current user is trg)

$ mail trg1
Sending mail on remote system
$ mail recipient@remote_system.domain_info
TCS Confidential

1
185

Receiving mail
$ mail

Shows the message received

?
q
s [file]
w [file]
d
n or +
r
m [user]
! cmd

Prompt
quit
save (defalut mbox)
write without header
delete
next message
186
previous
reply to the sender
mail to user
run the shell command

* The message could be saved or deleted

* Environment could be setup in .mailrc

TCS Confidential

1
186

User Communication commands Contd...

Write

Allows sending messages to a user terminal

from
another terminal
187
The message
is ended by pressing ^D.
$ write <username>

Talk

Allows interactive dialogue between users at

two
different

terminals.

$ talk <username>
TCS Confidential

1
187

Post
Messages can be posted on a bulletin board for
previewing by all users.
News

first

*
Keeps the user informed of current events
Refers file /usr/news

e.g.,

$ news

Prints all the news , latest

$ news reorg Prints

188 the news named reorg
mesg The terminal can be write protected to prevent other
users from
writing onto your terminal
e.g.,

$ mesg [option]
option : - y
write allowed
-n

mesg is y or n

$ mesg

remove write permission on the

terminal
prints the current status whether
TCS Confidential

1
188

Networking Commands
*

uucp

uuto

telnet

rlogin

ftp

rcp

uux

ct

cu

189

TCS Confidential

1
189

uucp

* allows copying files from one unix system

to
another
* Refers file uuname and uulog
uucp [options] source-files destination files
options

190
-d make necessary
directory
-c do not copy local file to spool directory
-m send mail to the requester when the
copy is completed

$ uucp -d -m file? sys_2!/usr/spool/uucppublic

Copies the files file? creating
appropriate
directories onto systemTCS Confidential
sys_2 onto
1
190
/usr/spool/uucppublic

uuto

*
*

Simplified version of uucp

Sends file to /usr/spool/uucppublic

$ uuto filename(s) system!login

191
$ uuto -m test.c
sys2!trg2
Sends a mail to the sender when the job is over
rlogin

*
Allows remote login over the network
*
Similar to telnet, but a much more flexible
facility.
$ rlogin system_name!username
TCS Confidential

1
191

Networking Commands Contd...

ftp

*
*

File transfer protocol

Transfers files from one system to another

$ ftp
ftp > open sys1

Name login:
passwd
192

ftp > ?

Gives you the help

ftp > get filename

Gets the file onto the current system

ftp > put filename

Puts the file onto the \

sys1

ftp > close

$
TCS Confidential

1
192

rcp

Similar to ftp, but a much more flexible

facility
uux
remote

* Unix to Unix command execution on

systems
193 various computers, run a
* Gather files from
command on a specified computer and

redirect the
stdout to a file on a specified computer
$ uux [options] commandstring
All shell special characters must be quoted

$ pr minutes | uux -p host!lp

TCS Confidential
Command line queues the file minutes to be printed
1
on the area of the computer host
193

ct

*
*

Connect terminal to remote terminal

Connects your terminal to remote
terminal which is equipped with a

modem
This command dials the 194
phone number of the modem
$ ct [options] telno
$ ct -h -w5 -s1200 9=5553497
1200
is not

ct will call modem using a dialer operating at a speed of

baud , wait for 5 minutes before quitting if dialer
Confidential
available and use the -h option not to disconnect TCS
the

local

1
194

cu

*
*

connects local system to remote system

allows you to be logged on to both

simultaneously
Examples
cu -s2400 9=5557867

Connected

195

login :
~%take proposal
computer to local
~%put minutes minutes.bak
file minutes on the
minutes.bak

Takes proposal from remote

computer
Vice versa of take and puts the
remote computer as

~%!comma

run the command on local system

~%$command
output

run the command on local and send the

to the remote terminal

~%.

terminate the link

TCS Confidential

1
195

Chapter 7
196

System calls

TCS Confidential

1
196

System Calls :
1.

Introduction

2.

File related system calls

open()
read()
write()
creat()
chmod()
chown()
197
lseek()

3.

Process related system calls

fork()
getpid(), getppid(), getpgrp()
wait()
execl()

TCS Confidential

1
197

Library functions and system calls

Both are C - functions

*
Difference lies in their incorporation in the UNIX
System
*

Library functions are referred to as add-ons

System calls are part of 198

the UNIX Kernel

*
be

Library functions themselves use system calls and can

System Calls generally common across UNIX versions

System calls share the concept of fd -file descriptor

File descriptor is an integer used to identify a file

expanded by the user

TCS Confidential

1
198

Basic tasks in file operations

-

opening files
reading files
writing in files
creating files
199
changing the permission of files
changing the owner and group of files
seeking to file-location
closing files

TCS Confidential

1
199

* open() an existing file

open ()

int open (filename,mode)

char *filename;
int mode;
filename - character pointer to the name of the file
mode - integer signifying the mode
0 for read
1 for write
200
2 for read and write
open()

returns the file descriptors on success and returns -1 on error

Example :
Opening the file test in read mode and checking for the error condition
fd = open(/usr/trg/test,0);
if ( fd == -1 )
{
printf(error in opening file test);
exit(1);
}

TCS Confidential

2
200

read ()
*
*

File should be opened in read mode

read() an opened file
int read (filedesc,buffer,nbytes)
int filedesc;
char *buffer;
int nbytes;

201

filedesc

File descriptor indicating which file to be read

buffer
read

An area of buffer storage for holding the Characters

nbytes

number of characters to be read at a time

read() returns the number of characters read and 0 in case ofend of file (EOF)
and returns -1 on error
TCS Confidential

2
201

Example :
Reading the file test 100 characters at a time
while ( ( n = read(fd,buff,100)) > 0 )
{
printf(file test has contents %s ,buff);
}if ( n == 0 )
printf ( End of file );
if ( n == -1 )
printf (Error in reading202
file test);
When each read is finished the pointer advances by 100 bytes so
that the next read picks from there
*

If the number of characters left are less than nbytes ( in this

example - 100 ) then read() will pick up what is left over

TCS Confidential

2
202

write ()
*

File should be opened in write mode

write() to an opened file

int write (filedesc, buffer, nbytes)
int filedesc;
char *buffer;
int nbytes;

203

filedesc -

File descriptor indicating which file to be written

buffer
file

The function takes from buffer and writes them to indicated

nbytes

number of characters to be written at a time

write() returns the number of characters written and returns -1 on error

TCS Confidential

2
203

Example :
while ( ( n = read (fd,buff,100)) > 0 )
{
n1 = write (1,buff,100);
/* writing to standard output */
/* file id - 1 is for stdout */
if ( n1 == -1 )
printf (Error in writing on stdout");
}

204
if ( n == -1 )
printf (Error in reading file test ");

TCS Confidential

2
204

creat ()
*

creat() creates a new file or overwrites on the existing file

int creat(filename, mode)
char *filename;
int mode;

205

filename -

character pointer to the name of the file

mode

Integer signifying the mode

The mode is specified in octal code

creat returns the file descriptor on success and returns -1 on error

TCS Confidential

2
205

creat ()

Contd...

Example :
umask(0000);
fd = creat(newfile,0666);
if ( fd == -1 )
{
printf(error in creating
206file newfile );
}
Creates a file called newfile in mode 0666 i.e., read and write
permissions for owner, group, and others
Note : while creating a new file ensure umask is set to zero
Otherwise,
If umask had been 0022 in the environment variable, then the
effective permission would be
mode & ~0022

TCS Confidential

2
206

chmod ()
*

chmod() set permissions for the file

int chmod (filename, mode)
char *filename;
int mode;

207

filename -

character pointer to the name of the file

mode

Integer signifying the mode

The mode is specified in octal code

chmod() returns 0 on success

and returns -1 on error
TCS Confidential

2
207

Example :
ret = chmod(test.c,0600);
if ( ret == -1 )
{
printf(error in changing the file permission);
}

208

Changes the permission of file test.c

i.e., read and write permissions for owner

TCS Confidential

2
208

chown ()
*

System call chown()

chown() set ownership for the file

int chown (filename,owner
,group)
209
char *filename;
int owner , group ;

filename
the
owner

character pointer to the name of

file
owner id

group
group id
chown() returns 0 on success and returns -1 on error
TCS Confidential

2
209

chown ()

Contd.

Example :
ret = chown(test.c,0,1);
if ( ret == -1 )
{

210

printf(error in changing the owner and group of the file);

}
Changes the owner and group of the file test.c as root and others
respectively

TCS Confidential

2
210

lseek ()
* lseek() changes the position of read-write pointer for the file descriptor
int lseek(filedes, offset, origin);
int filedes, origin;
long offset;

211 on success
lseek() returns new-value of the pointer
returns -1 on error
The value of the pointer depends on origin :
0 set the pointer to offset bytes from the begining of the file
1 increment the current value of the pointer by offset
2 set the pointer to the size of the file plus offset bytes

TCS Confidential

2
211

lseek () Contd...
Example :
ret = lseek(fd,1000,0);
if ( ret == -1 )
{

212
printf(error in seeking to the 1000th byte of the file );

}
lseek(fd,1000,0) skips the first 1000 bytes of the file and starts
reading from the 1001th byte

TCS Confidential

2
212

fork ()
* fork() creates a new process which is a child process
* Child process is a logical copy of the parent process
* Parents return value is the process id of the child
* Childs return value is 0

213

TCS Confidential

2
213

getpid () getppid () getpgrp ()

* getpid() returns the process id of the calling process
* getppid() returns the parent process
id of the calling process
214
* getpgrp() returns the process group of the calling process

TCS Confidential

2
214

EXAMPLE
fork () , getpid () , getppid (), getpgrp ()
#include <stdio.h>
main()
{
int id ;
int pid , pgrp ;
215
int ppid ;
id = fork();
printf (PPID - %d PID- %d id - %d \n, getppid() , getpid() ,id );
printf (PGRP - %d \n , getpgrp() );
}

TCS Confidential

2
215

EXAMPLE
fork () , getpid () , getppid (), getpgrp ()
Output
PPID - 371 PID - 372 id - 0 - From child Process
PGRP - 136
PPID - 136 PID - 371 id - 372 - From Parent Process
PGRP - 136

216

PID - Process-ID
PPID - Parent Process - ID
ID - Returned value from fork()
PGRP - Process-group-ID
If PID is equal to the process PGRP then the process is the group leader
TCS Confidential

2
216

wait ()

&

execl ()

* wait() causes a parent to stop running and await the termination of a child process
* execl() overlays the original process with a new set of instructions
Example on execl()
217
#include <stdio.h>
main()
{
int id;
printf ( Parent process \n);
if ( ( id = fork() ) == 0 )
{
printf(Statement from child process\n);
execl(/bin/date,date,0);
}
printf( Parent process again \n);
}

TCS Confidential

2
217

EXAMPLE ON execl()
Output
Parent Process
Statement from child process
Parent process again
Tue Sep 10 11:34:17 1991
Process forked two processes and parent process avoided execl() to print the
final statement i.e parent process did not wait for the child to finish
To make the parent wait for the child to finish - wait() can be used
The example on execl() gets modified
218 as
#include <stdio.h>
main()
{
int id;
printf ( Parent process \n);
if ( ( id = fork() ) == 0 )
{
printf(Statement from child process\n);
execl(/bin/date,date,0);
}
wait();
printf( Parent process again \n);
}

TCS Confidential

2
218

OUTPUT OF THE MODIFIED EXAMPLE

Parent Process
Statement from child process

219

waits for the

child to finish

Tue Sep 10 11:34:17 1991

Parent Process again

TCS Confidential

2
219

220

TCS Confidential

2
220