Introduction to Unix CS 21

Lecture 4

Lecture Overview

Useful commands that will illustrate

todays lecture
Streams of input and output
File redirection
Piping between different programs
Putting it all together for some powerful
tools

Good Commands To Know: wc

Stands for word count

Will report the number of lines, words, and

characters in a file
Works with all sorts of files, but only
makes sense with text files

Useful for all sorts of applications

Either file direction or piping is required

Example Of wc

sort

Will sort the lines of a file into

alphabetical order
Useful on data files or output from
programs
Usage: sort [OPTIONS] [FILE]

Example Of sort

uniq

Will go through a file and remove

duplicate copies of a line

Will only remove duplicate copies of lines

that are adjacent

Useful on files that might have repeated

unnecessary data
Usage: uniq [OPTIONS] [FILE]

Example Of uniq

Wouldnt It Be Nice If We
Could sort And uniq A File?

No command exists that does both

By the end of the lecture, we should
see if this is possible

Do you think this will be possible?

Hint: Remember Unix was designed so that
tasks could be easily automated and so
that tasks could work together

What Exactly Is Input And

Output?

Input

Everything a Unix program reads

Usually from the keyboard

What you type

Output

Everything a Unix program writes

Usually to the screen

What you see

Streams Of Input And Output

stdin

stdout

Input from the keyboard

Output to the screen

stderr

Error messages that show up on the screen

stdin

Everything you type into the terminal

The programs mostly expect all of their
input from the user (you) typing at the
keyboard

stdout

The results of a running program

Goes to the terminal you ran the
program from

stderr

Appears the same location as stdout

Used for error messages

Allows for the error messages to be

separated from normal output

Redirecting These Streams

Deciphering echo hi > helloFile

>

Sends all output that normally would go to

stdout into a file
Completely overwrites whatever file was
there or creates a new file if none existed

Example Of >

A Tricky Catch To Remember

You must be careful because > will

overwrite files

Redirecting stdin

<

Makes all of the input for a file come from

a file as opposed to the keyboard
Allows you to input a lot of data without a
lot of typing

Exercise: Go to lab and run cat without

any parameters

Example Of <

Redirection Of Output Without

File Overwrite

What if you want to add information to

a file instead of overwriting it?

>>
Appends all output of a program to a file
Will create a file if it doesnt exist, but
wont overwrite any information currently
in a file

Example Of Append
Redirection

One More Stream

Dont forget about stderr

2>

Not some weird smiley

This says redirect stderr to a file
Important to note why this notation exists

0 = stdin
1 = stdout
2 = stderr

Example Of Redirecting stderr

Example Of The Difference

Between stdout And stderr

Is This Ever Useful?

Short answer: Yes!

Longer answer:

Whenever you want to save the output of

a program, redirect it
Whenever you want to avoid typing input,
redirect it
Whenever you want to isolate any error
messages, redirect it

Possible Uses For File

Redirection

Saving data generated by one program

for use by another
If you continually type the same input
into different programs, place that data
into a file and redirect it into the
different programs

How To Remember Which

Arrow Does What

cat < inputFile

Think of this as the inputFile dumping all of

its contents into cat
In other words, inputFile is going into cat

cat > outputFile

In this case, cat is dumping its results into

outputFile
In other words, cat is going into outputFile

What About Other Command

Flags?

The input or output file must come directly

after the redirection operator

Example:

Right: ls l * > listingOutput

Wrong: ls > -l * listingOutput
Question: How do you think this line is interpreted?

As a general rule of thumb, redirection should

come at the end of a line

Redirection should come after all arguments to the

program to avoid problems like the above

Combining These Flags

Entirely possible to have both < and >

on the same line
Each redirection operator assumes the
next argument is what is being
redirected
Question: Can you have > and >> on
the same line?

Answer

What Does This Command

Do?

ls /bin/doesntExist 2>> errorFile

Appends stderr to the end of errorFile

What About These

Commands?

Piping: Tying Programs

Together

We can combine some of the previous

commands together with a pipe
Pipes feed the output of one program
into another
|

Yes, thats a vertical bar

What Does A Pipe Look Like?

sort And uniq

Answering our previous question, we

see that it should be possible to both
sort and uniq a file at once using pipes
How exactly would we do it?

sort And uniq With Redirection

Same Example With Piping

How To Read Pipes

You can have more than one pipe per

line if youd like
Always read left to right, and output
flows left to right

Putting It All Together

What if we want to have both file

redirection as well as piping?
Well, things start to look ugly, but this
is entirely doable

Complex Example

Really Complex Example

Is All This Stuff Worth It?

Well, lets compare this with how you do

things in Windows

With piping, several tasks can be combined and

run in one single step
What if I wanted to sort and uniq a Word file?

Open the file and highlight all of the text

Select Table->Sort (not intuitive)
Specify how to sort and click o.k.
Word doesnt have a uniq feature