UNIX Shell-Scripting Basics

Agenda
lWhat is a shell? A shell script? lIntroduction to bash lRunning Commands lApplied Shell Programming

What is a shell?

What is a shell?

/bin/bash

What is a shell?

#!/bin/bash

What is a shell?
INPUT

shell

OUTPUT

ERROR

What is a shell?
lAny Program lBut there are a few popular shells

Bourne Shells

l /bin/sh l /bin/bash Bourne-Again Shell

Steve Bourne

Other Common Shells

lC Shell (/bin/csh) lTurbo C Shell (/bin/tcsh) lKorn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

lC Shell (/bin/csh) lTurbo C Shell (/bin/tcsh) lKorn Shell (/bin/ksh)

An aside: What do I mean by /bin ?

l /bin, /usr/bin, /usr/local/bin l /sbin, /usr/sbin, /usr/local/sbin l /tmp l /dev l /home/borwicjh

What is a Shell Script?

lA Text File lWith Instructions lExecutable

What is a Shell Script?

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? A Text File

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

An aside: Redirection
l cat > /tmp/myfile l cat >> /tmp/myfile l cat 2> /tmp/myerr l cat < /tmp/myinput l cat <<INPUT Some input INPUT l cat > /tmp/x 2>&1

INPUT 0 env OUTPUT 1 ERROR 2

What is a Shell Script? How To Run

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? What To Do

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Executable

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

What is a Shell Script? Running it

% cat > hello.sh <<MY_PROGRAM #!/bin/sh echo Hello, world MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world

Finding the program: PATH

l% ./hello.sh lecho vs. /usr/bin/echo l% echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin l% which echo /usr/bin/echo

Variables and the Environment

% hello.sh bash: hello.sh: Command not found % PATH=$PATH:. % hello.sh Hello, world

An aside: Quoting
% echo $USER $USER % echo $USER borwicjh % echo \ % echo deacnet\\sct deacnet\sct % echo \ \

Variables and the Environment

% env [variables passed to sub-programs] % NEW_VAR=Yes % echo $NEW_VAR Yes % env [PATH but not NEW_VAR] % export NEW_VAR % env [PATH and NEW_VAR]

Welcome to Shell Scripting!

How to Learn
l man
man bash man cat man man

l man k
man k manual

l Learning the Bash Shell, 2nd Ed. l Bash Reference Cards l http://www.tldp.org/LDP/abs/html/

Introduction to bash

Continuing Lines: \
% echo This \ Is \ A \ Very \ Long \ Command Line This Is A Very Long Command Line %

Exit Status
l$? l0 is True % ls /does/not/exist % echo $? 1 % echo $? 0

Exit Status: exit

% cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh % ./test.sh % echo $? 3

Logic: test
% % 0 % % 1 test 1 -lt 10 echo $? test 1 == 10 echo $?

Logic: test
ltest l[ ]
[ 1 lt 10 ]

l[[ ]]
[[ this string =~ this ]]

l(( ))
(( 1 < 10 ))

Logic: test
l [ l [ l [ l [ -f /etc/passwd ] ! f /etc/passwd ] -f /etc/passwd a f /etc/shadow ] -f /etc/passwd o f /etc/shadow ]

An aside: $(( )) for Math

% echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0

Logic: if
if something then : # elif a contraction of else if: elif something-else then : else then : fi

Logic: if
if [ $USER eq borwicjh ] then : # elif a contraction of else if: elif ls /etc/oratab then : else then : fi

Logic: if
# see if a file exists if [ -e /etc/passwd ] then echo /etc/passwd exists else echo /etc/passwd not found! fi

Logic: for
for i in 1 2 3 do echo $i done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: for
for i in /* do echo Listing $i: ls -l $i read done

Logic: C-style for

for (( expr1 expr2 expr3 do list done ; ; ))

Logic: C-style for

LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo n $a done

Logic: while
while something do : done

Logic: while
a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a a=$(( a + 1 )) done

Counters
COUNTER=0 while [ -e $FILE.COUNTER ] do COUNTER=$(( COUNTER + 1)) done lNote: race condition

Reusing Code: Sourcing

% cat > /path/to/my/passwords <<_PW_ FTP_USER=sct _PW_ % echo $FTP_USER % . /path/to/my/passwords % echo $FTP_USER sct %

Variable Manipulation
% FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis

It takes a long time to become a bash guru

Running Programs

Reasons for Running Programs

lCheck Return Code
$?

lGet Job Output

OUTPUT=`echo Hello` OUTPUT=$(echo Hello)

lSend Output Somewhere

Redirection: <, > Pipes

Pipes
lLots of Little Tools echo Hello | \ wc -c
INPUT 0 echo OUTPUT 1 ERROR 2

A Pipe!
INPUT 0 wc OUTPUT 1 ERROR 2

Email Notification
% echo Message | \ mail s Heres your message \ borwicjh@wfu.edu

Dates
% DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date

FTP the Hard Way

ftp n u server.wfu.edu <<_FTP_ user username password put FILE _FTP_

FTP with wget

l wget \ ftp://user:pass@server.wfu.edu/file l wget r \ ftp://user:pass@server.wfu.edu/dir/

FTP with curl

curl T upload-file \ -u username:password \ ftp://server.wfu.edu/dir/file

Searching: grep
% % % % % grep grep grep grep grep rayra /etc/passwd r rayra /etc r RAYRA /etc ri RAYRA /etc rli rayra /etc

Searching: find
% find /home/borwicjh \ -name *.lis [all files matching *.lis] % find /home/borwicjh \ -mtime -1 name *.lis [*.lis, if modified within 24h] % man find

Searching: locate
% locate .lis [files with .lis in path] % locate log [also finds /var/log/messages]

Applied Shell Programming

Make Your Life Easier

lTAB completion lControl+R lhistory lcd lStudy a UNIX Editor

pushd/popd
% cd /tmp % pushd /var/log /var/log /tmp % cd .. % pwd /var % popd /tmp

Monitoring processes
lps lps ef lps u oracle lps C sshd lman ps

DOS Mode Files

l#!/usr/bin/bash^M lFTP transfer in ASCII, or ldos2unix infile > outfile

sqlplus
JOB=ZZZTEST PARAMS=ZZZTEST_PARAMS PARAMS_USER=BORWICJH sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_

sqlplus
sqlplus $USER/$PASS @$FILE_SQL \ $ARG1 $ARG2 $ARG3 if [ $? ne 0 ] then exit 1 fi if [ -e /file/sql/should/create ] then [use SQL-created file] fi l Ask Amy Lamy! J

Passing Arguments
% cat > test.sh <<_TEST_ echo Your name is \$1 \$2 _TEST_ % chmod +x test.sh % ./test.sh John Borwick ignorethis Your name is John Borwick

INB Job Submission Template

$1: user ID $2: password $3: one-up number $4: process name $5: printer name % /path/to/your/script $UI $PW \ $ONE_UP $JOB $PRNT

Scheduling Jobs
% 0 0 * 0 % % crontab -l 0 * * * daily-midnight-job.sh * * * * hourly-job.sh * * * * every-minute.sh 1 * * 0 1AM-on-sunday.sh EDITOR=vi crontab e man 5 crontab

Other Questions?
lShells and Shell Scripts lbash lRunning Commands lbash and Banner in Practice