Beruflich Dokumente
Kultur Dokumente
A hands-on approach
Roger Hill (unixman@charter.net) Independent author 12 October 2010
If you're new to Linux or UNIX administration and want to get up to speed on bash scripting techniques, or you're a Windows engineer running something like a Cygwin UNIX sub-shell on your system, you need to know the hows, whys, and how-to's for bash shell scripting. Learn everyday usage of bash on a UNIX or Linux system; see how to become a bash power user by chaining bash commands together; and dive into variables, syntax structure, and loops in bash. A UNIX shell is essentially the API between the user, the kernel, and the system hardware. The shell is very important on any UNIX or Linux system and is one of the most vital aspects to learn proper systems administration and security. Typically driven by a CLI, the shell can literally make or break your system. The open source bash shell that this article examines is one of the most powerful, practical, and extensible shells available. In this article, you will learn the basic techniques of bash scripting, its everyday uses, and methods for employing it to create nearbulletproof shell scripts.
developerWorks
ibm.com/developerWorks/
Bash command syntax and keywords have taken and improved upon the architecture and technical details of the Korn shell (ksh) and the C shell (csh). In addition, bash's syntax has many extensions that other shells lack. Integer calculations are more efficiently completed in bash than in other shells, and bash can redirect standard output (stdout) and standard error (stderr) more easily than older shells. Bash is also ideal for secure environments, having a restricted start mode that can confine a user's ability within the shell to a brief, determinable list of commands. Open to modification, you can edit your own set of bash shell login control filesnamely hidden files, such as .bashrc, .bash_profile, .bash_logout, and .profileto customize your login shell.
Now, take a look at a typical Linux users .bash_profile (Listing 1) and .bashrc (Listing 2) scripts. These scripts are loaded from the user's home directory.
Page 2 of 18
ibm.com/developerWorks/
developerWorks
In the .bashrc file in Listing 2, some user aliases are configured, and the global bashrc file is loaded if it exists.
The system level process init spawns the getty or agetty process. This process in turn calls the / bin/login program, which presents the familiar UNIX or Linux login prompt. The user proceeds to login to access the system, at which point the login program spawns a new bash shell. When a bash shell user logs in, the following events occur in this order: the global profile is read; the user's individual bash profile is read; and then the user's personal '.bashrc' is read, which will typically set aliases, define additional functions, and define the user's individual environmental variables. Lastly, the bash user is presented with a bash shell prompt, and the motd file is read and displayed. Figure 2 below is an illustrated example.
Page 3 of 18
developerWorks
ibm.com/developerWorks/
The user then has a standard command set of programs available to him or her from within your bash shell environment $PATH variable. If a command isn't in the user's $PATH, but he or she has permissions to execute the command, you must use the full path, as shown in Listing 3.
This issue comes up when the binary program ifconfig is not within the users defined PATH variable. However, if you know the full path of the command, you can run it as shown in Listing 4.
Listing 4. Using the full path of a command to overcome a $PATH issue in the bash shell
[fred.smythe@server01 ~]$ /sbin/ifconfig -a eth0 Link encap:Ethernet HWaddr 00:50:56:96:2E:B3 inet addr:10.14.33.60 Bcast:10.14.33.255 Mask:255.255.255.0
Listing 5 shows a method using an alias to overcome the $PATH issue. Within your bash scripts, you may want to run commands with the full path, depending on who will run the intended script.
Page 4 of 18
ibm.com/developerWorks/
developerWorks
Forking
When a command or the bash shell itself initiates (or spawns) a new shell sub-process to perform a task, it is known as forking. When this new processthe child processis executing, the parent process will still be running. If a parent process should die before the child process, the child process can become a defunct processalso known as a zombie process which usually results in a hung process or hung application. Thus, this hung process must be killed or terminated abnormally. Although a parent process can access the process ID of its child process and can thus pass arguments to it, the reverse is false. When a shell script process exits or returns to the parent process, the exit code should be 0. If it is anything else, there was likely an error or problem with the process. The exit code of the last command executedecho $?is shown in Listing 6.
Listing 7 illustrates a parent and child process shown within a bash environment.
Page 5 of 18
developerWorks
ibm.com/developerWorks/
:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip= 01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio= 01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35: MAIL=/var/spool/mail/fred.smythe PATH=/usr/local/bin:/bin:/usr/bin:/home/fred.smythe/bin INPUTRC=/etc/inputrc PWD=/home/fred.smythe LANG=en_US.UTF-8 SHLVL=1 HOME=/home/fred.smythe LOGNAME=fred.smythe SSH_CONNECTION=10.14.43.183 56513 10.14.43.43 22 LESSOPEN=|/usr/bin/lesspipe.sh %s G_BROKEN_FILENAMES=1 _=/bin/env
In the listing, commands were issued one at a time. But, you could have run them together using the semi-colon (;) separator, as shown in Listing 10.
On the bash command-line, command completion shortens the typing needed for everyday tasks. Simply type the beginning of a command, then press the Tab key. Note that if you cannot access the command or file because of permissions restrictions, command completion won't function for you.
ibm.com/developerWorks/
developerWorks
The bash shell contains two types of commands: builtins, which are internal, and external programs (or external filters and commands), which are typically self-contained binary program files. Listing 15 shows how to find the built-in commands in bash using the alias command.
To identify a specific command in bash, use the type command, as shown in Listing 16.
Page 7 of 18
developerWorks
ibm.com/developerWorks/
Listing 16. Finding the built-ins commands using the type command
[root@apache-02 htdocs]# type lsof lsof is /usr/sbin/lsof [root@apache-02 htdocs]# type alias alias is a shell builtin
Listing 17 provides an example of lsof, an external command. This command is actually a binary file residing in the Linux file system; it's installed via a package of the same name.
Here's an example of a for loop written on the fly (shown in Listing 19). Note that here, you're typing interactively at the shell prompt; at each >, you type the next line of your interactive shell script.
Note that alternatively, you could have run this code as sequential commands separated by semicolons.
Using keywords
The command for is not a program but a kind of special built-in known as a keyword. The list of keywords in the average distribution of bash on Linux are shown in Listing 20.
Page 8 of 18
ibm.com/developerWorks/
developerWorks
You should avoid these keywordsor what are known as bash shell reserved wordswhen choosing names for shell variables.
You can use complex combined command lines, for example, to find Apache permission-denied errors by searching within all found compressed error logs and counting the number of errors.
$ find ./ -name 'error_log.*.gz' | xargs zcat | grep 'Permission denied'| wc -l 3
developerWorks
DATEFMT=`date "+%m/%d/%Y %H:%M:%S"` echo "$DATEFMT: My message" | tee -a /tmp/myscript.log
ibm.com/developerWorks/
If your scripts write to a log file, always create a new log file and include the date, hour, minute, and even the seconds in the new log's file name. Then, with a simple find command, you can easily rotate and compress your script's own logs each time you run it:
DATELOG=`date "+%m%d%y"` LOGFILE="/data/maillog/mail_stats.log.$DATELOG" # gzip the old mail_stats logs, older than 7 days find /logs -type f -name "mail_stats.log.????????????" -mtime +7 | xargs gzip -9 # remove the old gzipped mail_stats logs after 30 days find /logs -type f -name "mail_stats.log.*.gz" -mtime +30 -exec rm {} \; # mail_log utility log resets echo "" > /var/log/mail_stats.log.$DATELOG
Build error-checking logic into your scripts and never assume anything. Doing so will save a tremendous amount of pain and grief later. Incorporate the use of functions and shell script libraries (by sourcing in another script) where possible within your script. Doing so reuses tested and proven code and will save you from repeating script code and making potential errors. Filter input parameters from users:
NUMPARAMETERS="$#" if [ $NUMPARAMETERS != 3 ];then echo "Insufficient number of parameter passed! echo Please run script in format ./myscript.sh $1 $2 $3 exit 2 fi
Consider adding a debug mode or function within your scriptsay, with the set x command. Add functions to alert or send a trap for particular events within your script. You can do this via SNMP commands or an audible bell (echo x), and then send an email message with the mail command. If you're writing a script that will be used like an interactive menu, consider the user's environmental shell and the commands they have access to. If in doubt, use full paths for all commands within your scripts. Add individual and unique return codes to your bash shell script. When you write a large script, you'll be able to easily identify exactly where an error or problem is occurring by pinpointing the return code:
if [ $ERRCHECK != ];then echo Error Detected : $ERRCHECK ! Cannot continue, exit $EXITVAL value exit $EXITVAL fi
Test your scripts thoroughly in a lab environment for all possible conditions. Have other users perform testing on the scripts, as well, and intentionally try to "break" them. If your script is operating on input data from users or a data input file, always thoroughly filter, check, and validate the input data. Scripts that work from a data list can be run on multiple, different sets of data lists, as desired. For scripts that will run for a long time, consider placing a timeout function within the script to terminate or stop after n number of minutes:
stty icannon min 0 time 1800
ibm.com/developerWorks/
developerWorks
For scripts you write for a specific purpose, you may want to add an interactive warning prompt message to give the user information about the script's purpose. For example, the script in Listing 23 retrieves remote logs and creates a report email.
For an improved version of the same script with more robust features, download the source code for this article.
Page 11 of 18
developerWorks
# if the variable $F is not ALREADY set to a value, assign "String Value 4" ... F=${F:="String Value 4"} echo echo echo echo echo echo "A=$A" "B=$B" "C=$C" "D=$D" "E=$E" "F=$F"
ibm.com/developerWorks/
exit 0 $ ./a.sh A=String B=String C=9675 D=96.75 E=String F=String Value 1 Value 2 Value 3 Value 4
To use a function within your bash shell script, you use the methods shown in Listing 26.
Page 12 of 18
ibm.com/developerWorks/
developerWorks
drwxr-xr-x 2 jjones progdevel 4096 Aug 16 20:42 ff_hosting_files -rw-r- 1 rhill heng 1440 Aug 22 19:07 myscript.log
The function keyword is optional. The only rule is that you need to declare the function before you can use it later in the program. You invoke the function simply by calling its name and passing on to it any further required or optional input parameters.
A while loop within bash allows you to execute a statement a given number of times, or until a certain condition is met. Listing 29 provides an example.
Page 13 of 18
developerWorks
ibm.com/developerWorks/
A case statement within bash lets you test for more than one condition or value and act accordingly. Using such a statement can sometimes be a better route than nested for loops or if statements to cut down on repetitive code or just for better structure. Listing 30 shows a brief case statement that make calls to functions, depending on the condition of the variable $key.
ibm.com/developerWorks/
developerWorks
Has no particular set of standard functions, whereas many modern programming languages have intrinsic or built-in functions for various programmatic needs.
Conclusion
Now that you have the basics of bash shell scripting well in hand, you must promise to only use your powers for the greater good. So, go out and save the *nix world with your newly found crontabs, one-line wonders, and interactive menus. Don't be afraid to be a "bash" administrator!
Page 15 of 18
developerWorks
ibm.com/developerWorks/
Downloads
Description
Sample bash shell scripts
Name
Bash_Source.zip
Size
8KB
Page 16 of 18
ibm.com/developerWorks/
developerWorks
Resources
Learn Check out the GNU bash reference manuala user guide to bash syntax. Wikipedia provides a good definition of the bash shell. Examine a comparison of shells to see how bash stacks up against the others. Check out bash basics for beginners. AIX and UNIX developerWorks zone: The AIX and UNIX zone provides a wealth of information relating to all aspects of AIX systems administration and expanding your UNIX skills. New to AIX and UNIX? Visit the New to AIX and UNIX page to learn more. Technology bookstore: Browse the technology bookstore for books on this and other technical topics. Get products and technologies Download bash and get the resources you need to start working with the bash shell now. Peruse the list of shells available for UNIX and Linux. Discuss developerWorks blogs: Check out our blogs and get involved in the developerWorks community. Follow developerWorks on Twitter. Participate in the AIX and UNIX forums: AIX Forum AIX Forum for developers Cluster Systems Management Performance Tools Forum Virtualization Forum More AIX and UNIX Forums
Page 17 of 18
developerWorks
ibm.com/developerWorks/
Page 18 of 18