BasicsofCreatingandUtilizingBASHScripts

PeteNesbitt May2006

LinuxBASHShellScripting

TheinformationpresentedhereshouldactasaguidetocreatingqualityscriptsusingtheLinuxbuiltin BASHscriptinglanguage.BASH(BourneAgainShell)isascriptinglanguageaswellasthedefault commandinterpretorinmostLinuxdistributions,includingRedHatLinux.Theabilitytocreate qualityscriptsisarguablythemosttimeanderrorsavingaspectofLinuxSystemsAdministration.The factthatthedefaultshelloruserenvironmentinLinuxisBASHmakesmakeslearningtocreate scriptsbothveryvaluableaswellassimple.Inthefollowingdocumentwewilllookatfundamentalsof scripts,bestpracticescreatingandtestingofscripts.Shellscriptscanbeverysimpleorextremely complexinvolvingdozensofsupportfiles,butregardlessofthecomplexity,theyshouldbecreatedina consistent,selfexplanatoryandeasytoreadformat. Wewon'tcoverregularexpressions,orshellutilitieslikeloopssincethatinformationisreadily availableontheInternetandinbooks.Ifafterlookingatanexample,youdonotunderstandanaspect likewhilereadxin`ls`,besuretoresearchhowthesefunctionswork,oraskamoreexperienced staffmemberforhelp.Mimickingaprocesswithoutunderstandingisoflittleornovalue,and misunderstandingtheprocesscanleadtodangerousscriptsandfrustration. Inordertoachievethebestvaluefromthisdocument,youshouldbefamiliarwithcommoncommands, basicregularexpressionsandGNUutilities.Anythingusedwithintheexamplescanberesearched usingthemancommandorbysearchingGoogle.Forexample,youwillwanttobefamiliarwithgrep, sed,awk,andcut. IMPORTANTNOTE:Ifyoutryandcutandpastetheexamplesinthisdocument,youwillprobably needtomanuallyreplacethespecialcharacterssincethewordprocessormodifiesthemfor presentation.Onescopiedfromthecommandlinewillbeokay,butonestypeddirectlyintothis documentwillneedtobecorrected.Theseinclude,butarenotlimitedto'`/\| TheBashShellmantra: Everyformalscriptshouldbeginwithcommandlineinterpreterdefinition. InBASH,itisavirtualpoem: HashBangSlash BinSlashBash Literally: #!/bin/bash
1of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

InadditiontowhatIcallaformalscript,youcanachievescriptlikefunctionalitybyenteringaseries ofcommandattheprompt.

WhatcanIdowithascript? Anytaskyoucanrunatthecommandlinecanbeincorporatedintoascript.Thisisbecauseworkingat thecommandlineandwritingascriptareessentiallythesamething.UnixandLinuxutilitiesand commandstendtobeverysimpleandsingularintask.However,stringingaseriesofsuchutilities togethercancreateveryflexible,powerfulandextensiblecommandlineorscript.Thereisno environmentthatcanbemanipulatedtoperformwithsuchflexibilityandadaptabilitythanthe commandlineandit'srelatedscriptfiles. Anytaskthatiscomplicated,repetitiveorneedstobeperformednumeroustimesshouldbescripted. Thissavestime,eliminatesthechancesoftypos,andoftencanbereusedlater. Creatingascript: 1. startbylayingouttheflowwithonlycommentsofwhateachstepwillperform,don'tworry abouttheactualcodeuntilyoudefinethestepsyouwanttoperform. 2. usethecommandlinetotestpiecesofyourscript 3. takesmallbitsofcodethatyouhavetestedatthecommandline,thenmovethemintothescript 4. alwaystestinasafearea,notworkingonimportantdataorfilesuntilyouhaveconfirmedthe scriptsworksasexpected. LearningtowritequalityBASHScripts: 1. Beconsistent.Developalayoutforyourscriptsandsticktothatstyle. 2. Takeyourtime.Scriptingisnotoverlycomplicated,butdoesrushingthingswilljustresultin errorsandfrustration. 3. Practicemakesperfect.Tryandscriptasmanythingsaspossible,especiallycomplicatedtasks. Thiswillhelpdevelopyourskillsworkingwithsetsofcommandsasopposedtodoingone thing,savingtoafile,workingonthatfile,thensavingthoseresultsandsoon.Evenifitisonly tobeusedonce,unlesstimeisoftheessence,goscriptcrazy.Youwillbeabetter,more
2of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

valuableSystemsAdministratorinthelongrun. 4. Reviewtheworkofothers.Inscripting,likealmosteverythingelseUnixorLinuxrelated,there aremanywaystodoanygiventask.Seehowothersdothings,thendecidewhatmakessense foryou. BestPractices: Alwayswriteyourscriptsinawaythatotherscaneasilyfollowtheflowandfunctionality. giveyourscriptsameaningfulnameandinclude.shasasuffix alwaysdefinethecommandinterpreteronthefirstline includeyournameandthedate includethefilenameandabriefdescriptionofwhatthescriptwilldo definefunctions,variablesandincludefilesbeforethemaincode includecomments(linesstaritnwitha#)andblanklinesforreadability indentstanzasorcodeblocksforreadability fullyqualifyyourcommands thisaddssecurityaswellasmakingitcronsafe althoughthismaybreakportability,youcanuseanothershellscripttointerigatethesystem andensureportabilitywhendefiningcommandsasvariables. addan'EndOfFile'comment Whattodoifyourscriptdoesn'twork: 1. addsomeoutputtoseehowthingsprogress addanecholineateachmainstep,thatwilltellyouwhatdoeswork example: afteralinelikeletx=$x+1 addechothevalueofxisnow:$xtoseeifthevalueischangingasexpected 2. ifyougetanerrorstatingpermissiondeniedthescriptisnotexecutable 3. trysnipitsonthecommandlineorinasmallerscripttofindandfixafailingline 4. useGoogleorotherresourcestoconfirmyourcode,suchasaloopformat

3of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

SampleScriptTemplate:(thisisnotaphysicaltemplate,butageneralformatyoumaywanttoadopt) #!/bin/bash # #file:myscript.sh #createdbyPeteNesbittMay2006 #thisscriptwillperformsomefancytask. # #setsomevariablestobeusedthroughoutthescript TARGET=remote.host.com DATE_STAMP=`date+%y%m%d` EMAIL_RECIPIENTS=admin@yourdomain.com #herewewillmanipulatesomething # somecodegoeshere #nowweneedtodothisotherthing someothercodegoeshere #eof Functions: Afunctionissimpleacodeblockthatcanbecalledseveraltimeswithouttheneedtorewritethewhole thingeverytime. InBASHafunctionisdefinedas: my_function() { thecodeblock } Thencanbeusedanywhereinthescriptlikeso: dosomestuff... my_function somemorecode SeetheSamplessectionattheendforrealworldexamples.
4of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

BuildingaFunctionsLibrary: Whetheryouusetheincludestatementorjusthaveaninventoryofcodesnipits,youcanbeconsistent andavoidrecreatingthewheelbyaccumulatingalibraryofshellfunctions,tasksandvariables. ASimpleScripttoTry: Firstletsmakeaquickscripttocreatesomefilesthatwecanlatermanipulate,thenwewillrename them.Sincethesearesimpleexamples,letscheatabithere,usingthecommandlineinsteadofaformal script. Createtheoriginalfiles: count=0;while[$countlt10];dotouchfile_$count;letcount=${count}+1;done Wenowhave10files,namedfile_0thrufile_9 Letsrenamethem.Wecoulddoeachseparatelyusing: mvfile_0new_name_0;mvfile_1new_name_1;...mvfile_9new_name_9 butthatseemslikealotofworkandalotofopportunityforerror. Asimplerandmoreefficientmethodwouldbetocreatealoopthatwouldrenameeachfile automatically.Thisisverysimilartothewaywecreatedthefiles.Thistimethough,wewilldoeach stepseparately.NotethepromptchangeswhenBASHexpectsmoreinformationbeforeitcancomplete thecommandstring.Thisiscalledthesecondlevelprompt(theprimarypromptisa$formostusers anda#fortherootuser). $forxin`ls` >do >n="`echo$x|cutd_f2`" >mv$xnew_file_$n >done Wenowhave10files,namednew_file_0thrunew_file_9 Ifyounowdoanuparrow,thepreviouscommandisdisplayedas: forxin`ls`;don="`echo$x|cutd_f2`";mv$xnew_file_$n;done Wheneverdoingmasschanges,itisprudenttocreateatestdirectorytousewhenbuildingthescriptor commandline,soyoudonotdamageyourrealsourcefilesinthecaseofanerror.
5of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

EXITCODES: Wheneveracommandisexecuted,uponcompletionsanumericvaluesetwhichindicateswhetherthe commandwassuccessfulornot.Anexitvalueof0(zero)meanssuccess.Anynonzerovalueindicates nonsuccess,usually1(one)andusuallyfailure.Inthecaseofacommandlikegrep,andexitcodeof1 doesnotreallymeanthecommandfailed,butjustthatitdidnotfindanythingthatmatchedthequery. Anexitcodecanbeforcedwitha'exit'stringsuchasthisexampleplacewithinan'if'statement: if ...somecode... else echo"OperationFailed!" exit1 fi Ifthe'else'lineismatched,thenthescriptexitswithastatusof1 Youcanalsotestforsuccessofacommandbyretrievingtheexitcode,whichisrepresentedbythe variable$?usingsomethinglikethis: ...somecommand... if[$?eq0];then ...success,carryon or,moreyoumaypreferthis,slightlymorerefinedmethod: ...somecommand... RET_VAL=$? if[$RET_VALeq0];then ...success,carryon Thethingtorememberisthatthevariable$?willbeoverwrittenwiththenextcommands completionunlessyousaveitintoauniquevariableasinthesecondexample.

6of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

HowtouseINCLUDEFILES: Youcanstorefunctionsinoneormorefilesoutsideyourmainscript.Inordertoutilizethese functions,youmusthaveanincludestatement,thenyoucanjustcallthemasyouwouldanyother locallydefinedfunction.Theincludelinedoesnotactuallyhaveasyntax,itsimplyreadsinadditional functions. Hereweareaddingthefileinstall_lib,whichcontainsaseriesoffunctions. .src/install_libs(notetheleading'dotspace'meaning'lookinthisfileplus...') INTERIGATINGFORVARIABLES: AlthoughtheBASHshellisincludedinmostUnixandLinuxsystems,someofthesystemcommands orutilitiesarelocatedindifferentdirectories.Ifyouwantyourscripttobetrulyportable,youmust takethisintoaccount.Belowisoneofmanywaystodoit. Wereadafileifthecommandnames,formattedonecommandperline. It'scontentsmaylooklikethis: rm grep sed awk Thenwesearchforthemandsavetheoutputtoanincludefile: (theshortfallhereisthe'which'commandmustbeavailable,amorecomplexmethodisusedinthe FlacJacketscripts) whilereadCMD_VAR do CMD_NAME="`echo$CMD_VAR|\`whichtr\`[:lower:][:upper:]`" echo${CMD_NAME}=`which${CMD_VAR}`>>command_include_file done<command_list.txt REALWORLDSAMPLESANDEXAMPLES: Note,somelinesmaywrapinthefollowingexamples.Thiscanbeavoidedinthescriptwitha\to escapethelinereturn.Also,thesearenotnecessarilyfollowingthe'BestPracticesmodel,however,I
7of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

didnotwanttoaltertheexamples. Foracompletepackageinvolvingmostthingswehavediscussedhere,downloadFlacJacket,agood exampleofcomplexBASHscripting. http://nesbitt.yi.org:6690/flac_jacket/index.shtml(downloadflac_jacketcurrent.tgz) Asimplescripttosendamonthlyemailreminder: #!/bin/bash # #sendemailremindertovisityimonthly #runfromcronmonthly /usr/bin/printf"Timetoupdateyi.org\n\ntheurlishttp://www.whyi.org/admin/\n"|/bin/mails"Update YI"pete Thisscriptbacksupalocalunix/linuxsystemtoaWindowsserver. #!/bin/bash # #scripttobackuplocalfilesonremotesystemviasshpubkeyathentication. #bothsystemsmustbesetupforsshkeyauth. #June26,2000.PeteNesbitt #filetoreadlistofsourcedirsfrom. SOURCES=/etc/backup_src.txt EXCLUDE=/tmp/tar_exclude.tmp #targethostanddir TARGET_HOST=192.219.60.22 TARGET_DIR=$TARGET_HOST:e:/bupnesbitt whilereadLINE do TARFILE="/tmp/`echo$LINE|sedes/'\/'/_/g`.tar"
8of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

find`echo$LINE`types>$EXCLUDE find`echo$LINE`typep>>$EXCLUDE #checkwhichtar,thePwilleitherGNU="notremovetheleading/"orSolaris="don'taddatrailing /" /bin/tarcPf$TARFILEX$EXCLUDE$LINE /bin/gzip9$TARFILE #nextlineforces"openssh"asthatiswhattheserveruses,open&comdon'tplaywell. subackupc"/usr/bin/scp$TARFILE.gz$TARGET_DIR/" rmf$TARFILE.gz done<$SOURCES

AnexampleofaFUNCTION: Thisisaclipofthefirsttwoofseveralpossibleerrormessages,theyutilizeastringdefinedlaterinthe script,justbeforecallingthefunction.Thisallowsformoremeaningfulmessages.Followingthetwo functions,isaclipofhowoneisusedlaterintheprogram. error_1() { echo"ERROR:Thereisa\"${ERR_TARGET}\"directory,butyoucannotwritetoit." error_mesg_perm echo"" exit1 } ###### error_2() { echo"ERROR:Thereisafilenamed\"${ERR_TARGET}\",butitdoesnotappeartobea directory." echo"Removeorrenamethe\"${ERR_TARGET}\"file,orchooseadifferentlocation,andtry again."

9of10

BasicsofCreatingandUtilizingBASHScripts
PeteNesbitt May2006

LinuxBASHShellScripting

echo"" exit1 } Later,theycanbecalledlikethis: (notethelocallydefinedERR_TARGETaswellasthefunctioncallsuchaserror_1) #doesthethumbnailsexist,ifsoisitadir,ifsocanwewritetoit ERR_TARGET="thumbnails" if[e$THUMBNAILS];then if[d$THUMBNAILS];then #makesurewecanwritetoit if![w$THUMBNAILS];then error_1 fi else error_2 fi else mkdir$THUMBNAILS #runatestforsuccess if[d$THUMBNAILS];then if![w$THUMBNAILS];then error_3 fi fi fi

EOF

10of10