Des|gn And Ana|ys|s of A|gor|thm

Introduct|on
An algorlLhm named afLer Lhe nlnLh cenLury scholar Abo Iofot MobommoJ lbo Moso Al
kbowotlzml ls deflned as follows
An algorlLhm ls a seL of rules for carrylng ouL calculaLlon elLher by hand or on a machlne
An algorlLhm ls a flnlLe sLepbysLep procedure Lo achleve a requlred resulL
An algorlLhm ls a sequence of compuLaLlonal sLeps LhaL Lransform Lhe lnpuL lnLo Lhe ouLpuL
An algorlLhm ls a sequence of operaLlons performed on daLa LhaL have Lo be organlzed ln daLa
sLrucLures
An algorlLhm ls an absLracLlon of a program Lo be execuLed on a physlcal machlne (model of
CompuLaLlon)
Lvery algorlLhm musL have followlng properLles
|n|teness 1he number of sLeps lnvolved ln an algorlLhm Lo solve a problem musL be flnlLe
Def|n|teness 1he number of sLeps ln a glven algorlLhm musL be preclse clear and unmlsLakable
S|mp||c|ty AlgorlLhm musL be slmple enough so LhaL ouLpuL can be compuLed easlly by paper
and pencll
Input 1o solve any problem Lhe algorlLhm musL recelve some lnpuL
Cutput AlgorlLhm musL produce aL leasL one ouLpuL whlch ls Lhe soluLlon of Lhe problem
Correctness AlgorlLhm musL perform Lhe Lask lL ls deslgned Lo perform for all lnpuL values
Genera||ty 1he subalgorlLhm lnherlLed ln a speclflc algorlLhm should be appllcable Lo all
algorlLhms of same general form wlLh mlnor modlflcaLlon
Iarlous noLaLlons used Lo develop an algorlLhm 1he synLax used for algorlLhm should
be language lndependenL and an algorlLhm should be well sLrucLured and deLalled so
LhaL Lhe reader can easlly LranslaLe lnLo compuLer program uslng any compuLer
programmlng language
A|gor|thm notat|ons
Name of A|gor|thm AlgorlLhm ls ldenLlfled by a name wrlLLen ln caplLal
Introductory Descr|pt|on AlgorlLhm name ls followed by an lnLroducLory descrlpLlon
of Lask performed by algorlLhm and assumpLlons made
Steps AlgorlLhm ls made up of a sequence of flnlLe number of sLeps execuLed one
afLer anoLher beglnnlng wlLh flrsL sLep (SLaLemenLs ln each sLep are execuLed from lefL
Lo rlghL)
Comments CommenLs are wrlLLen ln or begln wlLh // Lo descrlbe of any sLep
8|ocks are lndlcaLed by
Var|ab|e Name Iarlable name ln an algorlLhm musL be ln upper case Slngle characLer
name varlables used as subscrlpL or counLer are also ln upper
Input ]Cutput 8ead/lnpuL ls used Lo Lake lnpuL ln a varlable and CuLpuL/WrlLe ls used
Lo ouLpuL daLa
Ass|gnment Statements or or used Lo asslgn a valued Lo a varlable
8oo|ean va|ues Lrue and false can be used
and or not and Lhe relaLlonal operaLors can be used
Contro| Statements
Cond|t|ona| Statements
lf condlLlon Lhen
SLaLemenL(s)
Llse opLlonal
SLaLemenL(s)
end of lf sLrucLure
kepeat Structure
kepeat wh||e |oop kepeot step N to M whi/e condition
kepeat for |oop kepeot step N to M for condition
Goto and Lx|t Loop Statements CoLo sLep n and LxlL loop
SubalgorlLhm A blgger or a Complex algorlLhm can be dlvlded ln small sub algorlLhm
and sub algorlLhms can be called
Sub_a|go_name(parameters)

keturn
Call Lo sub algorlLhm
Sub_a|go_name(parameters) or ca|| Sub_a|go_name(parameters)
Lxamples of algorlLhms
Swap Lwo numbers/ lnpuL Lwo numbers and add Lhem
9erformance Ana|ys|s
Cr|ter|a on wh|ch we can [udge an a|gor|thm
oes lL do whaL we wanL lL Lo do ?
oes lL work correcLly accordlng Lo Lhe orlglnal speclflcaLlons of Lhe Lask ?
ls documenLaLlon descrlbes how Lo use lL and how lL works?
Are procedure creaLed ln such a way LhaL Lhey perform loglcal subfuncLlons?
ls Lhe code readable ?
All Lhese are noL dlrecLly relaLed Lo performance
1he Cr|ter|a d|rect|y re|ated to performance of the a|gor|thm
storage requ|rement and comput|ng t|me
|e Space Comp|ex|ty and 1|me Comp|ex|ty
1he space comp|ex|ty of an a|gor|thm |s the amount of memory |t need to run to comp|et|on
1he t|me comp|ex|ty of an a|gor|thm |s the amount of computer t|me |t need to run to
comp|et|on
9erformance eva|uat|on can be d|v|ded |nto two phases
1) 9r|or| est|mate (performance ana|ys|s) 2) poster|or| test|ng (performance measurement)
Space Comp|ex|ty
Space needed by each a|gor|thm |s the sum of the fo||ow|ng components
|xed 9art space for code space for s|mp|e var|ab|e space for constant etc
Var|ab|e 9art space needed by var|ab|es whose s|ze |s depend on part|cu|ar prob|em
reference var|ab|e s|ze and recurs|on stack space
Space Comp|ex|ty |s of ma[or concern when programs |mp|ement|ng a|gor|thms are made to
run on mu|t|user system where space |s shared among users
A|though space comp|ex|ty |s |mportant bun |nexpens|ve memory has reduced the s|gn|f|cance
of space comp|ex|ty
1|me Comp|ex|ty
1|me taken to execute the a|gor|thm and get the |ntended resu|t |s ca||ed t|me comp|ex|ty |e
(comp||e t|me + run t|me)
1|me comp|ex|ty may depends
kAM
Speed of computer
Cperat|ng system used on computer
ua||ty of code
and many more
1hese factors together const|tute the abso|ute t|me of the a|gor|thm
Change |n any of these resu|t |n change |n abso|ute t|me
A better method
A mathemat|ca| mode| that ana|yze a|gor|thm |ndependent|y of a spec|f|c |mp|ementat|on
Ana|yze the t|me comp|ex|ty by count|ng the number of key operat|ons |n an a|gor|thm (t|me
|nvo|ved |n perform|ng other operat|ons much |ess then or a|most proport|ona| to t|me for key
operat|ons
eg |n search|ng a|gor|thm number of key operat|ons are the number of compar|son made
Cont|nue
1he number of key operat|ons |s funct|on of the number of e|ements to be processed
So the t|me comp|ex|ty of an a|gor|thm |s funct|on f{n) that descr|bes the eff|c|ency of an
a|gor|thm |n term of s|ze n of |nput data
kepeat for I1 to 100
statements(s)
f(n) n
kepeat for I1 to 100 by 2
statements(s)
f(n) n]2
I 1
kepeat wh||e I 100
statement (s)
II*2
end of |oop
I100
kepeat wh||e I 1
statement (s)
II]2
end of |oop
Mu|t|p|y I100
I I*2 |terat|on
1 2 1
2 4 2
4 8 3
8 16 4
16 32 S
32 64 6
64 128 7
128 Noth|ng
D|v|de I1
I I]2 |terat|on
100 S0 1
S0 2S 2
2S 12 3
12 6 4
6 3 S
3 1 6
1 0 7
0 Noth|ng
or Mu|t|p|y 2
|terat|on
100 or d|v|de 100]2
|terat|on
1
|e f(n) |og
2
n
Log
2
2 |terat|ons |og
2
100
Iterat|ons |og
2
100]|og
2
2
|og
2
100 (because |og
2
21)
kepeat for I 1 to 20
kepeat for I1 to 20
Statement(s)
end of |nner |oop
end of outer |oop
f(n) n
2
(uadrat|c |oop |oop |ns|de |oop)
I1
kepeat wh||e I20
I1
kepeat wh||e I20
Statement(s)
II*2
end of |nner |oop
II+1
Lnd of |oop
(n) n |og n
kepeat for I1 to 20 by 1
8epeaL for !1 Lo l by 1
SLaLemenL(s)
end of lnner loop
end of ouLer loop
Lxact Measurement of A|gor|thm's 1|me comp|ex|ty
Average(AN) G|ven an array A of N e|ements th|s a|gor|thm ca|cu|ate average of array
e|ements
1 SUM0 AVG0 2
2 kepeat Step 3 to 4 for I1 to N ass|gnment 1 compar|son n
3 SUMSUM+AI n
4 II+1 n
end of |oop
S AVGSUM]N 1
6 Wr|te AVG 1
7 keturn 1
1he comp|ex|ty funct|on f(n) |s 3n + 6
lnner loop 210/20 21/2 (20+1)/2
CuLer loop 20 Llme
So LoLal Llme ls 20 * (20+1)/2
(n) n *(n+1)]2
Wh||e measur|ng the exact t|me comp|ex|ty of an a|gor|thm |t |s observed that ca|cu|at|ng the
t|me comp|ex|ty funct|on f(n) |s d|ff|cu|t and comp|ex
If an a|gor|thm process |arge amount of data |t |s observed that there |s certa|n term |n the
comp|ex|ty funct|on f(n) that grows the fastest
So any term(s) that do not cons|derab|y contr|bute to the funct|on's magn|tude shou|d be
e||m|nated
So resu|t|ng funct|on |s an approx|mate t|me comp|ex|ty of or|g|na| funct|on
nowever th|s approx|mat|on |s c|ose to the or|g|na| (espec|a||y funct|ons processes |arge
amount of data)
1h|s measurement of t|me comp|ex|ty of an a|gor|thm where we d|sregard certa|n terms of the
funct|on to express the eff|c|ency of a|gor|thm |s ca||ed Asymptot|c Comp|ex|ty
1o understand th|s Let f(n) n
2
+3n+10
n f(n) n2 3n 10
Ialue Ialue Ialue
0 10 0 0 0 0 10 100
10 140 100 714 30 214 10 72
100 10310 100000 9699 300 29 10 011
1000 1003010 10
6
9969 3000 039 10 001
10000 100030010 10
8
9996 30000 0029 10 0001
rom the above tab|e we ana|yze that for sma|| va|ue of n the terms 3n and 10 makes a
|arge contr|but|on to funct|on
8ut as n grow these terms are not so dom|nant as the|r contr|but|on to funct|on va|ue
|s |ess s|gn|f|cance
We conc|ude
or process|ng |arge amount of data we are on|y concerned about a dom|nant term |n
comp|ex|ty funct|on |e term w|th |arger order of magn|tude
1herefore we don't need to determ|ne the exact t|me comp|ex|ty
Cn|y need to determ|ne dom|nant term that determ|ne order of magn|tude of
funct|on
1o descr|be the order of magn|tude of a funct|on 8|gC notat|on was |ntroduced |n
1894 by 9au| 8achmann
Ma|n a|m of th|s notat|on |s to |dent|fy the dom|nant term |n determ|n|ng the t|me
comp|ex|ty of an a|gor|thm
If we have an a|gor|thm hav|ng t|me comp|ex|ty f(n) n
2
+3n+10 then |ts 8|gC notat|on
w||| be expressed as C(n
2
)
1he 8|gC notat|on can be der|ved from t|me comp|ex|ty funct|on f(n) us|ng fo||ow|ng
steps
1 Set the coeff|c|ent of each term to 1
2 keep the dom|nat|ng term |n the funct|on and |gnore the other terms
12
8|gC Notat|on
W eL n be a nonnegaLlve lnLeger represenLlng
Lhe slze of Lhe lnpuL Lo an algorlLhm
W eL f(n) and g(n) be Lwo poslLlve funcLlons
represenLlng Lhe number of baslc calculaLlons
(operaLlons lnsLrucLlons) LhaL an algorlLhm
Lakes (or Lhe number of memory words an
algorlLhm needs)
13
8|gC Notat|on (conLd)
W f(n)C(g(n)) lff Lhere exlsL a poslLlve consLanL
C and nonnegaLlve lnLeger n0 such LhaL
W f(n) A Cg(n) for all nKn0
W g(n) ls sald Lo be an upper bound of f(n)
14
8|gC Notat|on
(Lxamp|es)
W f(n) 3n+2 C(n) // g(n) n
f(n) A 6n for n K 3 (C6 n
0
3)
W f(n)n/2 3 C(n)
f(n) A 03 n for n K 0 (C03 n
0
0)
W n
2
n C(n
2
) // g(n) n
2
n
2
n A n
2
for n K 0 (C1 n
0
0)
W n(n+1)/2 C(n
2
)
n(n+1)/2 A n
2
for n K 0 (C1 n
0
0)
13
8|gC Notat|on
(ln racLlce)
W When compuLlng Lhe complexlLy
f(n) ls Lhe acLual Llme formula
g(n) ls Lhe slmpllfled verslon of f(n)
W Slnce f(n) sLands ofLen for Llme we use 1(n)
lnsLead of f(n)
W ln pracLlce Lhe slmpllflcaLlon of 1(n) occurs
whlle lL ls belng compuLed by Lhe deslgner
16
S|mp||f|cat|on Methods
W lf 1(n) ls Lhe sum of a consLanL number of
Lerms drop all Lhe Lerms excepL for Lhe mosL
domlnanL (blggesL) Lerm
W rop any mulLlpllcaLlve facLor of LhaL Lerm
W WhaL remalns ls Lhe slmpllfled g(n)
W a
m
n
m
+ a
m1
n
m1
++ a
1
n+ a
0
C(n
m
)
W n
2
n+log n C(n
2
)
17
8|gC Notat|on
(Common ComplexlLles)
W 1(n)C(1) // consLanL Llme
W 1(n)C(log n) // logarlLhmlc
W 1(n)C(n) // llnear
W 1(n)C(n
2
) //quadraLlc
W 1(n)C(n
3
) //cublc
W 1(n)C(n
c
) cK 1 // polynomlal
W 1(n)C(log
c
n) cK 1 // polylogarlLhmlc
W 1(n)C(nlog n)
18
8|gC Notat|on
(Character|st|cs)
W 1he blgC noLaLlon ls a slmpllflcaLlon
mechanlsm of Llme/memory esLlmaLes
W lL loses exacLness for slmpllclLy
W 8eLalns enough lnformaLlon Lo glve a ballpark
ldea of speed/cosL of an algorlLhm and Lo be
able Lo compare compeLlng algorlLhms
19
Common ormu|as
W 1+2+3++n n(n+1)/2 C(n
2
)
W 1
2
+2
2
+3
2
++n
2
n(n+1)(2n+1)/6 C(n
3
)
W 1+x+x
2
+x
3
++x
n
(x
n+1
1)/(x1) C(x
n
)
20
Lxamp|e of 1|me Comp|ex|ty Ana|ys|s and
8|gC
W seudocode of flndlng a maxlmum of xn
double Mx0
for l1 Lo n1 do
lf (xl M)
Mxl
endlf
endfor
reLurn M
21
Comp|ex|ty of the a|gor|thm
W 1(n) a+(n1)(b+a) C(n)
W Where a" ls Lhe Llme of one asslgnmenL and
b" ls Lhe Llme of one comparlson
W 8oLh a" and b" are consLanLs LhaL depend
on Lhe hardware
W Cbserve LhaL Lhe blg C spares us from
8elaLlvely unlmporLanL arlLhmeLlc deLalls
Pardware dependency