Sie sind auf Seite 1von 63

1

Analysis and Design of


algorithms
2
Books
Fundamentals of Computer algorithms
Horowitz, Sahani and Rajasekaran

Introduction to Algorithms
Coremen, Leiserson

The Design and Analysis of Computer

Algorithms
Aho, Hopcroft and Ullman

3

ALGORITHM


A finite set of instructions which if
followed accomplish a particular task.
In addition every algorithm must satisfy
following criteria:


4
1. Input: zero or more quantities externally
supplied
2. Output: at least one quantity is produced
3. Definiteness: Each instruction must be
clear and unambiguous.
4. Finiteness: In all cases algorithm must
terminate after finite number
of steps.
5. Effectiveness: each instruction must be
sufficiently basic.
5
Efficiency of Algorithms
Two algorithms on two systems
Algorithm A
1
50 n lg n
Algorithm A
2
2 n
2



Super computer
10
8
ins/sec
A
2

P.C
10
6
ins /sec
A
1

6
For n = 10
6


Time taken by Super Computer

= 2.(10
6
)
2
/ 10
8
= 20,000 sec

Time taken by P .C.
= 50 . 10
6
lg 10
6
/ 10
6
= 1,000 sec

7

Thus by using a fast algorithm , the personal
computer gives results
20 times faster than the result given by
super computer using a slow algorithm.

Thus a good algorithm is like a sharp knife,
it does exactly what it is supposed to do
with a minimum amount of effort.
8
Pseudocode is an English language like
representation of the code required for an
algorithm.

It is partly English, partly structured code.

The English part provides a relaxed syntax that
is easy to read.

The code part consists of an extended version of
the basic algorithmic constructs-sequence,
selection and iteration.
Pseudocode
9
Sequence, selection, loop
A sequence is a series of statements that do not
alter the execution path within an algorithm.
Statements such as assign and add are
sequence statements.
A call to another algorithm is also considered a
sequence statement.
Selection statements evaluate one or more
alternatives. Paths are followed based on its
result.
10
The typical selection statement is the two
way selection
if (condition) action 1 else action 2.
The part of the loop are identified by
indentation.
Loop iterates a block of code. It closely
resembles the while loop. It is a pretest
loop.
11

1 i= 0
2 Loop(all data are read)
1 i = i + 1
2 read numbers into array[i]
3 sum = sum + number
3 Average = sum / I
4 Print (average)
5 J = 0
6 Loop (j < i)
1 j = j+ 1
2 dev = array[j] average
3 print (array [ j] . Dev)
7 Return
8 End deviation

Linear loop
1 l=1
2 Loop(I <= 1000)

1 application code
2 I = I + 1

The body of the loop is repeated 1000 times.

1 I = 1
2 Loop (I <= n)
1 Application code
2 I = I + 2
For this the code time is proportional to n
Logarithm Loops
Multiply loops Divide
loops
1 I = 1 1 I = n
2 Loop (I < n) 2 loop( l>= 1)
1 application code 1 application code
2 I = l*2 2 I=I/2


F(n) = [log n] F(n) = [log n]

Nested loop- linear logarithmic
1 I = 1
2 loop(I <= n)
1 j = 1
2 loop( j < = n)
1 application code
2 j = j *2
3 I = I + 1
F(n ) = [n log n]

Dependent Quadratic
1 I = 1
2 loop ( I < = n)
1 j = 1
2 loop( j < = l)
1 application code
2 j = j + 1
3 I = I + 1
no of iterations in the body of the inner loop is
1 + 2 + 3 + 4 + + 10 = 55 I.e. n(n +1)/2
On an average = ( n+1/)2
thus total no of iterations = n (n+1)/2

Quadratic
1 l=1
2 Loop (I < = n)
1 j = 1
2 Loop( j < = l)
1 application code
2 j = j+1
3 I = j+1

F(n) = n
2

Algorithm Analysis
18
Complexity
Some questions to answer:
How fast can we solve a
problem?
There may be many algorithms for a
given problem. Which algorithm to use?
What are the classical algorithm
design techniques ?
Are there problems inherently difficult
to solve?

19

How do we express the complexity of algorithm?

Resources : Time and Space

Complexity lower bounds for problems.

Complexity classes P, NP etc.
Analysis vs. Design
Analysis: predict the cost of an algorithm in
terms of resources and performance

Design: design algorithms which minimize
the cost
Machine model: Generic Random Access
Machine (RAM)
Executes operations sequentially
Set of primitive operations:
Arithmetic. Logical, Comparisons, Function calls

Simplifying assumption: all ops cost 1 unit
Eliminates dependence on the speed of our computer,
otherwise impossible to verify and to compare

L1.22
Review of Complexity
Most of the primary sorting algorithms run on different
space and time complexity.
Time Complexity is defined to be the time the computer
takes to run a program (or algorithm in our case).

Space complexity is defined to be the amount of
memory the computer needs to run a program.
(Note:-What is Running Time and it is expressed as
function of the size of its input.)

Time Complexity
Real Time:

To analyze the real time complexity of a program we
need to determine two numbers for each statement in it:

amount of time a single statement will take.

No. of times it is executed.

Product of these two, will be the total time taken by the
statement.


First no. depends upon the machine and
compiler used , hence the real time
complexity is machine dependent.


Frequency count
To make analysis machine independent
it is assumed that every instruction
takes the same constant amount of
time for execution.

Hence the determination of time
complexity of a program is the matter
of summing the frequency counts of all
the statements.
L1.26
Kinds of analysis
Worst-case: (usually)
T(n) = maximum time of algorithm on any
input of size n.
Average-case: (sometimes)
T(n) = expected time of algorithm over all
inputs of size n.
Need assumption of statistical distribution
of inputs.
Best-case: (NEVER)
Cheat with a slow algorithm that works fast
on some input.
L1.27
INSERTION SORT
Insertion sort keeps making the left side
of the array sorted until the whole array is
sorted.
Real life example:
An example of an insertion sort occurs in
everyday life while playing cards. To sort the
cards in your hand you extract a card, shift
the remaining cards, and then insert the
extracted card in the correct place.
L1.28
Insertion sort
INSERTION-SORT (A, n) A[1 . . n]
for j 2 to n
do key A[ j]
i j 1
while i > 0 and A[i] > key
do A[i+1] A[i]
i i 1
A[i+1] = key
pseudocode
i j
key
sorted
A:
1 n
L1.29
Example of insertion sort
8 2 4 9 3 6
L1.30
Example of insertion sort
8 2 4 9 3 6
L1.31
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
L1.32
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
L1.33
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
L1.34
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
L1.35
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
L1.36
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
L1.37
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
L1.38
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
L1.39
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
L1.40
Selection sort
How does it work:
first find the smallest in the array and exchange it with the element in
the first position, then find the second smallest element and
exchange it with the element in the second position, and continue in
this way until the entire array is sorted.
How does it sort the list in a non increasing order?
Selection sort is:
The simplest sorting techniques.
a good algorithm to sort a small number of elements
an incremental algorithm induction method

Selection sort is Inefficient for large lists.

Incremental algorithms process the input elements one-by-
one and maintain the solution for the elements processed so
far.

L1.41
Selection Sort Algorithm
Input: An array A[1..n] of n elements.
Output: A[1..n] sorted in nondecreasing order.
1. for i 1 to n - 1
2. k i
3. for j i +1 to n {Find the i th smallest element.}
4. if A [ j ] < A[ k] then k j
5. end for
6. if k = i then
7. temp A[ i ] (interchange A[ i ] and A[ k ] )
8. A[ i] A[ k]
9. A[k ] temp
10. end for
Selection Sort in Action
L1.43
Analysis of Algorithms
Algorithm analysis: quantify the performance of the algorithm,
i.e., the amount of time and space taken in terms of n.
T(n) is the total number of accesses made from the beginning
of selection_sort until the end.
selection_sort itself simply calls swap and find_min_index as i
goes from 1 to n-1

=
+ =
1
1
min ) (
n
i
swap element find n T
= n-1 + n-2 + n-3 + + 1 = n(n-1)/2
Or = (n - i) = n (n - 1) / 2 - O(n
2
)
Binary search
sorted sequence : (search 9)
1 4 5 7 9 10 12 15
step 1 |
step 2 |
step 3 |
best case: 1 step = O(1)
worst case: (log
2
n+1) steps = O(log n)
average case: O(log n) steps

Growth Rate of an Algorithm (Order of Growth)
We often want to compare the performance of algorithms

When doing so we generally want to know how they
perform when the problem size (n) is large

Since cost functions are complex, and may be difficult to
compute, we approximate them using Asymptotic
notation
Example of a Cost Function
Cost Function: t
A
(n) = n
2
+ 20n + 100
Which term dominates?
It depends on the size of n
n = 2, t
A
(n) = 4 + 40 + 100
The constant, 100, is the dominating term
n = 10, t
A
(n) = 100 + 200 + 100
20n is the dominating term
n = 100, t
A
(n) = 10,000 + 2,000 + 100
n
2
is the dominating term
n = 1000, t
A
(n) = 1,000,000 + 20,000 + 100
n
2
is the dominating term
Asymptotic notation Introduction
Exact counting of operations is often difficult
(and tedious), even for simple algorithms
Often, exact counts are not useful due to other
factors, e.g. the language/machine used, or the
implementation of the algorithm (different types
of operations do not take the same time anyway)
Asymptotic-notation is a mathematical language
for evaluating the running-time (and memory
usage) of algorithms.
Big O Notation
O notation approximates the cost function of an
algorithm
The approximation is usually good enough, especially
when considering the efficiency of algorithm as n gets
very large
Allows us to estimate rate of function growth
Instead of computing the entire cost function we
only need to count the number of times that an
algorithm executes its barometer instruction(s)
The instruction that is executed the most number of
times in an algorithm (the highest order term)
The general idea is
when using Big-O notation, rather than giving a
precise figure of the cost function using a
specific data size n
express the behaviour of the algorithm as its
data size n grows very large
so ignore
lower order terms and
constants
O Notation Examples
All these expressions are O(n):
n, 3n, 61n + 5, 22n 5,
All these expressions are O(n
2
):
n
2
, 9 n
2
, 18 n
2
+ 4n 53,
All these expressions are O(n log n):
n(log n), 5n(log 99n), 18 + (4n 2)(log (5n + 3)),

Asymptotic Notation (cont.)
Note: Even though it is correct to say 7n - 3 is O(n
3
), a
better statement is 7n - 3 is O(n), that is, one should make
the approximation as tight as possible

Simple Rule: Drop lower order terms and constant factors
7n-3 is O(n)
8n
2
log n + 5n
2
+ n is O(n
2
log n)
Worst/ best/average cases

Worst case is the longest running time for
any input of size n
O-notation represents upper bound I.e. an
upper bound for worst case.
Best case is time taken for some input
data set that results in best possible
performance.You cannot do better. This is
lower bound.
Average case is the average performance
The Big-Oh Notation:
given functions f(n) and g(n), we
say that f(n) is O(g(n) ) if and only
if there are positive constants c
and n
0
such that f(n) c g(n) for
n n
0
Example
f(n) = 2n + 6
For functions f(n)
and g(n) (to the
right) there are
positive constants c
and n
0
such that:
f(n) c g(n)
for n n
0
conclusion:
2n+6 is O(n).
Asymptotic Notation (terminology)
Special classes of algorithms:

constant O(1)
logarithmic: O(log n)
linear: O(n)
quadratic: O(n
2
)
polynomial: O(n
k
), k 1
exponential: O(a
n
), a > 1



EXAMPLE

Consider 1/3 n
2
5n

The dominating term is n
2

Therefore it should be of O(n
2
)

Given a positive constant c , a positive integer
n
0
to be found such that
1/3 n
2
- 5 n s c n
2

Asymptotic Analysis of The Running Time
Use the Big-Oh notation to express the
number of primitive operations executed as a
function of the input size.
Comparing the asymptotic running time
-an algorithm that runs in O(n) time is better
than one that runs in O(n
2
) time
-similarly, O(log n) is better than O(n)
-hierarchy of functions: log n << n << n
2
<<
n
3
<< 2
n

Categories of algorithm efficiency

Efficiency
Constant
Big O
O(1)
Logarithmic O(log n)
Linear O(n)
Linear logarithmic O(n log n)
Quadratic O(n
2
)
Polynomial O(n
k
)
Exponential O(c
n
)
Factorial O(n!)
O-notation
O(g(n)) = {f(n): - +ve
constants c
1
, c
2
, and n
0

such that 0 s c
1
g(n) s
f(n) s c
2
g(n),
n >

n
0
}
For function g(n), O(g(n)) is
given by:
g(n) is an asymptotically tight bound for f(n).
Intuitively: Set of all functions that
have the same rate of growth as g(n).
O -notation
O(g(n)) = {f(n): - +ve
constants c and n
0
such that
0 s f(n) s cg(n), n >

n
0
}
For function g(n), O(g(n))
is given by:
g(n) is an asymptotic upper bound for f(n).
Intuitively: Set of all functions whose rate
of growth is the same as or lower than that
of g(n).
f(n) = O(g(n)) f(n) = O(g(n)).
O(g(n)) c O(g(n)).
O -notation
O(g(n)) = {f(n): - +ve
constants c and n
0
such that
0 s cg(n) s f(n), n >

n
0
}
For function g(n), O(g(n))
is given by:
g(n) is an asymptotic lower bound for f(n).
Intuitively: Set of all
functions whose rate of
growth is the same as or
higher than that of g(n).
f(n) = O(g(n)) f(n) = O(g(n)).
O(g(n)) c O(g(n)).
Relations Between O, O, O
Practical Complexity
0
250
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n

Das könnte Ihnen auch gefallen