Sie sind auf Seite 1von 46

Design and Analysis of Algorithm

Course Code : 5009


Lecture 1

Dr. Huma Qayyum


Department of Software Engineering
huma.ayub@uettaxila.edu.pk
Course Objectives
• This course introduces students to the analysis and
design of computer algorithms. Upon completion of
this course, students will be able to do the following:
– Analyze the asymptotic performance of algorithms.
– Show a understanding with major algorithms and data
structures.
– Apply important algorithmic design examples and methods
of analysis.
Lecture # Lecture Contents Task
1. Introduction and Asymptotic Notations  
2. Mathematical Tools and Application  
3. Amortized analysis  
4. Inductive tools and algorithm correctness Quiz 01, Assignment 01
5. Minimum Cost Spanning trees  
6. Divide and Conquer  
7. Greedy method Quiz 02
8. Decrease-and-Conquer  
     
  MIDTERM  
9. Dynamic Programming I  
10. Dynamic Programming II  
11. BacktrackingI  
12. BacktrackingII Quiz 03, Assignment 02
13. Branch And Bound Techniques I  
14. Branch And Bound Techniques II  
15. NP Complete Problems I  
16. NP Complete Problems II Quiz 04
FINAL EXAMS
Course Information

• Office hours
– 9:00-11:00 Wednesday and Friday
• Grading policy
– Assign and Quiz. 20%, Midterm: 30%, Final: 50%
More Information
• Textbook
– Anany Levitin-Introduction to the Design and Analysis of Algorithms
-Addison Wesley (Pearson Education Inc.) (2011).pdf

• Others
– Introduction to Design & Analysis Computer Algorithm 3rd, Sara
Baase, Allen Van Gelder, Adison-Wesley, 2000.
– Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice Hall,
2004.
– Introduction to The Design and Analysis of Algorithms 2nd Edition,
Anany Levitin, Adison-Wesley, 2007.
An Algorithm
• An Algorithm is a finite sequence of instructions, each of which has a
clear meaning and can be performed with a finite amount of effort in
a finite length of time. No matter what the input values may be, an
algorithm terminates after executing a finite number of instructions.
In addition every algorithm must satisfy the following criteria:
• Input: there are zero or more quantities, which are externally
supplied; Output: at least one quantity is produced;
• Definiteness: each instruction must be clear and unambiguous;
• Finiteness: if we trace out the instructions of an algorithm, then for
all cases the algorithm will terminate after a finite number of steps;
• Effectiveness: every instruction must be sufficiently basic that it can
in principle be carried out by a person using only pencil and paper.
Algorithmics

It is the science that lets designers study and


evaluateALGORITHMICS
the effect of algorithms based on
various factors so that the best algorithm is
selected to meet a particular task in given
circumstances. It is also the science that tells
how to design a new algorithm for a particular
job.
What is a program?
• A program is the expression of an algorithm in
a programming language
• a set of instructions which the computer will
follow to solve a problem
Characteristics of an algorithm:-

·Must take an input.


· Must give some output(yes/ no, value etc.)
· Definiteness –each instruction is clear and
unambiguous.
· Finiteness –algorithm terminates after a finite
number of steps.
· Effectiveness –every instruction must be basic
i.e. simple instruction.
Let us write down the algorithm for a problem
that is familiar to us

Converting a decimal number into binary


Convert 75 to Binary
2 75 remainder
2 37 1
2 18 1
2 9 0
2 4 1
2 2 0
2 1 0
0 1

1001011
Algorithm for Decimal-to-Binary Conversion

1. Write the decimal number

2. Divide by 2; write quotient and remainder

3. Repeat step 2 on the quotient; keep on repeating


until the quotient becomes zero

4. Write all remainder digits in the reverse order (last


remainder first) to form the final result
Algorithm (Better Definition)
1st Definition:
Sequence of steps that can be taken to
solve a problem

Better Definition:
A precise sequence of a limited number
of unambiguous, executable steps that
terminates in the form of a solution
Analysis of Algorithms
• Analysis in the context of algorithms is concerned with
predicting the resources that are requires:
– Computational time
– Memory
• However, Time – generally measured in terms of the
number of steps required to execute an algorithm - is the
resource of most interest

• By analyzing several candidate algorithms, the most


efficient one(s) can be identified
Time taken by an algorithm?

• Performance measurement or Apostoriori Analysis


• Implementing the algorithm in a machine and
then calculating the time taken by the system to
execute the program successfully.
• Performance Evaluation or Apriori Analysis.
Before implementing the algorithm in a
system. This is done as follows
Selecting Among Algorithms

When choosing among competing, successful


solutions to a problem, choose the one which is the
least complex

This principle is called the “Ockham’s Razor,” after


William of Ockham - famous 13th century English
philosopher
• 1. How long the algorithm takes :-will be
represented as a function of the size of
• the input f(n)→how long it takes if ‘n’ is the size
of input.
• 2. How fast the function that characterizes the
running time grows with the input
• size. “Rate of growth of running time”.
• The algorithm with less rate of growth of running
time is considered better.
Looking for Efficiency
• Investment on computing equipment
• Investment on algorithmics
Investment on computing equipment
• Computer solves a particular problem in 10-4 x 2n sec
where n is size of instances
- If n = 10, 10-4 x 210 = 0.1024 sec
- If n = 20, 10-4 x 220 = 104.85 sec
- If n = 30, 10 x 230 = 107374 sec = 29.82 hours
-4

- If n = 38, 10-4 x 238 = 7635.49 hours = 254 days


• Purchase of new machine which solves the problem
in 10-6 x 2n sec
- If n = 38, 10-6 x 238 = 763.54 hours = 31 days
- If n = 45, 10-6 x 245 = 11 years
• No big gain on investment on new machine
Looking for Efficiency
Investment on algorithmics
• A cubic algorithm can solve the problem at very high
speed. An old machine can compute the result in 10 -2
xn 3
sec.
- If n = 10, 10-2 x 103 = 10 sec
- If n = 20, 10-2 x 203 = 80 sec
- If n = 30, 10-2 x 303 = 270 sec = 4.5 minutes
- If n > 200, 10-2 x 2003 = 1 day
• New algorithm offers a much greater improvement
than the purchase of new computing equipment
• New algorithm + new computing machine can provide
results 100 times faster and enables to solve
instances 4 or 5 times bigger than with the new
algorithm alone in the same length of time.
What kinds of problems are solved by
algorithms?
• Searching (Linear and Non-Linear)
• Sorting ( Elementary and Advanced)
• String Processing ( Pattern Matching, Parsing,
Compression, Cryptography)
• Optimization Problem ( Shortest routs, Minimum costs)
• Image Processing( Compression, Conversion, Matching)
• Data Mining Algorithms ( Clustering, Cleansing, Rules
Mining)
• Mathematical Algorithms ( Random Number generator,
Matrix Operation etc. )
Types of Algorithmic techniques
• Divide and Conquer
• Greedy Algorithms
• Dynamic Programming
• Heuristics
• Brute Force Technique
• Backtracking
Algorithm Representation

• Generally, SW developers represent them in one of


three forms:
– Pseudo code
– Flowcharts
– Actual code
Classification of Algorithms
Complexity of Algorithms

The complexity of an algorithm M is the function f(n) which gives the running time and/or
storage space requirement of the algorithm in terms of the size ‘n’ of the input data.
Mostly, the storage space required by an algorithm is simply a multiple of the data size ‘n’.
Complexity shall refer to the running time of the algorithm.
The function f(n), gives the running time of an algorithm, depends not only on the size ‘n’
of the input data but also on the particular data. The complexity function f(n) for certain
cases are:

1. Best Case : The minimum possible value of f(n) is called the best case.

2. Average Case : The expected value of f(n).

3. Worst Case : The maximum value of f(n) for any key possible input.
The field of computer science, which studies efficiency of algorithms, is known as analysis
of algorithms.

Algorithms can be evaluated by a variety of criteria. Most often we shall be interested in


the rate of growth of the time or space required to solve larger and larger instances of a
problem. We will associate with the problem an integer, called the size of the problem,
which is a measure of the quantity of input data.
Rate of Growth:

The following notations are commonly use notations in


performance analysis and used to characterize the complexity of an
algorithm:

1. Big–OH (O) 1,

2. Big–OMEGA (Ω),

3. Big–THETA () and

4. Little–OH (o)
Big–OH O (Upper Bound)
f(n) = O(g(n)), (pronounced order of or big oh), says that the growth rate of f(n) is less
than or equal (<) that of g(n).
Little–OH (o)
Analysis where Results Vary
Example: for some sorting algorithms, a sorting routine
may require as few as N-1 comparisons and as many
as 2
N
Types of analyses: 2
– Best-case: what is the fastest an algorithm can run for a
problem of size N?
– Average-case: on average how fast does an algorithm run
for a problem of size N?
– Worst-case: what is the longest an algorithm can run for a
problem of size N?
Computer scientists mostly use worst-case analysis
How to Compare Formulas?
Which is better: 3N  N  21  4  3 or
2 N

50 N 2  31N 3  24 N  15
Answer depends on value of N:
N 50 N 2  31N 3  24 N  15 3 N 2  N  21  4  3 N
1 120 37
2 511 71
3 1374 159
4 2895 397
5 5260 1073
6 8655 3051
7 13266 8923
8 19279 26465
9 26880 79005
10 36255 236527
What Happened?
3 N 2  N  21  4  3 N 4  3N %ofTotal
N
1 37 12 32.4
2 71 36 50.7
3 159 108 67.9
4 397 324 81.6
5 1073 972 90.6
6 3051 2916 95.6
7 8923 8748 98.0
8 26465 26244 99.2
9 79005 78732 99.7
10 236527 236196 99.9

– One term dominated the sum


As N Grows, Some Terms Dominate
Function 10 100 1000 10000 100000
log 2 N 3 6 9 13 16
N 10 100 1000 10000 100000
N log 2 N 30 664 9965 105 106

N2 102 104 106 108 1010


N3 103 106 109 1012 1015
2N 103 1030 10301 103010 1030103
Order of Magnitude Analysis
Measure speed with respect to the part of the sum
that grows quickest

50 N 2  31N 3  24 N  15

3 N 2  N  21  4  3 N
Ordering:

1  log 2 N  N  N log 2 N  N  N  2  3
2 3 N N
Order of Magnitude Analysis (cont.)
Furthermore, simply ignore any constants in front of
term and simply report general class of the term:
grows proportionally to when comparing algorithms,
determine formulas to count operation(s) of interest,
then compare dominant terms of formulas

50 N  31N  24 N  15
2 3 N3

3 N  N  21  4  3
2 N 3N
Calculating the running time of a program
Example 2
Example 3
Example 4
Insertion sort
Run this algorithm of following input
5 7 0 3 4 2 6 1
Example Insertion Sort
• 57034261

57034261 (0)
05734261 (2)
03574261 (2)
03457261 (2)
02345761 (4)
02345671 (1)
01234567 (6)
Analysis Of Recursive Binary Search

public int binarySearch (int target, int[] array,


int low, int high) {
if (low > high)
return -1;
else {
int middle = (low + high)/2;
if (array[middle] == target)
return middle;
else if(array[middle] < target)
return binarySearch(target, array, middle + 1, high);
else
return binarySearch(target, array, low, middle - 1);
}
}
• The recurrence relation for the running time of the method is:
T(1) = a if n = 1 (one element array)
T(n) = T(n / 2) + b if n > 1
Analysis Of Recursive Binary Search

Expanding:
T(n) = T(n / 2) + b
= [T(n / 4) + b] + b = T (n / 22) + 2b
= [T(n / 8) + b] + 2b = T(n / 23) + 3b
= ……..
= T( n / 2k) + kb

When n / 2k = 1  n = 2k  k = log2 n, we have:

T(n) = T(1) + b log2 n


= a + b log2 n

Therefore, Recursive Binary Search is O(log n)

Das könnte Ihnen auch gefallen