SS ZG519 - Data Structures and Algorithms Design: BITS Pilani

Lecture 1 22-Jul-2017
SS ZG519 - Data Structures

and Algorithms Design
BITS Pilani Sankha Mallick
Computer Science & Engineering
Pilani | Dubai | Goa | Hyderabad
BITS Pilani
Content Structure
SS ZG519 : Data Structures and 2

3/23/2019
Algorithms Design
Contact Sessions -
Contact Hour 1:
1.1 Theoretical Foundation –
– Algorithms and it’s Specification.
– Random Access Machine Model.
– Counting Primitive Operations.
– Notion of best case, average case and worst case.
1.2 Characterizing Run Time –

– Use of asymptotic notation.
– Big-Oh Notation, Little-Oh, Omega and Theta Notations.
Contact Hour 2:
1.3 Correctness of Algorithms.
1.4 Analyzing Recursive Algorithms –
– Recurrence relations.
– Specifying runtime of recursive algorithms.
– Solving recurrence equations.
1.5 Case Study: Analyzing Algorithms.
3
SS ZG519 : Data Structures and
3/23/2019
Algorithms Design
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
Contact Hour 3 :
2.1 Stacks –
– Stack ADT and Implementation.
– Applications.
Contact Hour 4 :
2.2 Queues –
– Queue ADT and Implementation.
– Applications.
Contact Hour 5 – 6 :
2.3 List –
– Notion of position in lists.
– List ADT and Implementation
4
3/23/2019
Algorithms Design
Contact Hour 7 – 13 :
3.1 Trees.
3.2 Binary Trees.
3.3 Binary Search Trees.
3.4 Height Balanced Trees / AVL Trees.
3.5 Multi way Tree.
3.6 Heaps.
Contact Hour 14 - 15 :
4.1 Unordered Dictionary.
4.2 Hash Tables.
4.3 Ordered Dictionary.
5
3/23/2019
Algorithms Design
Contact Hour 16 - 21 :
5.1 Greedy Method.
5.2 Divide & Conquer Method.
5.3 Dynamic Programming.
5.4 Graph Algorithms.
Contact Hour 22 :
6. Complexity Classes.
Course Review.
6
3/23/2019
Algorithms Design
BITS Pilani
Text Book(s) & Reference Book(s)

3/23/2019
Algorithms Design
T1 Algorithms Design: Foundations, Analysis and Internet Examples
Michael T. Goodrich, Roberto Tamassia, 2006, Wiley (Students
Edition)
R1 Data Structures, Algorithms and Applications in C++, Sartaj Sahni, Second

Ed, 2005, Universities Press
R2 Introduction to Algorithms, TH Cormen, CE Leiserson, RL Rivest, C Stein,

Third Ed, 2009, PHI
R3 Algorithm Design, Jon Kleinberg, Eva Tardos, First Ed., Pearson
8
3/23/2019
Algorithms Design
BITS Pilani
Evaluation Scheme

3/23/2019
Algorithms Design
No Name Type Duration Weight Day, Date, Session, Time
EC-1 Quiz-I/ Assignment-I Online - 5% August 26 to September 4, 2017
Quiz-II Online - 5% September 26 to October 4, 2017
Quiz-III Online - 5% September 20 to 30, 2017
Lab Online 10% September 20 to 30, 2017
EC-2 Mid-Semester Test Closed 2 hours 30% 23/09/2017 (AN) 2 PM – 4 PM

Book
EC-3 Comprehensive Exam Open Book 3 hours 45% 04/11/2017 (AN 2 PM – 5 PM
Syllabus for Mid-Semester Test (Closed Book): Topics in Session Nos. 1 to 11
Syllabus for Comprehensive Exam (Open Book): All topics (Session Nos. 1 to 22)
10
3/23/2019
Algorithms Design
BITS Pilani
Important links and information

3/23/2019
Algorithms Design
Elearn portal: https://elearn.bits-pilani.ac.in
Students are expected to visit the Elearn portal on a regular
basis and stay up to date with the latest announcements
and deadlines.
Contact sessions: Students should attend the online

lectures as per the schedule provided on the Elearn
portal.
12
3/23/2019
Algorithms Design
Evaluation Guidelines:
EC-1 consists of either two Assignments or three Quizzes. Students will

attempt them through the course pages on the Elearn portal.
Announcements will be made on the portal, in a timely manner.
For Closed Book tests: No books or reference material of any kind will be
permitted.
For Open Book exams: Use of books and any printed / written reference
material (filed or bound) is permitted. However, loose sheets of paper will
not be allowed. Use of calculators is permitted in all exams.
Laptops/Mobiles of any kind are not allowed. Exchange of any material is
not allowed.
13
3/23/2019
Algorithms Design
If a student is unable to appear for the Regular Test/Exam due to
genuine exigencies, the student should follow the procedure to
apply for the Make-Up Test/Exam which will be made available on
the Elearn portal. The Make-Up Test/Exam will be conducted only at
selected exam centres on the dates to be announced later.
It shall be the responsibility of the individual student to be regular in

maintaining the self study schedule as given in the course handout,
attend the online lectures, and take all the prescribed evaluation
components such as Assignment/Quiz, Mid-Semester Test and
Comprehensive Exam according to the evaluation scheme provided
in the handout.
14
3/23/2019
Algorithms Design
BITS Pilani
1.1 Theoretical Foundation

3/23/2019
Algorithms Design
Algorithm & Data Structure -
An algorithm is a step-by-step procedure for

solving a problem in a finite amount of time.
Input Algorithm Output
Data Structures: A systematic way of organizing and

accessing data.
--No single data structure works well for ALL purposes.
16
3/23/2019
Algorithms Design
Algorithm Criteria -
Input : Zero or more quantities that are externally applied.
Output : At least one quantity is produced.
Definiteness : Each instruction should be clear and unambiguous.
Finiteness : Algorithm terminates after finite number of steps for all test
cases.
Effectiveness : Each instruction is basic enough for a person to carried out

using a pen and paper. That means ensure not only definite but
also check whether feasible or not.
17
3/23/2019
Algorithms Design
Algorithm Descriptions -
Nature languages: Chinese, English, etc.

Pseudo-code: codes very close to computer
languages, e.g., C programming language.
Programs: C programs, C++ programs, Java programs.
Goal:
• Allow a well-trained programmer to be able to
implement.
• Allow an expert to be able to analyze the running
time.
18
3/23/2019
Algorithms Design
19
3/23/2019
Algorithms Design
20
3/23/2019
Algorithms Design
Pseudocode -
Example: find max

High-level description of
element of an array
an algorithm
More structured than Algorithm arrayMax(A, n)
English prose Input array A of n integers
Less detailed than a Output maximum element of A
program
currentMax ← A[0]
Preferred notation for for i ← 1 to n − 1 do
describing algorithms
if A[i] < currentMax then
Hides program design
currentMax ← A[i]
issues
return currentMax
21
3/23/2019
Algorithms Design
Pseudocode Details -
Control flow • Expressions

– if … then … [else …] ← Assignment
– while … do … (like = in Java)
– repeat … until … ← Equality testing
– for … do … (like == in Java)
– Indentation replaces braces n2 Superscripts and other
mathematical formatting
Method declaration allowed
Algorithm method (arg [, arg…])
Input …
Output …
22
3/23/2019
Algorithms Design
Primitive Operations (time unit) -
Basic computations performed

by an algorithm • Examples:
– Evaluating an
Identifiable in pseudocode expression
Largely independent from the – Assigning a value
to a variable
programming language
– Indexing into an
Exact definition not important array
(we will see why later) – Calling a method
Assumed to take a constant – Returning from a
method
amount of time in the RAM – Comparison x==y
model x>Y
23
3/23/2019
Algorithms Design
Problems and Algorithms -
Abstractly, a problem is just a function

p : Problem instances -> Solutions
Example: Sorting a list of integers
Problem instances = lists of integers
Solutions = sorted lists of integers
p : ` L-> Sorted version of L
An algorithm for p is a program which computes p.
There are three related questions which warrant consideration:
24
3/23/2019
Algorithms Design
Problems and Algorithms -
Question 1: Given an algorithm A to solve a problem p, how

“good” is A?
Question 2: Given a particular problem p for which several

algorithms are known to exist, which is best?
Question 3: Given a problem p, how does one design good

algorithms for p?
25
3/23/2019
Algorithms Design
Algorithm Analysis -
The main focus of algorithm analysis in this course will be upon the
“quality” of algorithms already known to be correct.
This analysis proceeds in two dimensions:
• Time complexity: The amount of time which execution of the
algorithm takes, usually specified as a function of the size of the
input
• Space complexity: The amount of space (memory) which
execution of the algorithm takes, usually specified as a function
of the size of the input.
Such analyses may be performed both experimentally and
analytically.
26
3/23/2019
Algorithms Design
BITS Pilani
Algorithm Classification

3/23/2019
Algorithms Design
Designing Algorithms -
It is important to study the process of designing good algorithms in

the first place.
There are two principal approaches to algorithm design.
• By problem: Study sorting algorithms, then scheduling
algorithms, etc.
• By strategy: Study algorithms by design strategy. Examples of
design strategies include:
Divide-and-conquer.
The greedy method.
Dynamic programming.
Backtracking.
Branch-and-bound.
28
3/23/2019
Algorithms Design
Classification by implementation -
Recursion or iteration: A recursive algorithm is one that invokes (makes

reference to) itself repeatedly until a certain condition matches. Iterative
algorithms use repetitive constructs like loops and sometimes additional
data structures like stacks to solve the given problems. Every recursive
version has an equivalent (but possibly more or less complex) iterative
version, and vice versa.
Logical: An algorithm may be viewed as controlled logical deduction. This

notion may be expressed as: Algorithm = logic + control.
29
3/23/2019
Algorithms Design
Serial or parallel or distributed: Algorithms are usually discussed with the assumption that computers
execute one instruction of an algorithm at a time. Those computers are sometimes called serial
computers. An algorithm designed for such an environment is called a serial algorithm. Parallel
algorithms take advantage of computer architectures where several processors can work on a
problem at the same time, whereas distributed algorithms utilize multiple machines connected with a
network. Parallel or distributed algorithms divide the problem into sub problems and collect the
results back together. Sorting algorithms can be parallelized efficiently, but their communication
overhead is expensive. Iterative algorithms are generally parallelizable. Some problems have no
parallel algorithms, and are called inherently serial problems.
Deterministic or non-deterministic: Deterministic algorithms solve the problem with exact decision at
every step of the algorithm whereas non-deterministic algorithm solves problems via guessing
although typical guesses are made more accurate through the use of heuristics.
Exact or approximate: While many algorithms reach an exact solution, approximation algorithms seek
an approximation that is close to the true solution. Approximation may use either a deterministic or a
random strategy.
30
3/23/2019
Algorithms Design
Classification by Design Paradigm -
Divide and conquer : A divide and conquer algorithm repeatedly reduces an

instance of a problem to one or more smaller instances of the same problem
(usually recursively), until the instances are small enough to solve easily. One
such example of divide and conquer is merge sorting. Sorting can be done on
each segment of data after dividing data into segments and sorting of entire
data can be obtained in conquer phase by merging them. A simpler variant
of divide and conquer is called decrease and conquer algorithm, that solves
an identical sub problem and uses the solution of this sub problem to solve
the bigger problem. Divide and conquer divides the problem into multiple
sub problems and so conquer stage will be more complex than decrease and
conquer algorithms. An example of decrease and conquer algorithm is binary
search algorithm.
31
3/23/2019
Algorithms Design
Dynamic programming : When a problem shows optimal substructure, meaning the
optimal solution to a problem can be constructed from optimal solutions to sub problems,
and overlapping sub problems, meaning the same sub problems are used to solve many
different problem instances, a quicker approach called dynamic programming avoids
recomputing solutions that have already been computed. For example, the shortest path
to a goal from a vertex in a weighted graph can be found by using the shortest path to the
goal from all adjacent vertices.
The main difference between dynamic programming and divide and conquer is that sub
problems are more or less independent in divide and conquer, whereas sub problems
overlap in dynamic programming.
32
3/23/2019
Algorithms Design
Greedy method : A greedy algorithm is similar to a dynamic programming algorithm,
but the difference is that solutions to the sub problems do not have to be known at
each stage; instead a "greedy" choice can be made of what looks best for the
moment. The greedy method extends the solution with the best possible decision
(not all feasible decisions) at an algorithmic stage based on the current local
optimum and the best decision (not all possible decisions) made in previous stage. It
does not give accurate answer to many problems. But when it works, it will be the
fastest method. The most popular greedy algorithm is finding the minimal spanning
tree as given by Kruskal.
33
3/23/2019
Algorithms Design
Reduction - This technique involves solving a difficult problem by
transforming it into a better known problem for which we have (hopefully)
asymptotically optimal algorithms. The goal is to find a reducing algorithm
whose complexity is not dominated by the resulting reduced algorithm's. For
example, one selection algorithm for finding the median in an unsorted list
involves first sorting the list (the expensive portion) and then pulling out the
middle element in the sorted list (the cheap portion). This technique is also
known as transform and conquer.
Search and enumeration - Many problems (such as playing chess) can be

modeled as problems on graphs. A graph exploration algorithm specifies rules
for moving around a graph and is useful for such problems.
34
3/23/2019
Algorithms Design
Probabilistic and heuristic paradigm - Algorithms belonging to this class fit the
definition of an algorithm more loosely.
Probabilistic algorithms are those that make some choices randomly (or pseudo-
randomly); for some problems, it can in fact be proven that the fastest solutions must
involve some randomness.
Genetic algorithms attempt to find solutions to problems by biological evolutionary

processes, with a cycle of random mutations yielding successive generations of
"solutions". Thus, they emulate reproduction and "survival of the fittest".
Heuristic algorithms, whose general purpose is not to find an optimal solution, but an
approximate solution where the time or resources are limited. They are not practical to
find perfect solutions. An example of this would be local search, or simulated
annealing.
35
3/23/2019
Algorithms Design
BITS Pilani
Analysis of Algorithms

3/23/2019
Algorithms Design
Analysis of Algorithms -
• Estimate the running time.

• Estimate the memory space required.
Time and space depend on the input size.
37
3/23/2019
Algorithms Design
Running Time -
Most algorithms transform

input objects into output
objects.
The running time of an
algorithm typically grows
with the input size.
Average case time is often
difficult to determine.
We focus on the worst case
running time.
– Easier to analyze
– Crucial to applications such as
games, finance and robotics
38
3/23/2019
Algorithms Design
Experimental Studies -
Write a program
implementing the algorithm
Run the program with inputs
of varying size and
composition
Use a method like
System.currentTimeMillis() to
get an accurate measure
of the actual running time
Plot the results
39
3/23/2019
Algorithms Design
Limitations of Experiments -
It is necessary to implement the algorithm, which may be

difficult.
Results may not be indicative of the running time on other

inputs not included in the experiment.
In order to compare two algorithms, the same hardware

and software environments must be used
40
3/23/2019
Algorithms Design
Random Access
Machine(RAM) Model -
41
3/23/2019
Algorithms Design
Instead of trying to determine the specific execution time of each primitive
operation, we will simply count how many primitive operations are executed,
and use this number as a high-level estimate of the running time of the
algorithm.
42
3/23/2019
Algorithms Design
Counting Primitive Operations -
By inspecting the pseudocode, we can determine the

maximum number of primitive operations executed by
an algorithm, as a function of the input size
Algorithm arrayMax(A, n)
Input array A of n integers
Output maximum element of A
currentMax ← A[0]
for i ← 1 to n − 1 do
if A[i] < currentMax then
currentMax ← A[i]
return currentMax
43
3/23/2019
Algorithms Design
44
3/23/2019
Algorithms Design
45
3/23/2019
Algorithms Design
Estimating Running Time -
Algorithm arrayMax executes 7n − 2 primitive operations in

the worst case and 5n primitive operations in the best
case.
Define:
a = Time taken by the fastest primitive operation
b = Time taken by the slowest primitive operation
Let T(n) be the time of arrayMax. Then
a (5n) ≤ T(n) ≤ b(7n − 2)
Hence, the running time T(n) is bounded by two linear
functions.
46
3/23/2019
Algorithms Design
Growth Rate of Running Time -
Changing the hardware/ software

environment
– Affects T(n) by a constant factor, but
– Does not alter the growth rate of T(n)
47
3/23/2019
Algorithms Design
The Growth Rate of the Six Popular
functions -
n logn n nlogn n2 n3 2n
4 2 4 8 16 64 16
8 3 8 24 64 512 256
16 4 16 64 256 4,096 65,536
32 5 32 160 1,024 32,768 4,294,967,296
64 6 64 384 4,094 262,144 1.84 * 1019
128 7 128 896 16,384 2,097,152 3.40 * 1038
256 8 256 2,048 65,536 16,777,216 1.15 * 1077
512 9 512 4,608 262,144 134,217,728 1.34 * 10154
1024 10 1,024 10,240 1,048,576 1,073,741,824 1.79 * 10308 48

3/23/2019
Algorithms Design
In analysing algorithms we will not consider the
following information although they are very important:
➢The machine we are executing on.
➢Its machine language instruction set.
➢The time required by each machine language.
➢The translation, a compiler will make from the

source code to the machine language.
49
3/23/2019
Algorithms Design
Consider the three program segments:
x = x + 1;
for( i = 1; i <= n; i++)

x = x + 1;
for( i = 1; i <= n; i++)

for( j = 1; i <= n; j++)
x = x + 1;
50
3/23/2019
Algorithms Design
❖In the first program segment, we assume that
the statement x = x + 1 is not contained within
any loop. Then its frequency count is one.
❖In the second program segment, the same

statement will be executed n times.
❖In the third program segment, it will be

executed n2 times.
51
3/23/2019
Algorithms Design
Hence, 1 , n and n2 are said to be different and increasing
orders of magnitude.
52
3/23/2019
Algorithms Design
BITS Pilani
References
53
3/23/2019 SS ZG519 : Data Structures and Algorithms Design

Algorithms Design: Foundations, Analysis and Internet
Examples Michael T. Goodrich, Roberto Tamassia,
2006, Wiley (Students Edition).
54
3/23/2019
Algorithms Design
BITS Pilani
THANKS
55
3/23/2019 SS ZG519 : Data Structures and Algorithms Design

SS ZG519 - Data Structures and Algorithms Design: BITS Pilani

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

SS ZG519 - Data Structures and Algorithms Design: BITS Pilani

Hochgeladen von

Copyright:

Verfügbare Formate

Lecture 1 22-Jul-2017

SS ZG519 - Data Structures

SS ZG519 : Data Structures and 2

1.2 Characterizing Run Time –

Text Book(s) & Reference Book(s)

SS ZG519 : Data Structures and 7

R1 Data Structures, Algorithms and Applications in C++, Sartaj Sahni, Second

R2 Introduction to Algorithms, TH Cormen, CE Leiserson, RL Rivest, C Stein,

R3 Algorithm Design, Jon Kleinberg, Eva Tardos, First Ed., Pearson

SS ZG519 : Data Structures and 9

Quiz-II Online - 5% September 26 to October 4, 2017

Quiz-III Online - 5% September 20 to 30, 2017

Lab Online 10% September 20 to 30, 2017

EC-2 Mid-Semester Test Closed 2 hours 30% 23/09/2017 (AN) 2 PM – 4 PM

EC-3 Comprehensive Exam Open Book 3 hours 45% 04/11/2017 (AN 2 PM – 5 PM

Syllabus for Mid-Semester Test (Closed Book): Topics in Session Nos. 1 to 11

Important links and information

SS ZG519 : Data Structures and 11

Contact sessions: Students should attend the online

EC-1 consists of either two Assignments or three Quizzes. Students will

It shall be the responsibility of the individual student to be regular in

1.1 Theoretical Foundation

SS ZG519 : Data Structures and 15

An algorithm is a step-by-step procedure for

Input Algorithm Output

Data Structures: A systematic way of organizing and

Input : Zero or more quantities that are externally applied.

Output : At least one quantity is produced.

Definiteness : Each instruction should be clear and unambiguous.

Effectiveness : Each instruction is basic enough for a person to carried out

Nature languages: Chinese, English, etc.

Example: find max

Control flow • Expressions

Basic computations performed

Abstractly, a problem is just a function

Question 1: Given an algorithm A to solve a problem p, how

Question 2: Given a particular problem p for which several

Question 3: Given a problem p, how does one design good

SS ZG519 : Data Structures and 27

It is important to study the process of designing good algorithms in

Recursion or iteration: A recursive algorithm is one that invokes (makes

Logical: An algorithm may be viewed as controlled logical deduction. This

Divide and conquer : A divide and conquer algorithm repeatedly reduces an

Search and enumeration - Many problems (such as playing chess) can be

Genetic algorithms attempt to find solutions to problems by biological evolutionary

SS ZG519 : Data Structures and 36

• Estimate the running time.

Time and space depend on the input size.

Most algorithms transform

It is necessary to implement the algorithm, which may be

Results may not be indicative of the running time on other

In order to compare two algorithms, the same hardware

By inspecting the pseudocode, we can determine the

Algorithm arrayMax executes 7n − 2 primitive operations in

Changing the hardware/ software

16 4 16 64 256 4,096 65,536

32 5 32 160 1,024 32,768 4,294,967,296

64 6 64 384 4,094 262,144 1.84 * 1019

128 7 128 896 16,384 2,097,152 3.40 * 1038

256 8 256 2,048 65,536 16,777,216 1.15 * 1077

512 9 512 4,608 262,144 134,217,728 1.34 * 10154

1024 10 1,024 10,240 1,048,576 1,073,741,824 1.79 * 10308 48

➢The machine we are executing on.