1

Master of Computer Application (MCA) – Semester 4

MC0080 – Analysis and Design of Algorithms – 4 Credits (Book ID:
B0891)
Assignment Set – 1 (60 Marks)
Answer the following : 6*10 = 6

1. Describe the characteristics of algorithms? Explain the procedure and

recursion in algorithm?

Let us try to present the scenario of a man brushing his own teeth(natural denture) as an algorithm as
follows.

Step 1. Take the brush

Step 2. Apply the paste

Step 3. Start brushing

Step 4. Rinse

Step 5. Wash

Step 6. Stop

If one goes through these 6 steps without being aware of the statement of the problem, he could possibly
feel that this is the algorithm for cleaning a toilet. This is because of several ambiguities while
comprehending every step. The step 1 may imply tooth brush, paint brush, toilet brush etc. Such an
ambiguity doesn’t an instruction an algorithmic step. Thus every step should be made unambiguous. An
unambiguous step is called definite instruction. Even if the step 2 is rewritten as apply the tooth paste, to
eliminate ambiguities yet the conflicts such as, where to apply the tooth paste and where is the source of
the tooth paste, need to be resolved. Hence, the act of applying the toothpaste is not mentioned.
Although unambiguous, such unrealizable steps can’t be included as algorithmic instruction as they are
not effective.

The definiteness and effectiveness of an instruction implies the successful termination of that instruction.
However the above two may not be sufficient to guarantee the termination of the algorithm. Therefore,
while designing an algorithm care should be taken to provide a proper termination for algorithm.

Thus, every algorithm should have the following five characteristic feature

1. Input

2. Output

3. Definiteness

4. Effectiveness

5. Termination

Therefore, an algorithm can be defined as a sequence of definite and effective instructions, which
terminates with the production of correct output from the given input.

In other words, viewed little more formally, an algorithm is a step by step formalization of a mapping
function to map input set onto an output set.
 2

Procedures

A procedure is a recipe or method for accomplishing a result such as solving a problem or performing a
task. A procedure is usually considered to have the following characteristics:

• It consists of a finite sequence of discrete steps.

• Each step is an operation or instruction that can be performed by the agent or device expected
to carry out the procedure.
• Each operation or instruction is finite in that it is represented by a finite number of symbols.
• The device or agent can carry out the procedure automatically, without additional instructions or
resort to outside guidance, by simply performing the sequence of operations comprising the
procedure.

Thus, a procedure is a step by step method for performing a task or solving a problem, presented with
sufficient precision and detail, and in an appropriate form and language, that it is completely and
unambiguously interpretable and executable by the particular agent or device intended to perform the
procedure. Furthermore, all possible circumstances that might arise in the course of performing the
procedure are taken into account so that the agent or device can carry out the procedure automatically
without requiring any additional instructions.

Procedures consist of a finite number of discrete steps each of which is comprised of a rule, or an
instruction or operation that can be applied, performed or executed by the device or agent that is
expected to carry out the procedure. Although the steps of a procedure are ordered in a sequence, and
are performed in this order, some steps can consist of tests such as those required to determine whether
or not a problem has been solved, or a task has been completed. Such tests can also determine what
particular sequences of steps in the procedure are to be performed when certain conditions are met,
rather than others which might be performed under different conditions.

Other steps in a procedure can consist of instructions that cause sequences of operations or instructions
to be repeated or iterated a number of times until specified conditions are met. Consequently, although a
procedure might consist of a relatively small number of steps, some of these steps can be repeated so
that the device or agent performing the procedure may actually execute a relatively large number of
operations to carry out some tasks or solve some problems.

Procedures may be devised that build or change things in the real world. These procedures might be
performed by machines or by human beings. The procedures of interest here, however, are computational
in nature. By "computational" in this context, it is meant that steps of a computational procedure that a
machine or a human being performs can consist of operations such as the reading, writing, comparing,
erasing, and replacing of symbols. The operations of a computational procedure could include storing
symbols in, and retrieving them from memory; but these operations are simply special kinds of writing
and reading operations. The symbols of interest to us include graphemes or word forms, and symbols such
as syntactic category labels.

Procedures might also be used as models of the operation of physical systems. For example, some
procedures are devised as computational models of the sequences of operations that the cognitive system
is conjectured to perform as it processes sentences. The operations comprising the steps of such
procedures are determined by the psychological evidence that might be available, and by the
psycholinguistic theory of sentence processing that is applied in developing the model.

Equivalent Procedures. Several different procedures might be devised to solve the same problem or
perform the same task. Thus, there is normally no unique procedure for a given problem or task;
however, since they solve the same problem or perform the same task, the different procedures can be
described as equivalent. That is,

Two procedures are equivalent just in case they accomplish the same result (that is, they perform the
same task(s) or solve the same problem(s)).
The selection of a particular, preferred procedure from among several equivalent procedures is often
based upon considerations of relative efficiency. A procedure might be said to be more efficient than
 3

another if fewer steps must be performed, and hence less time is required to solve the problem or
perform the task.

Decision Procedures. A typical problem to which a procedure is applied in computational linguistics is

that of determining whether or not a given string a over a vocabulary VT is a sentence in the language
L(VT). A procedure of this nature will report "yes" if a Î L(VT), but will report "no" if a Ï L(VT); that is, the
procedure will display "no" if a Î Lc(VT), where Lc(VT) = V*T - L(VT), the complement of the language L(VT)
relative to the closure of the vocabulary VT.

A procedure of this kind is called a decision procedure; that is, if a problem requires only a "yes" or "no"
answer, then a procedure designed to solve the problem is called a decision procedure. A decision
procedure for a language L(VT) can be called a recogniser or acceptor for L(VT).

For any string a Î V*T, an acceptor A for the language L(VT) reports "yes," and is said to accept a, just in
case a Î L(VT); otherwise, A reports "no."

Equivalence of Procedures and Grammars. If there is a grammar G that licenses all, and only, the
sentences of L(VT), we can write L(G) = L(VT), where L(G) denotes the language licensed by G. Then an
acceptor for L(VT) is an acceptor for L(G). An acceptor A for a language L(G) is said to be equivalent to G.
In other words, we can say that

A procedure A and a grammar G are equivalent, and A is an acceptor for L(G), just in case A accepts all,
and only, the sentences in L(G).
An acceptor A for the language L(G) licensed by a grammar G can be called an acceptor for G. Although
the A may be equivalent to G, A might not perform operations that correspond to the rules of G.

Some grammars license syntactic structures or derivation trees for the sentences that they license. These
trees are made up of subtrees each of which consists of a mother node and its daughter nodes. Each of
these subtrees corresponds to a rule of the grammar.

Consider that G is a grammar that licenses derivation trees for the sentences it licenses, and that A is an
acceptor for G. Then, assume that, for each sentence licensed by G, A produces the derivation tree (or
trees) licensed by G. The procedure A can be called a parser for G.

Although the operations comprising the procedure A may not be identical to the rules of grammar G, the
operations of A at least have the same effect as the rules of G. We can therefore claim that the
operations of A correspond to the rules of G, and that A and G are strongly equivalent. In other words, we
can say that

A procedure A and a grammar G are strongly equivalent, and A is a parser for G, just in case A produces
all, and only, the derivation trees licensed by G.

In some psycholinguistic models, the grammar for a language is assumed to provide evidence for the
operations performed by the human language processor as it produces or analyses sentences. This
assumption has been formulated by Bresnan and Kaplan as the Strong Competence Hypothesis:

There is a direct correspondence between the rules of a grammar and the operations performed by the
human language processor.
Underlying this hypothesis is the further assumption that the language processing components of the
cognitive system must perform a procedure to produce and analyse sentences. The Strong Competence
Hypothesis is thus a statement of the strong equivalence of this neural procedure and a grammar.

Turing's Thesis. It was stated in the introductory paragraph of this note that grammars do not necessarily
include instructions that determine how their rules are applied to analyse sentences. Thus, in
computational and psycholinguistic models of sentence analysis, it is often the case that procedures are
devised that specify how the rules of a chosen grammar are applied. Different procedures might be
developed on the basis of different psychological or computational theories. But it is generally assumed
 4

that these procedures can be implemented by writing computer programs that perform the operations of
the procedure.

This assumption is well-founded in that no computational procedure has been found that cannot be
implemented as a program that runs on a computer with sufficient resources. It is nonetheless an
assumption: it has never been proven that a computer with sufficient resources can perform any given
computational procedure. This assumption is essentially Turing's Thesis, which may be paraphrased as
follows:

A Turing Machine can perform any computational procedure.

Turing never proved his thesis; but no counterexample has yet been identified.

One can, in principle, write a program to be executed by a general-purpose digital computer with
sufficient resources that emulates the operation of a Turing Machine. The "sufficient resources" here
refers to the availability of a memory device that can emulate the infinite tape of a Turing Machine. It
turns out in practice that, for the procedures we normally devise, the infinite memory of a Turing
Machine is not required. Thus, by making an assumption comparable to Turing's thesis, we can claim that
it is possible to write a computer program that is equivalent to a given computational procedure, where

A computer program C is equivalent to a computational procedure A just in case C achieves the same
result as A (that is, C solves the same problem(s) or performs the task(s) as A).
If the operations comprising the procedure A can be emulated by a computer program C, then we can say
that C is strongly equivalent to A. In other words
A computer program C and a computational procedure A are strongly equivalent just in case C is
equivalent to A and C emulates the operations of A.
If a computer program C is equivalent to a procedure A, and if A is an acceptor for a grammar G, then C is
an acceptor for G. We can therefore state that
A computer program C and a grammar G are equivalent just in case C accepts all, and only, the sentences
in L(G).
Furthermore, if a computer program C is strongly equivalent to a procedure A, and if A is an parser for a
grammar G, then C is a parser for G.
Hence, we can say that A computer program C and a grammar G are strongly equivalent just in case C
produces all, and only, the derivation trees licensed by G.

Algorithms

A procedure, by definition, can consist of only a finite number of steps; but, as a consequence of
iteration, some sequences of steps may be performed infinitely many times, depending upon the
particular problem or task, or perhaps upon procedure itself. Thus, although it might in fact be the case
that a string a is a sentence in the language L(VT), it could require infinitely many steps of the procedure
to confirm that a Î L(VT). Furthermore, if a Ï L(VT), it might take an infinite number of steps to establish
that a Î Lc(VT). Consequently, because infinitely many steps must be performed to solve some problems, a
procedure might not yield a solution, and stop, until infinite time has elapsed.

An algorithm is a procedure for solving a problem or performing a task that will always stop within a finite
time. Thus, if the problem has a solution, or the task has an attainable goal, an algorithm will yield the
solution or achieve the goal in a finite number of steps, and hence, in finite time. Furthermore, an
algorithm will also stop after finitely many steps have been executed if the the problem actually has no
solution or the goal of the task is unattainable. Algorithms are therefore sometimes called effective
procedures.

A decision procedure that requires at most finitely many steps to yield a "yes" or "no" result is called a
decision algorithm. For example, a decision algorithm will require a finite amount of time to determine
whether or not an arbitrary string a over a vocabulary VT is a sentence in the language L(VT).

If a decision algorithm can be devised for the problem of determining whether or not a Î L(VT), then this
problem is said to be algorithmically decidable. The language L(VT) itself is said to be recursively
decidable or recursive.
 5

It is possible that a decision procedure will report "yes" after at most finitely many steps when a string a Î
L(VT); but if a Ï L(VT), the procedure may not stop in a finite number of steps. Infinitely many steps might
be required in such cases to establish that a Î Lc(VT). Procedures of this nature are called decision semi-
algorithms. A language L(VT) for which there is at best only a semi-algorithm to determine whether or not
a string a Î L(VT) is not recursively decidable.

It is inherent in the nature of some problems and tasks that, although they may have solutions or
attainable goals, any procedure could require infinitely many steps to yield a solution or goal. No
algorithm, or semi-algorithm, can be devised to solve such problems. Although it is possible that some
particular solutions might be discovered within a finite number of steps, such problems are considered to
be intractable in general.

Problems for which algorithms can be devised to solve them are considered to be tractable in principle.
The caveat "in principle" is attached because, although algorithms yield solutions in finitely many steps,
the actual number of steps performed, and consequently the time required, can be quite large. In such
cases, a problem might be intractable in practice. For example, the time needed to determine whether
or not a string a Î L(VT) could increase exponentially with the length of a so that, notwithstanding L(VT) is
recursively decidable, the time required could be extremely large for longer strings.

2. If ,

3. Explain the concept of bubble sort and also write the algorithm for bubble sort.

If you are sorting content into an order, one of the most simple techniques that exists is the bubble sort
technique. It works by repeatedly stepping through the list to be sorted, comparing each pair of adjacent
items and swapping them if they are in the wrong order. The pass through the list is repeated until no
swaps are needed, which indicates that the list is sorted.

Step-by-step example

Let us take the array of numbers "5 1 4 2 8", and sort the array from lowest number to greatest number
using bubble sort algorithm. In each step, elements written in bold are being compared.
First Pass:
( 5 1 4 2 8 ) ( 1 5 4 2 8 ), Here, algorithm compares the first two elements, and swaps them.
( 1 5 4 2 8 ) ( 1 4 5 2 8 ), Swap since 5 > 4
( 1 4 5 2 8 ) ( 1 4 2 5 8 ), Swap since 5 > 2
( 1 4 2 5 8 ) ( 1 4 2 5 8 ), Now, since these elements are already in order (8 > 5), algorithm does not swap
them.
Second Pass:
(14258)(14258)
( 1 4 2 5 8 ) ( 1 2 4 5 8 ), Swap since 4 > 2
(12458)(12458)
(12458)(12458)
Now, the array is already sorted, but our algorithm does not know if it is completed. The algorithm needs
one whole pass without any swap to know it is sorted.
Third Pass:
(12458)(12458)
(12458)(12458)
(12458)(12458)
(12458)(12458)
Finally, the array is sorted, and the algorithm can terminate.
 6

4. Prove that “If n >= 1, then for any n-key, B-tree T of height hand minimum degree t

>=2 ,

5. Explain briefly the concept of breadth-First search(BFS)

Breadth First Search

The Breadth First search is an extremely useful searching technique. It differs from the depth-first search
in that it uses a queue to perform the search, so the order in which the nodes are visited is quite
different. It has the extremely useful property that if all of the edges in a graph are unweighted (or the
same weight) then the first time a node is visited is the shortest path to that node from the source node.
You can verify this by thinking about what using a queue means to the search order. When we visit a node
and add all the neighbors into the queue, then pop the next thing off of the queue, we will get the
neighbors of the first node as the first elements in the queue. This comes about naturally from the FIFO
property of the queue and ends up being an extremely useful property. One thing that we have to be
careful about in a Breadth First search is that we do not want to visit the same node twice, or we will lose
the property that when a node is visited it is the quickest path to that node from the source.

The basic structure of a breadth first search will look this:

void bfs(node start) {
queue s;
s.push(start);
while (s.empty() == false) {
top = s.front();
s.pop();
mark top as visited;
check for termination condition (have we reached the node we want to?) add all of top's unvisited
neighbors to the stack.
}
}
Notice the similarities between this and a depth-first search, we only differ in the data structure used and
we don't mark top as unvisited again.

The problem we will be discussing in relation to the Breadth First search is a bit harder than the previous
example, as we are dealing with a slightly more complicated search space. The problem is the 1000 from
Division 1 in SRM 156, Pathfinding. Once again we will be dealing in a grid-based problem, so we can
represent the graph structure implicitly within the grid.

A quick summary of the problem is that we want to exchange the positions of two players on a grid. There
are impassable spaces represented by 'X' and spaces that we can walk in represented by ' '. Since we have
two players our node structure becomes a bit more complicated, we have to represent the positions of
person A and person B. Also, we won't be able to simply use our array to represent visited positions any
more, we will have an auxiliary data structure to do that. Also, we are allowed to make diagonal
movements in this problem, so we now have 9 choices, we can move in one of 8 directions or simply stay
in the same position. Another little trick that we have to watch for is that the players can not just swap
positions in a single turn, so we have to do a little bit of validity checking on the resulting state.

First, we set up the node structure and visited array:

class node {
int player1X, player1Y, player2X, player2Y;
int steps; // The current number of steps we have taken to reach this step
}

bool visited[20][20][20][20];
Here a node is represented as the (x,y) positions of player 1 and player 2. It also has the current steps
that we have taken to reach the current state, we need this because the problem asks us what the
minimum number of steps to switch the two players will be. We are guaranteed by the properties of the
Breadth First search that the first time we visit the end node, it will be as quickly as possible (as all of our
edge costs are 1).
 7

The visited array is simply a direct representation of our node in array form, with the first dimension
being player1X, second player1Y, etc. Note that we don't need to keep track of steps in the visited array.

Now that we have our basic structure set up, we can solve the problem (note that this code is not
compilable):
int minTurns(String[] board) {
int width = board[0].length;
int height = board.length;

node start;
// Find the initial position of A and B, and save them in start.

queue q;
q.push(start);
while (q.empty() == false) {
node top = q.front();
q.pop();

// Check if player 1 or player 2 is out of bounds, or on an X square, if so continue

// Check if player 1 or player 2 is on top of each other, if so continue

// Make sure we haven't already visited this state before

if (visited[top.player1X][top.player1Y][top.player2X][top.player2Y]) continue;
// Mark this state as visited
visited[top.player1X][top.player1Y][top.player2X][top.player2Y] = true;

// Check if the current positions of A and B are the opposite of what they were in start.
// If they are we have exchanged positions and are finished!
if (top.player1X == start.player2X && top.player1Y == start.player2Y &&
top.player2X == start.player1X && top.player2Y == start.player1Y)
return top.steps;

// Now we need to generate all of the transitions between nodes, we can do this quite easily using some
// nested for loops, one for each direction that it is possible for one player to move. Since we need
// to generate the following deltas: (-1,-1), (-1,0), (-1,1), (0,-1), (0,0), (0,1), (1,-1), (1,0), (1,1)
// we can use a for loop from -1 to 1 to do exactly that.
for (int player1XDelta = -1; player1XDelta <= -1; player1XDelta++) {
for (int player1YDelta = -1; player1YDelta <= -1; player1YDelta++) {
for (int player2XDelta = -1; player2XDelta <= -1; player2XDelta++) {
for (int player2YDelta = -1; player2YDelta <= -1; player2YDelta++) {
// Careful though! We have to make sure that player 1 and 2 did not swap positions on this turn
if (top.player1X == top.player2X + player2XDelta && top.player1Y == top.player2Y + player2YDelta &&
top.player2X == top.player1X + player1XDelta && top.player2Y == top.player1Y + player1YDelta)
continue;

// Add the new node into the queue

q.push(node(top.player1X + player1XDelta, top.player1Y + player1YDelta,
top.player2X + player2XDelta, top.player2Y + player2YDelta,
top.steps + 1));
}
}
}
}
}

// It is not possible to exchange positions, so

// we return -1. This is because we have explored
// all the states possible from the starting state,
// and haven't returned an answer yet.
return -1;
}
This ended up being quite a bit more complicated than the basic Breadth First search implementation
 8

6. Explain Kruskal’s Algorithm.

Let G = (V, E) be the given graph, with | V| = n

Start with a graph T = (V, ) consisting of only the

vertices of G and no edges; /* This can be viewed as n

connected components, each vertex being one connected component */

Arrange E in the order of increasing costs;

for (i = 1, i n - 1, i + +)

{ Select the next smallest cost edge;

if (the edge connects two different connected components)

add the edge to T;

• At the end of the algorithm, we will be left with a single component that comprises all the
vertices and this component will be an MST for G. Proof of Correctness of Kruskal's Algorithm

Theorem: Kruskal's algorithm finds a minimum spanning tree.

Proof: Let G = (V, E) be a weighted, connected graph. Let T be the edge set that is grown in
Kruskal's algorithm. The proof is by mathematical induction on the number of edges in T.

o We show that if T is promising at any stage of the algorithm, then it is still promising
when a new edge is added to it in Kruskal's algorithm
o When the algorithm terminates, it will happen that T gives a solution to the problem
and hence an MST.
 9

Basis: T = is promising since a weighted connected graph always has at least one MST.

Induction Step: Let T be promising just before adding a new edge e = (u, v). The edges T divide
the nodes of G into one or more connected components. u and v will be in two different
components. Let U be the set of nodes in the component that includes u. Note that

o U is a strict subset of V
o T is a promising set of edges such that no edge in T leaves U (since an edge T either has
both ends in U or has neither end in U)
o e is a least cost edge that leaves U (since Kruskal's algorithm, being greedy, would have
chosen e only after examining edges shorter than e)

The above three conditions are precisely like in the MST Lemma and hence we can conclude that
the T {e} is also promising. When the algorithm stops, T gives not merely a spanning tree but
a minimal spanning tree since it is promising.

Figure 8.13: An illustration of Kruskal's algorithm

 10

• Program

void kruskal (vertex-set V; edge-set E; edge-set T)

int ncomp; /* current number of components */

priority-queue edges /* partially ordered tree */

 11

mfset components; /* merge-find set data structure */

vertex u, v; edge e;

int nextcomp; /* name for new component */

int ucomp, vcomp; /* component names */

makenull (T); makenull (edges);

nextcomp = 0; ncomp = n;

for (v V) /* initialize a component to have one vertex of V*/

{ nextcomp++ ;

initial (nextcomp, v, components);

for (e E)

insert (e, edges); /* initialize priority queue of edges */

while (ncomp > 1)

e = deletemin (edges);

let e = (u, v);

ucomp = find(u, components);

vcomp = find(v, components);

if (ucomp! = vcomp)

merge (ucomp, vcomp, components);

ncomp = ncomp - 1;

}
 12

Implementation

• Choose a partially ordered tree for representing the sorted set of edges
• To represent connected components and interconnecting them, we need to implement:

1.
MERGE (A, B, C) . . . merge components A and B in C and call the result A or B arbitrarily.
2.
FIND (v, C) . . . returns the name of the component of C of which vertex v is a member. This
operation will be used to determine whether the two vertices of an edge are in the same or in
different components.
3.
INITIAL (A, v, C) . . . makes A the name of the component in C containing only one vertex,
namely v

• The above data structure is called an MFSET

Running Time of Kruskal's Algorithm

• Creation of the priority queue

*
If there are e edges, it is easy to see that it takes O(elog e) time to insert the edges into a
partially ordered tree
*
O(e) algorithms are possible for this problem

• Each deletemin operation takes O(log e) time in the worst case. Thus finding and deleting least-
cost edges, over the while iterations contribute O(log e) in the worst case.
• The total time for performing all the merge and find depends on the method used.

O(elog e) without path compression

O(e (e)) with the path compression, where

(e) is the inverse of an Ackerman function.

Example: See Figure 8.13.

E = {(1,3), (4,6), (2,5), (3,6), (3,4), (1,4), (2,3), (1,2), (3,5), (5,6) }

Master of Computer Application (MCA) – Semester 4

MC0080 – Analysis and Design of Algorithms – 4 Credits (Book ID:
B0891)
Assignment Set – 2 (40 Marks)
 13

Answer the following : 6*10 = 6

1. Briefly explain chained matrix multiplication

Matrix chain multiplication is an optimization problem that can be solved using dynamic programming.
Given a sequence of matrices, we want to find the most efficient way to multiply these matrices
together. The problem is not actually to perform the multiplications, but merely to decide in which order
to perform the multiplications.

We have many options because matrix multiplication is associative. In other words, no matter how we
parenthesize the product, the result will be the same. For example, if we had four matrices A, B, C, and
D, we would have:

(ABC)D = (AB)(CD) = A(BCD) = A(BC)D = ....

However, the order in which we parenthesize the product affects the number of simple arithmetic
operations needed to compute the product, or the efficiency. For example, suppose A is a 10 × 30 matrix,
B is a 30 × 5 matrix, and C is a 5 × 60 matrix. Then,

(AB)C = (10×30×5) + (10×5×60) = 1500 + 3000 = 4500 operations

A(BC) = (30×5×60) + (10×30×60) = 9000 + 18000 = 27000 operations.

Clearly the first method is the more efficient. Now that we have identified the problem, how do we
determine the optimal parenthesization of a product of n matrices? We could go through each possible
parenthesization (brute force), but this would require time O(2n), which is very slow and impractical for
large n. The solution, as we will see, is to break up the problem into a set of related subproblems. By
solving subproblems one time and reusing these solutions many times, we can drastically reduce the time
required. This is known as dynamic programming.

To begin, let's assume that all we really want to know is the minimum cost, or minimum number of
arithmetic operations, needed to multiply out the matrices. If we're only multiplying two matrices, there's
only one way to multiply them, so the minimum cost is the cost of doing this. In general, we can find the
minimum cost using the following recursive algorithm:

• Take the sequence of matrices and separate it into two subsequences.

• Find the minimum cost of multiplying out each subsequence.
• Add these costs together, and add in the cost of multiplying the two result matrices.
• Do this for each possible position at which the sequence of matrices can be split, and take the
minimum over all of them.

For example, if we have four matrices ABCD, we compute the cost required to find each of (A)(BCD), (AB)
(CD), and (ABC)(D), making recursive calls to find the minimum cost to compute ABC, AB, CD, and BCD.
We then choose the best one. Better still, this yields not only the minimum cost, but also demonstrates
the best way of doing the multiplication: just group it the way that yields the lowest total cost, and do
the same for each factor.

Unfortunately, if we implement this algorithm we discover that it's just as slow as the naive way of trying
all permutations! What went wrong? The answer is that we're doing a lot of redundant work. For example,
above we made a recursive call to find the best cost for computing both ABC and AB. But finding the best
cost for computing ABC also requires finding the best cost for AB. As the recursion grows deeper, more
and more of this type of unnecessary repetition occurs.

One simple solution is called memorization: each time we compute the minimum cost needed to multiply
out a specific subsequence, we save it. If we are ever asked to compute it again, we simply give the saved
answer, and do not recompute it. Since there are about n2/2 different subsequences, where n is the
 14

number of matrices, the space required to do this is reasonable. It can be shown that this simple trick
brings the runtime down to O(n3) from O(2n), which is more than efficient enough for real applications.
This is top-down dynamic programming.

Pseudocode (without memoization):

// Matrix Ai has dimension p[i-1] x p[i] for i = 1..n

Matrix-Chain-Order(int p[])
{
// length[p] = n + 1
n = p.length - 1;
// m[i,j] = Minimum number of scalar multiplications (i.e., cost) needed to compute the matrix A[i]A[i+1]...A[j] = A[i..j]
// cost is zero when multiplying one matrix
for (i = 1; i <= n; i++)
m[i,i] = 0;

for (L=2; L<=n; L++) { // L is chain length

for (i=1; i<=n-L+1; i++) {
j = i+L-1;
m[i,j] = MAXINT;
for (k=i; k<=j-1; k++) {
// q = cost/scalar multiplications
q = m[i,k] + m[k+1,j] + p[i-1]*p[k]*p[j];
if (q < m[i,j]) {
m[i,j] = q;
// s[i,j] = Second auxiliary table that stores k
// k = Index that achived optimal cost
s[i,j] = k;
}
}
}
}
}

• Note : The first index for p is 0 and the first index for m and s is 1

Another solution is to anticipate which costs we will need and precompute them. It works like this:

• For each k from 2 to n, the number of matrices:

o Compute the minimum costs of each subsequence of length k, using the costs already
computed.

When we're done, we have the minimum cost for the full sequence. Although it also requires O(n3) time,
this approach has the practical advantages that it requires no recursion, no testing if a value has already
been computed, and we can save space by throwing away some of the subresults that are no longer
needed. This is bottom-up dynamic programming: a second way by which this problem can be solved.

Generalizations

Although this algorithm applies well to the problem of matrix chain multiplication, researchers such as
Gerald Baumgartner have noted that it generalizes well to solving a more abstract problem: given a linear
sequence of objects, an associative binary operation on those objects, and a way to compute the cost of
performing that operation on any two given objects (as well as all partial results), compute the minimum
cost way to group the objects to apply the operation over the sequence.

One common special case of this is string concatenation. Say we have a list of strings. In C, for example,
the cost of concatenating two strings of length m and n using strcat is O(m + n), since we need O(m) time
to find the end of the first string and O(n) time to copy the second string onto the end of it. Using this
cost function, we can write a dynamic programming algorithm to find the fastest way to concatenate a
sequence of strings (although this is rather useless, since we can concatenate them all in time
proportional to the sum of their lengths). A similar problem exists for singly linked lists.
 15

Another generalization is to solve the problem when many parallel processors are available. In this case,
instead of adding the costs of computing each subsequence, we just take the maximum, because we can
do them both simultaneously. This can drastically affect both the minimum cost and the final optimal
grouping; more "balanced" groupings that keep all the processors busy are favored. Heejo Lee et al.
describe even more sophisticated approaches.

2. Show is any regular expression.  

3. Explain the concept of transition diagrams with examples wherever necessary.

State transition diagrams have been used right from the beginning in object-oriented modeling. The basic
idea is to define a machine that has a number of states (hence the term finite state machine). The
machine receives events from the outside world, and each event can cause the machine to transition from
one state to another. For an example, take a look at figure 1. Here the machine is a bottle in a bottling
plant. It begins in the empty state. In that state it can receive squirt events. If the squirt event causes
the bottle to become full, then it transitions to the full state, otherwise it stays in the empty state
(indicated by the transition back to its own state). When in the full state the cap event will cause it to
transition to the sealed state. The diagram indicates that a full bottle does not receive squirt events, and
that an empty bottle does not receive cap events. Thus you can get a good sense of what events should
occur, and what effect they can have on the object.

State transition diagrams were around long before object modeling. They give an explicit, even a formal
definition of behavior. A big disadvantage for them is that they mean that you have to define all the
possible states of a system. Whilst this is all right for small systems, it soon breaks down in larger systems
as there is an exponential growth in the number of states. This state explosion problem leads to state
transition diagrams becoming far too complex for much practical use. To combat this state explosion
problem, object-oriented methods define separate state-transition diagrams for each class. This pretty
much eliminates the explosion problem since each class is simple enough to have a comprehensible state
transition diagram. (It does, however, raise a problem in that it is difficult to visualize the behavior of the
whole system from a number of diagrams of individual classes - which leads people to interaction and
activity modeling).

The most popular variety of state-transition diagram in object methods is the Harel Statechart as in
Figure 1. This was introduced by Rumbaugh, taken up by Booch and adopted in the UML. It is one of the
more powerful and flexible forms of state transition diagram. A particularly valuable feature of the
approach is its ability to generalize states, which allows you to factor out common transitions (thus I can
show that the break event applies to both full and empty states by creating the super-state of in-
progress). It also has a flexible approach to handling processing. Processes that are instantaneous (i.e.
cannot be interrupted) can be bound to the transitions or to the entry or exit of a state, these are called
actions. Processes that are long (and can be interrupted) are bound to states, these are called activities.
Transitions can also have a condition attached to them, which means that the transition only occurs if the
condition is true. There is also a capability for concurrent state diagrams, allowing objects to have more
than one diagram to describe their behavior.
 16

Figure 1: A Harel statechart

Not all methods use Harel Statecharts. One of the most notable dissenters is Shlaer/Mellor who use a
simpler Moore model state diagram. This form only allows processes to occur when in a state (hence the
extra state in Figure 8) and has no superstates. This is a good example of the question of expressiveness
in techniques. The Harel Statechart is clearly a more expressive technique, but since it is more expressive
there is more to learn when using it. In addition, it is more difficult to implement. Shlaer/Mellor take the
view that since state diagrams do not get too complex when drawn for a single object, the extra
expressiveness is not worthwhile.

Figure 2: Moore model state diagram as used by Shlaer/Mellor. It is the equivalent to figure 1.
 17

When to Use Them

State models are ideal for describing the behavior of a single object. They are also formal, so tools can be
built which can execute them. Their biggest limitation is that they are not good at describing behavior
that involved several objects, for these cases use an interaction diagram or an activity diagram.

People often do not find drawing state diagrams for several objects to be a natural way of describing a
process. In these cases you can try either drawing a single state diagram for the process, or using an
activity diagram. This defines the basic behavior, which you then need to refactor to split it across a
number of objects.

Where to Find Out More

UML uses Harel statecharts, which have also become the most popular style of state diagrams in object
methods. For an initial tutorial on them I would suggest either [Booch] or [Rumbaugh]. For a more in-
depth treatment take a look at [Cook and Daniels]; they give a lot of good details on formalisms, an
integration of design by contract, and a discussion of the use of state diagrams with subclassing.

4. If L1 and L2 are context- free languages, then L1 U L2 is a context free language.

5. Explain prim’s Algorithm.

This algorithm is directly based on the MST property. Assume that V = {1, 2,..., n}.
REF.
R.C. Prim. Shortest connection networks and some generalizations. Bell System Technical
Journal, Volume 36, pp. 1389-1401, 1957.

T= ;

U = { 1 };

while (U V)

let (u, v) be the lowest cost edge

such that u U and v V - U;

T=T {(u, v)}

U=U {v}

}
 18

• See Figure 8.11 for an example.

• O(n2) algorithm.

Proof of Correctness of Prim's Algorithm

Theorem: Prim's algorithm finds a minimum spanning tree.

Proof: Let G = (V,E) be a weighted, connected graph. Let T be the edge set that is grown in Prim's
algorithm. The proof is by mathematical induction on the number of edges in T and using the MST Lemma.

Basis: The empty set is promising since a connected, weighted graph always has at least one MST.

Induction Step: Assume that T is promising just before the algorithm adds a new edge e = (u,v). Let U be
the set of nodes grown in Prim's algorithm. Then all three conditions in the MST Lemma are satisfied and
therefore T U e is also promising.

When the algorithm stops, U includes all vertices of the graph and hence T is a spanning tree. Since T is
also promising, it will be a MST.

Implementation of Prim's Algorithm

Use two arrays, closest and lowcost.

• For i V - U, closest[i] gives the vertex in U that is closest to i

• For i V - U, lowcost[i] gives the cost of the edge (i, closest(i))

Figure 8.11: Illustration of Prim's algorithm

 19

Figure 8.12: An example graph for illustrating Prim's

algorithm

• At each step, we can scan lowcost to find the vertex in V - U that is closest to U. Then we update
lowcost and closest taking into account the new addition to U.
• Complexity: O(n2)

Example: Consider the digraph shown in Figure 8.12.

U= V - U = {2, 3,
{1} 4, 5, 6}
 20

closest lowcost
V-
U
U
2 1 6
3 1 1
4 1 5
5 1
6 1

Select vertex 3 to include in U

U = {1, V - U = {2, 4, 5,
3} 6}
closest lowcost

V-U U
2 3 5
4 1 5
5 3 6
6 3 4
Now select vertex 6

U = {1, 3, V - U = {2, 4,
6} 5, 6}
closest lowcost

V-U U
2 3 5
4 6 2
5 3 6
Now select vertex 4, and
so on

6. Give an algorithm for Greedy Knapsack problem. Analyse your algorithm?

 21

The knapsack problem or rucksack problem is a problem in combinatorial optimization: Given a set of
items, each with a weight and a value, determine the number of each item to include in a collection so
that the total weight is less than or equal to a given limit and the total value is as large as possible. It
derives its name from the problem faced by someone who is constrained by a fixed-size knapsack and
must fill it with the most useful items.

The problem often arises in resource allocation with financial constraints. A similar problem also appears
in combinatory, complexity theory, cryptography and applied mathematics.

The decision problem form of the knapsack problem is the question "can a value of at least V be achieved
without exceeding the weight W?"

Definition

In the following, we have n kinds of items, 1 through n. Each kind of item i has a value vi and a weight wi.
We usually assume that all values and weights are nonnegative. To simplify the representation, we can
also assume that the items are listed in increasing order of weight. The maximum weight that we can
carry in the bag is W.

The most common formulation of the problem is the 0-1 knapsack problem, which restricts the number
xi of copies of each kind of item to zero or one. Mathematically the 0-1-knapsack problem can be
formulated as:

• maximize

• subject to

The bounded knapsack problem restricts the number xi of copies of each kind of item to a maximum
integer value ci. Mathematically the bounded knapsack problem can be formulated as:

• maximize

• subject to

The unbounded knapsack problem (UKP) places no upper bound on the number of copies of each kind of
item.

Of particular interest is the special case of the problem with these properties:

• it is a decision problem,
• it is a 0-1 problem,
• for each kind of item, the weight equals the value: wi = vi.

Notice that in this special case, the problem is equivalent to this: given a set of nonnegative integers,
does any subset of it add up to exactly W? Or, if negative weights are allowed and W is chosen to be zero,
 22

the problem is: given a set of integers, does any nonempty subset add up to exactly 0? This special case is
called the subset sum problem. In the field of cryptography the term knapsack problem is often used to
refer specifically to the subset sum problem.

If multiple knapsacks are allowed, the problem is better thought of as the bin packing problem.

Computational complexity

The knapsack problem is interesting from the perspective of computer science because

• there is a pseudo-polynomial time algorithm using dynamic programming

• there is a fully polynomial-time approximation scheme, which uses the pseudo-polynomial time
algorithm as a subroutine
• the problem is NP-complete to solve exactly, thus it is expected that no algorithm can be both
correct and fast (polynomial-time) on all cases
• many cases that arise in practice, and "random instances" from some distributions, can
nonetheless be solved exactly.

The subset sum version of the knapsack problem is commonly known as one of Karp's 21 NP-complete
problems.

There have been attempts to use subset sum as the basis for public key cryptography systems, such as the
Merkle-Hellman knapsack cryptosystem. These attempts typically used some group other than the
integers. Merkle-Hellman and several similar algorithms were later broken, because the particular subset
sum problems they produced were in fact solvable by polynomial-time algorithms.

One theme in research literature is to identify what the "hard" instances of the knapsack problem look
like, or viewed another way, to identify what properties of instances in practice might make them more
amenable than their worst-case NP-complete behaviour suggests.

Several algorithms are freely available to solve knapsack problems, based on dynamic programming
approach, branch and bound approach or hybridizations of both approaches.

Dynamic programming solution

Unbounded knapsack problem

If all weights ( ) are nonnegative integers, the knapsack problem can be solved in
pseudo-polynomial time using dynamic programming. The following describes a dynamic programming
solution for the unbounded knapsack problem.

To simplify things, assume all weights are strictly positive (wi > 0). We wish to maximize total value
subject to the constraint that total weight is less than or equal to W. Then for each w ≤ W, define m[w]
to be the maximum value that can be attained with total weight less than or equal to w. m[W] then is the
solution to the problem.

Observe that m[w] has the following properties:

• (the sum of zero items, i.e., the summation of the empty set)

where vi is the value of the i-th kind of item.

 23

Here the maximum of the empty set is taken to be zero. Tabulating the results from m[0] up through
m[W] gives the solution. Since the calculation of each m[w] involves examining n items, and there are W
values of m[w] to calculate, the running time of the dynamic programming solution is O(nW). Dividing
by their greatest common divisor is an obvious way to improve the running
time.

The O(nW) complexity does not contradict the fact that the knapsack problem is NP-complete, since W,
unlike n, is not polynomial in the length of the input to the problem. The length of the W input to the
problem is proportional to the number of bits in W, logW, not to W itself.

0-1 knapsack problem

A similar dynamic programming solution for the 0-1 knapsack problem also runs in pseudo-polynomial
time. As above, assume are strictly positive integers. Define m[i,w] to be
the maximum value that can be attained with weight less than or equal to w using items up to i.

We can define m[i,w] recursively as follows:

•
•
• if (the new item is more than the current weight
limit)
• if .

The solution can then be found by calculating m[n,W]. To do this efficiently we can use a table to store
previous computations. This solution will therefore run in O(nW) time and O(nW) space. Additionally, if
we use only a 1-dimensional array m[w] to store the current optimal values and pass over this array i + 1
times, rewriting from m[W] to m[1] every time, we get the same result for only O(W) space.

Greedy approximation algorithm

George Danzig proposed a greedy approximation algorithm to solve the unbounded knapsack problem. His
version sorts the items in decreasing order of value per unit of weight, vi / wi. It then proceeds to insert
them into the sack, starting with as many copies as possible of the first kind of item until there is no
longer space in the sack for more. Provided that there is an unlimited supply of each kind of item, if m is
the maximum value of items that fit into the sack, then the greedy algorithm is guaranteed to achieve at
least a value of m / 2. However, for the bounded problem, where the supply of each kind of item is
limited, the algorithm may be far from optimal.