YS ND

P-06

IS
AL A S
AN R E
M TU
TH C
RI RU
GO ST
AL A
AT
D
SEQUENTIAL,PARALLEL
AND DISTRIBUTED
ALGORITHMS

PRESENTED BY: JHON PAUL GERVACIO

II-BSCPE
1
What is an Algorithm?
- An algorithm is a sequence of steps that take inputs fr
om the user and after some computation, produces an ou
tput.
- An algorithm is a sequence of instructions followed to
solve a problem. While designing an algorithm, we sho
uld consider the architecture of computer on which the a
lgorithm will be executed.

As per the architecture, there are two types of

computers:
Sequential Computer and Parallel
Computer
 Depending on the architecture of computers, we have two types
of algorithms:
SEQUENTIAL ALGORITHM AND PARALLEL
ALGORITHM
Sequential Algorithm 
-An algorithm in which some consecutive steps of
instructions are executed in a chronological order to
solve a problem.
- In computer science, a sequential algorithm or serial
algorithm is an algorithm that is executed sequentially
– once through, from start to finish, without other
processing executing – as opposed to concurrently or
in parallel.
"Sequential algorithm" may also refer specifically to
an algorithm for decoding a convolutional code.
Convolutional code is a type of error-correcting
code that generates parity symbols via the sliding
application of a boolean polynomial function to a data
stream. The sliding application represents the
'convolution' of the encoder over the data, which gives
rise to the term 'convolutional coding'.
Parallel Algorithm − the problem is divided into sub-
problems and are executed in parallel to get individual
outputs. Later on, these individual outputs are combined
together to get the final desired output.
-In computer science, a parallel algorithm, as opposed
to a traditional serial algorithm, is an algorithm which
can be executed a piece at a time on many different
processing devices, and then combined together again at
the end to get the correct result.

Note! It is not easy to divide a large problem

into sub-problems. Sub-problems may have data
dependency among them. Therefore, the processors
have to communicate with each other to solve the
problem.
- It has been found that the time needed by the
processors in communicating with each other is more
than the actual processing time. So, while designing a
parallel algorithm, proper CPU utilization should be
considered to get an efficient algorithm.

To design an algorithm properly, we must have a

clear idea of the basic model of
computation in a parallel computer.
Model of Computation
Both sequential and parallel computers operate
on a set (stream) of instructions called algorithms.
These set of instructions (algorithm) instruct the
computer about what it has to do in each step.
Depending on the instruction stream and data
stream, computers can be classified into four
categories −
 Four categories depending on the instruction
stream and data stream, computers:
• Single Instruction stream, Single Data stream
(SISD) computers
• Single Instruction stream, Multiple Data stream
(SIMD) computers
• Multiple Instruction stream, Single Data stream
(MISD) computers
• Multiple Instruction stream, Multiple Data stream
(MIMD) computers
SISD Computers (Single Instruction stream, Single
Data stream)
SISD computers contain one control unit, one processing
unit, and one memory unit.

In this type of computers, the processor receives a single

stream of instructions from the control unit and
operates on a single stream of data from the memory
unit.
SIMD Computers (Single Instruction stream,
Multiple Data stream)
SIMD computers contain one control unit, multiple
processing units, and shared memory or
interconnection network.
Here, one single control unit sends instructions to all
processing units. During computation, at each step, all
the processors receive a single set of instructions from
the control unit and operate on different set of data from
the memory unit.

MISD Computers (Multiple Instruction

stream, Single Data stream)
As the name suggests, MISD computers
contain multiple control units, multiple
processing units,and one common memory
unit.
MISD Computers

Here, each processor has its own control unit and they
share a common memory unit. All the processors get
instructions individually from their own control unit
and they operate on a single stream of data as per the
instructions they have received from their respective
control units. This processor operates simultaneously.
12
MIMD Computers (Multiple Instruction stream,
Multiple Data stream)
MIMD computers have multiple control units,
multiple processing units, and a shared memory or
interconnection network.

Here, each processor has its own control unit, local

memory unit, and arithmetic and logic unit.
They receive different sets of instructions from their
respective control units and operate on different sets
of data.

Note! Most standard computer algorithms are sequential

algorithms, and not specifically identified as such, as
sequentialness is a background assumption.
OTHER ALGORITHM:

Distributed Algorithm
A distributed algorithm is an algorithm designed to
run on computer hardware constructed from
interconnected processors. Distributed algorithms are
used in many varied application areas of distributed
computing, such as telecommunications,scientific
computing, distributed information processing, and
real-time process control. Standard problems solved
by distributed algorithms include leader
election, consensus, distributed search, spanning
tree generation,mutual exclusion, and resource
allocation.
The choice of an appropriate distributed algorithm to
solve a given problem depends on both the
characteristics of the problem, and characteristics of
the system the algorithm will run on such as the type
and probability of processor or link failures, the kind of
inter-process communication that can be performed, and
the level of timing synchronization between separate
processes.
When to know if algorithm is parallel?
An algorithm is parallel if there are several
processes (tasks, threads, processors) working
on it at the same time. Often the tasks run in the
same address space, and can
communicate/reference results by others freely
(low cost).
When to know if algorithm is distributed?

An algorithm is distributed if it is parallel and

the tasks run on separate machines (separate
address spaces), one task has no direct access
to the work of the others. It has to request
needed data, or just wait until it is sent to it.