Multiprocessor Systems | 1

UNIT 7: Multiprocessor Systems

Structure
7.0 Objective
7.1 Introduction
7.2 Motivation and Classification
7.3 Multi-Processor Interconnection
7.4 Types of Multi-Processor Operating System
7.5 Multi-Processor OS Functions and Requirements
7.6 Introduction of Parallel Computing (Distributed Operating System)
7.7 Introduction to Multi-Processor Synchronization
7.8 Summary
7.9 Exercises
Suggested Reading

7.0 Objective

After reading this tutorial, you should be able to understand.

Requirement and classification of multiprocessor system
Multiprocessor interconnection network
Different type of multiprocessor operating system (OS)
Function of multiprocessor operating system
Distributed operating system
Multiprocessor synchronization

7.1 Introduction

A multi-processor system is an interconnection of two or more CPUs within a single computer

system and share common RAM and Input/output devices.Multiprocessor systemsare categorized as
MIMD (multiple instruction stream-multiple data stream) system. After learning this chapter,in
section 7.2, you will understand the requirement and classification of multiprocessor operating
system. In section 7.3, you will be able to know various types of interconnection network in
2 | Operating System

multiprocessor operating system environment.Section 7.4,introduces three types of multiprocessor

operating systems. Section 7.5, helps you to learn about the function and requirement of
multiprocessing operating system. In section 7.6, you will acquire basics of parallel computing. In the
end,in section 7.7,you will acquire the knowledge about multiprocessor synchronization.

7.2 Motivation and Classification

Multiprocessor systems are required to process huge amount of data with very high speed. These
types of systems are normally used in the environment like weather forecasting, satellite control, etc.
Motivation: Importantfactors to design multiprocessor systems are:
Increasing computation power
Sharing resources
Increasing reliability
Increasing computation power:
Computations are disintegrated into consecutive components which will be executed
simultaneously.
This decomposition task is performed by parallel algorithms.
Parallel programming concept is used to program such algorithms.
A few programming languages have been given with appropriate constructs for expressing
parallel computations.
To perform parallel computation, sequential components of a program are scheduled by
operating systemon the available processors.
Sharing resources:
The end user of amultiprocessing system is able to use available resources of another
multiprocessing system.
In a multiprocessor system,information is sharedasmost commonresources.
Memories, processors, and devices are also shared, and the power of computingisdistributed
among the multiprocessing systems.
Increasing reliability: A reliable (error tolerant) system is able to:
Notice the breakdown of the system components.
Separate the failure component.
Reallocate the computing tasks in the system, that is, rearrange the system and reschedule its
activity.
Notify the user to know about the new configuration by postingappropriate messages to the
terminal of the user.
Classification: Multi-processor are classified by their organized memory, and classified into two
types
Tightly coupled or shared memory multiprocessor:In a tightly coupled system, two or
more processing units (more than one processor) sharescommon memory space and I/O
 Multiprocessor Systems | 3

devices, and are interconnected by a bus,in order to process the programs concurrently.Entire
computer system is managed by single operating system.These systems can perform better
and physically smaller than loosely-coupled system.This type of system is more expensive.In
this system, a short delay has occurred, if a message is sent from one computer to another
with high data rate; which means, the transferring of number of bits per second is large.For
instance, on the same printed circuit board the two CPU chips are connected by wire and
etched onto the board are likely to be tightly coupled.Tightly-coupledsystems ismuch more
energy efficient than clusters.Becauseof the considerable economies is realized through
designing the components to work altogether.The tightly coupled multiprocessor system is
shown in figure 7.1 below.

Figure 7.1: Tightly coupled multiprocessor system

Loosely coupled or distributed memory multiprocessor: Loosely coupled multiprocessor
contains distributed memory where each of the processors has their own memory and IO
channels. By interconnection switching or message passing,the processors can communicate
with each other. Each processor may have a different operating system and individual bus
control logic. This type of system is connected by high-speed communication networks.This
type of system is opposite of tightly coupled system.This type of system is less expensive.In
this system, the message delay is large with low data rate.For instance, two computers are
connected by a2200 bit/sec modem over the telephone system are loosely coupled
system.Here the processorsare interacting with each other through the use of interconnection
structure or by message passing using the send and receive functions.To access the remote
memory which is attached with the other processors takes more time for extra delay through
4 | Operating System

the interconnection network.Here the memory access time is non-uniform.The loosely

coupled multiprocessor system is shown in figure 7.2 below.

Figure 7.2:Loosely coupled multiprocessor system

7.3 Multi-Processor Interconnection

In multi-processing system, each processor can access any of the I/O units and memory modules. The
connectivity between these processors is achieved through interconnection networks to exchange data
between them. Therefore, interconnection networks determine the overall performance of
multiprocessor systems.
The physical forms to establish an interconnection network are
Time shared common bus.
Multiport memory
Crossbar switch
Multistage switching network.
Hypercube system.
Time shared common bus: A common bus is shared by all processors and memory of a system.In
this arrangement,all processors are attachedwitha common path to the memory unit. The local
memory or cache memory isattached with each of the CPU.
 Multiprocessor Systems | 5

Figure 7.3: Organization of time shared common bus

Disadvantages:
Only single processor is communicated with the memory and other processor at any given
time.
The average transfer rate is limited within the system by the speed of the single path.
Multiport memory: In this system, dissimilar memory modules and CPUs have distinct buses. The
module has internal control logic for determiningthe port which accesses memory at any specified
time. To resolve conflicts of memory access,priorities are assigned to each of the memory port.

Figure 7.4: Organization of multiport memory

6 | Operating System

Advantages:
Due to multiple paths, high transfer rate is achieved.
Disadvantages:
It needs expensive memory control logic and a big number of cables and connections.
Crossbar switch: It consists of a several cross-points and is assigned at intersections between
memory module paths and processor buses. A switch can determine a path from a processor to a
memory module.Figure 7.5 below shows the crossbar switch.

Figure 7.5: Crossbar Switch

Advantages:
It supports concurrent transfers from all memory modules.
Disadvantages:
To implement the switch, hardware is very complex and large.
Multistage switching network:
The basic components are a two-output, two-input interchange switch:
Basic component of multistage switching connection has been shown in Figure 7.6 below.
 Multiprocessor Systems | 7

Figure 7.6: Basic component of multistage switching connection.

This type of networks are designed with the small number of elementary crossbar switches and
connected in multiple layers. The switches are implementedin 4 types of connections: crossed
straight, upper and lower broadcast. All the elementary switches canbe controlled concurrently. The
extension cost for such a network is relatively low.
For this type of networks, there is a restriction to implement arbitrary connections in the switch if
some connections are already set. For this reason, these networks are called blocking networks. A
multistage connection network for parallel systems has been shown below in Figure 7.7 below.

Figure 7.7: A multistage connection network for parallel systems

8 | Operating System

Hypercube system: This type of network is also knownas a binary n-cube network with 2n nodes and
arranged as an n-dimensional cube. Suppose a single point is labeled as 0 and called a 0-cube. Now,
replicating this 0-cube and placing this copy at a distance of one unit away. Connect these two nodes
and labeled the replicated node as 1. As shown in figure 7.8 below. This is called 1-cube. Further by
extending this idea you will get 2- cube, 3-cube as shown in figure 7.8 below.

Figure 7.8: 0, 1, 2, and 3 dimensions of hypercube.

The labels of two nodes are different by exactly one bit change when they are connected by an
edge.For an n-dimensional hypercube, label of each node is represented by n bits.Each of these bits
isinverted (1->0 or 0->1 ), which indicates each node contains exactly n incident edges. n is the
diameter of an n-dimensional hypercube. This corresponds to a walk across n edges in a hypercube
from the first to the second label. The bisection width of an n -dimensional hypercube is 2n 1. The
number of edges in an n-dimensional hypercube is n 2n 1. A 4-dimension hypercube system is
shown in Figure 7.9 below.

Figure 7.9: Hypercube of 4-dimension.

 Multiprocessor Systems | 9

Advantages:
The bisection width can be very high and the diameter is low. It makes the hypercube an
attractive organization.
Drawbacks:
The number of edges per node is a function (logarithmic) of network size and making it
difficult to scale up, and the maximum length of edge increases when the network size
increases.

7.4 Types of Multi-Processor Operating System

In multi-processing system various types of organizations are possible. Below you will see three of
them.
Each CPU has its own operating system:
This type of multiprocessing operating system is organized by partitioning memory as the number of
CPUs is connected with the system and each partition of memory is allocated to each CPU as a
private memory. Each CPU also has its own copy of a private OS. Hence, n CPU means n separate
computer. For optimization purpose, each CPU shares operating system code and private memory to
store its own operating systems private data, as shown in Figure 7.10 below.

Figure 7.10:Partitioning multiprocessor memory for 4 CPUs

Here, the system call are handle by Each CPUs own operating system table. Each CPU has its own
set of processes to run. Sharing of processes could not be possible among CPUs and sharing of page
is not allowed.
Master-slave multiprocessors:
This model is shown in Figure 7.11 below, where CPU 1 contains a single copy of OS within a multi-
processor system and all system calls are processed by CPU 1. Here CPU 1 is master and all other
CPUs are slaves.
10 | Operating System

Figure 7.11: Master-slave multiprocessor.

Only CPU 1 has a single data structure to schedule the processes. Pages are shared among all the
processes at run time. This model will notwork in the environment of large number of processors.
Symmetric multiprocessors:
In this model, single copy of OS is stored in the memory and any of the CPU can use it to run its OS.
When a system call is placed by any of the CPU to the kernel then it will process by the kernel. This
model is shown in figure 7.12 below.

Figure 7.12: Symmetric Multi-processor model

This model manages processes and memory dynamically, since there has a single set of operating
system tables.When twoor more CPUs are running operating system code at the same time, problem
will occur. For example, two processors are accessing simultaneously one process to run. To solve
this problem, a mutex (Lock) is used in OS to assign critical region to a CPU. At that time none of
the CPU is allowed to enter into that critical region.

7.5 Multi-Processor OS Functions and Requirements

Functions: Multiprocessing system is also called symmetric multiprocessing model, where each of
the processor executes an indistinguishable copy of OS and they are communicated with each other.
 Multiprocessor Systems | 11

Here, a processor is given a certain task. The system is controlled by a master processor. This type
ofarrangement is called a master-slave relationship. It will save money in compare to single processor
systems, since the processors share the power supplies,peripherals, and additional devices. The key
advantage of this type of system is to do more workwithina less time period. Furthermore, these
systems provide more reliability in the circumstances where one processor fails and that could not
stop the system to work further; just the system will slow down for some time.
Requirements: To work multiprocessing OS efficiently, the computer system has required the
following:
Motherboard support: A motherboard has the ability to handlemany processors. That is
theextra slotsor sockets for the additional chips are able to handle the multiprocessing
environment.
Processor support: All of the tasks of multiprocessing system arehandled by the OS. The
operating systemassignsthe different tasks to the several processors within the system. The
Applicationsin multiprocessing system arecalled threaded, because they are divided intosub
routines to run independently. Which means the OSruns these threads on many
processorsconcurrently and improving the performance.Hence the processes can execute in
parallel. Usually the parallel processing is needed in the area like expert systemand artificial
intelligence,weather forecasting, image processing etc.
In this system, resources are shared dynamically to the several processors may cause a
potential bottleneck. There are three key sources of contention that is founded in this type of
operating system:
o Locking system: for providingthe safe access of resources which are shared tomany
processors and this requireda locking scheme to protect them. The main purpose of
the locking system is to serialize accesses of protected resources by many processors.
Unmanageablecustom of locking can severely reduce the systemperformance. This is
managed by the use of locking scheme, replacing locks with lock-free algorithms,
escaping long critical sections, ignoring sharing altogether, or whenever possible.
o Shared data:
Several processors are continuously accessing the shared data and many of them with
data write also. They are serialized by the protocol of cache coherence. Delay in
serialization can affect the system performance. Furthermore, the cache coherence
traffics bursts saturate the interconnection network or the bus of memory and slow
down the whole system.This type of problem is removed through avoiding the
sharing of data or decrease the write access rate of shared data by replication
mechanism.
o False sharing:
False sharingoccurs when distinct data items used by several processors are placed
next to each other within the memory.Updating simultaneously the individual
elements in the same cache line that has come from distinct processors will
12 | Operating System

invalidatethe whole cache lines. However, these updates are independent of each
other. Every update of element of a cache line will invalidate the line. And this will
be seen by the other processors. They have to fetch a recent copy of the line from
memory, although theaccessed element has not been modified. As we knowthe cache
coherency is maintained on thebasisof cache-line rather than for individual elements.
This will cause an increase in interconnect traffic and hence overhead. Also, when
the cache-line is updating, access to the elements in the line is prohibited.
Apart from this, this type of system developer mustdelivermaintenance for
effectivelyexecutingapplication of user on the multiprocessor. Some of the support
includestechniquesofmigration across processors, task placement, placement of
physical memory where most of the pages of memory are used by the application is
placedwithin local memory.

7.6 Introduction of Parallel Computing (Distributed Operating System)

In a distributed system,a number of processors do not share a clock or memory. Instead, each
processor contains an own local memory. The processors can communicate with each other overthe
several communication networks, for exampletelephone lines or high-speed buses.
Motivation:The processors of distributed system may differ in function and size. They include
workstations, small microprocessors, and minicomputers.These processors have different names, for
instancenodes,sites, machines,computers, and hosts.The structure of a distributed system is shown in
Figure 7.13 below.

Figure 7.13: A distributed system.

 Multiprocessor Systems | 13

There are four main reasons to design distributed systems:

resource sharing
computation speedup
reliability
communication
Resource sharing: When several sites with different capabilities are linked together, then one user at
a given site can access the available resources at another site. Hereresourcesharing means a
mechanism to sharefiles at remote sites, print files at remote sites, process information in a
distributed database, access remote specialized hardware devices (for example, ahigh-speed array
processor), and performvarious operations.
Computation speedup: Here, a specific computation isdivided into sub-computations and can run
simultaneously. Hence, this type of system can allow us for distributingthe sub-computations to the
many sites and sub-computations can be run simultaneously. Thus offera computation speedup.
Reliability: Failure of one systemin a distributed environment does not affect another system to
continue its operation. Hence, provide a better reliability to the system. The system must detect the
failure of a site and an appropriate action may be taken to recover from that failure. The system
should stop the use of the service of that site. However, if a function of the failed site is accessed by
another site, the system must handle the proper transfer of the function to that site. Lastly, when a
failure site is recovered, it must be brought into the system to work again.
Communication: If different sites are linked together through a communication network, users can
exchange information from various sites. Generally, a massage passing environment is used in all the
higher-level functionality, for example login, file-transfer, remote procedure calls, mail found in a
system is expanded to includein the distributed system.

7.7 Introduction to Multi-Processor Synchronization

In the synchronization primitives, a process on a uniprocessor environment produces a system call to

access critical kernel table. The kernel code disables interrupts and then accessing the table. Hence,
the process willaccess the table and knowing that no other processes will interact with the table. In a
multi-processor environment, only the CPU is disabled the interrupt will be affected and other CPU
can still access the critical table. Hence, to work mutual exclusion a mutex protocol must be
implemented.
A TSL (Test and Set Lock) instruction can read out a memory word and store it in a
register.Concurrently, it will write a 1 or other nonzero value into the memory word. Now, on a
multi-processorenvironment, from the Figure 7.14 is shown in below, the memory word 1000 is used
as a lock and initial value is 0. For step 1, CPU 1 is reading out the word and getting a 0. For step 2,
before CPU1 rewrites the value to 1, CPU 2 is reading the word andgetting a 0. For step 3, CPU 1 is
writing a 1 into thatmemory word. For step 4, CPU 2 is also writing a 1into the same word. Here, two
14 | Operating System

CPUs get a 0 value from the TSL instruction. Hence, both ofthem have access to the critical region
and failing the mutual exclusion property.

Figure 7.14: The TSL instruction fails when the bus is not locked
To avoid this difficulty, the TSL instruction must lock the bus, stoppingother CPUs from accessing it
and accessing both memories, after that the bus is unlocked.
Anothergood idea is giving each CPU to get the mutex as its own private lock variable, as shown in
Figure 7.15below. The variable must reside in adifferently unused cache block foravoiding conflicts.
This mechanism works as a CPU is failing to acquire thelock, and then it allocates a lock variable and
added itself at the last position of a list of CPUs where they are waitingfor that lock. If the present
lock holder leaves the critical region, it releases theprivate lock to the first CPU on that list. This
CPUthen accesses the critical region. After completing the task, it releases the lock to its successor,
and so on. It is effectiveand free from starvation.

Figure 7.15:Use of multiple locks to avoid cache thrashing

 Multiprocessor Systems | 15

7.8 Summary

A multiprocessor system is an interconnection of two or more CPUs within a single computer

system and share common RAM and Input/output devices.
Multiprocessor systems are required to process huge amount of data with very high speed.
Important factors to design multiprocessor systems are: increasing computation power,
sharing resources, increasing reliability.
Multi-processor are classified by their organized memory, and classified into two types:
loosely coupled and tightly coupled.
The physical forms to establish an interconnection network are time shared common bus,
multiport memory, crossbar switch, multistage switching network, hypercube system.
In multi-processing system various types of organizations are possible. For example, Each
CPU has its own operating system, master-slave multiprocessors, and symmetric
multiprocessors.
Multiprocessing system is also called symmetric multiprocessing model, where each of the
processor executes an indistinguishable copy of OS and they are communicated with each
other.
To work multiprocessing OS efficiently, the computer system has required the following:
processor Support and motherboard support.
In a distributed system, a number of processors do not share a clock or memory. Instead,
each processor contains an own local memory.
There are four main reasons to design distributed systems: resource sharing, computation
speedup, reliability, communication.
In a multi-processor environment, only the CPU is disabled the interrupt will be affected and
other CPU can still access the critical table. Hence, to work mutual exclusion a mutex
protocol must be implemented.

7.9 Exercises

1. Memory management on a multiprocessor must deal with all of found on

A. Uniprocessor computer
B. Computer
C. Processor
D. System
16 | Operating System

2. The feature of multi-microprocessor architecture is

A. Task dependent
B. Single bus provider for many processors
C. Design is for a specific task
D. All of the mentioned

3. A common synchronization mechanism used in multiprocessor operating system is

A. Complex
B. Locks
C. Lockstep
D. None

4. In distributed system each processor has its own

A. Local memory
B. Clock
C. Both A and B
D. None of the mentioned

5. Logical extension of computation migration is

A. Process migration
B. System migration
C. Thread migration
D. Data migration

6. Processes on the remote systems are identified by

A. Host ID
B. Host name and identifier
C. Identifier
D. Process ID

7. If one site fails in distributed system

A. The remaining sites can continue operating
B. All the sites will stop working
C. Directly connected sites will stop working
D. None of the mentioned

8. A multiprocessor operating system should perform

A. Mechanism to split a task into concurrent subtasks
B. Optimize the system performance
 Multiprocessor Systems | 17

C. Handling structural or architectural changes

D. All of the mentioned
9. Which routing technique is used in distributed system?
A. Fixed routing
B. Virtual routing
C. Dynamic routing
D. All of the mentioned

10. Network operating system runs on

A. Server
B. Every system in the network
C. Both A and B
D. None of the mentioned

11. What are the important factors to design multiprocessor systems? Explain each of the factors.
12. Write down the difference between the tightly coupled and loosely coupled multiprocessor
system.
13. What are the physical forms to establish an interconnection network of multiprocessor
system?
14. Explain Multistage switching network and Hypercube system interconnection network.
15. Describe three types of multiprocessor operating system.
16. Write down the function of multiprocessing operating system.
17. Write down the basic requirement of multiprocessing operating system to work efficiently.
18. Explain false sharing?
19. What is parallel computing? What are the main reasons to design distributed systems?
Briefly explain all of them.
20. What is multiprocessor synchronization?
21. How TSL (Test and Set Lock) instruction fails mutual exclusion property?

Suggested Reading
1. Silberschatz, Abraham, et al. Operating system concepts. Vol. 4. Reading: Addison-wesley,
1998.
2. Romero, Fernando. "Operating Systems. A concept-based approach." Journal of Computer
Science & Technology 9 (2009).