Beruflich Dokumente
Kultur Dokumente
Creative
Approach
'
-\"\\'v
mVi
INTRODUCTION
TO ALGORITHMS
A Creative Approach
INTRODUCTION
TO ALGORITHMS
A
Creative
Approach
UDIMANBER
University
of Arizona
\342\226\274\342\226\274
ADDISON-WESLEY
PUBLISHING
Menlo
Reading, Massachusetts \342\200\242
Don Mills, Ontario \342\200\242
Wokingham,
Bonn
\342\200\242
Sydney
\342\200\242
Singapore
COMPANY
Park, California
England
\342\200\242
Tokyo
\342\200\242
New
York
\342\200\242
Amsterdam
\342\200\242
Madrid
\342\200\242
San Juan
Cataloging-in-PublicationData
of Congress
Library
Udi.
Manber,
to algorithms.
Introduction
2. Algorithms.
I. Title.
'
005.7 3
1989
QA76.9.D35M36
88-2186
0-201-12037-2
ISBN
by Addison-Wesley
Reproduced
from camera-ready
copysuppliedby
the
author.
programs and applications presented in this book have been included for their
value.
but are not guaranteed for any
care,
They have been tested with
The publisher does not offer
or
warranties
purpose.
any
representation, nor does it accept
liabilities
with
to
or
the
any
respect
programs
applications.
The
instructional
Reprinted
with
Copyright
All
rights
system,
corrections
\302\251
1989
No part
transmitted,
photocopying, recording,
Printed
in the United
States
EFGHIJ-DO-943210
Publishing Company
by Addison-Wesley
reserved.
or
1989
October,
in
of
this
any
publication
form
or otherwise,
of America.
or
without
Inc.
Published
simultaneously
in Canada.
To my
PREFACE
book
grew
Like many
other
This
out of my
teachers,
not being
with
frustrations
that not
I discovered
is it
only
This
show
of
sequence
Instead,
'
the
'problem
but
rather
Our goals in
provides
depth.
use
intellectual
looks more
problems
when
an algorithm is
not
they arise
as important
how
only
in
the
as teaching
as finished
to solve
future.
program,
principles.
algorithm's
than
presentan
into a
to translate
the
drawbacks,
The goal is to
so on.
particular
Teaching
the details
improvements,
turns),
wrong
in
so on.
X, the
\"problem
these
overcoming
through a creativeprocess,rather
are to show
like
and
P\\\"
of the
The
products.
problems,
the
thinking
solution.
involved
in creating algorithms, an
process
in this book. This
is
used
designing algorithms
for
and it also
known
covers
algorithms,
many
techniques
designing
in more
an elegant intuitive
for explaining
framework
the design of algorithms
we do
It does not, however,
of designing
and
cover all possible ways
algorithms,
it exclusively.
between the
The heart
of the methodology
lies in an analogy
of proving mathematical theorems by induction
and that of designing
process
further
To
methodology
program
is to
purpose
described
are not
P, program
(including possible
algorithm
algorithms
in designing
\"old-new\"
A\\
explained
teaching
involved
algorithm
better
thus
are
but also
A, algorithm
Algorithms
and
relation to other methods and algorithms,\"
it easier for a programmer
not in a way
that makes
in a way
makes
that
it easier to understand
algorithms
not
at
of algorithm
algorithm.
usually
algorithm,
first attempts
algorithm
X,
sequence
straightforward
analysis,
side
reader
the
help
methodology
combinatorial
the
thinking
for
algorithms.
Although
Chapter 5.
these
are
more
vi
Preface
car, and
Consider
to get
directions
want
to
your
Suppose that
you
at an unfamiliar
arrive
would be
You
hotel.
impatient
quite
rent
city,
were
if you
the history
about
rather
have
worthwhile.
nonnumeric
The design of efficient
diverse fields, including
mathematics,
This book can serve as an introduction
general.
is becoming important
molecular
biology, and
in
algorithms
statistics,
and to
to algorithms
not
nonnumeric
deeply
many
engineering.
computations
involved
with
in
computers,
more
than
nonintellectual work. It
grungy
programming is nothing
But such a belief may lead to straightforward,
solutions,
trivial, inefficient
of
to
convince
solutions
exist.
One
this
book
is
more
efficient
where
goal
elegant,
is an elegant discipline, as well as an important one.
readersthat algorithm
design
The bookis self-contained.
The
is mostly intuitive,
and technicalities
presentation
are either kept to a minimum
In particular,
or are separated
from the main discussion.
from
the algorithm-design
ideas as much
as
implementation details are separated
There
are
of
that
were
possible.
many
algorithms
designed especially to
examples
illustrate
the principles
in
the
book.
The
in
this
book
is not
material
emphasized
as
to
be
series
of
mastered
and
memorized.
It
is
as
a
presented
something
presented
and so on.
ideas, examples, counterexamples,
modifications,
improvements,
Pseudocodesfor most
the descriptions.
Numerous exercises and
algorithms are given following
a discussionof further reading, with a relevant bibliography,
follow
In
each chapter.
most chapters, the exercises
are divided into two
drill exercises
and creative
classes,
exercises. Drill exercises
are meant to test the reader's
of the specific
understanding
believe
that
is.
sometimes
examplesand
the reader's
particular
(those
ability
algorithms,
whose
presented
algorithms
numbers
to
use
to solve
new
problems.
are underlined)
material. Chapter 2 is
in that chapter.
the techniques
an
ideas
follows.
introduction
are
at
given
the
end of
the
book.
The
chapters
introduced.
Chapters
to
through
mathematical
4 present
induction.
introductory
Mathematical
to algorithm
with
is, as we will
see,
very
important
design.
Experience
induction proofs is thereforevery helpful.
few computer-science students
Unfortunately,
2 may be quite
for some
difficult
get
proofs.
enough
exposure to induction
Chapter
the more difficult examples at first reading,
students. We suggestskipping
and returning
to them later. Chapter 3 is an introduction
to the analysis of algorithms.
It describes
the
and gives the basic tools one needs
to be able to perform
process of analyzing
algorithms,
induction
Preface
of the algorithms
analysis
simple
Readers
the book.
Chapter 4 is
with basic data structures
presented in
vii
a brief introduction
and
who have a
from
5
is
a good idea
background
directly
Chapter (it
always
5
the
basic
ideas
behind
the
though).
Chapter
presents
approach
the analogy to induction
It gives several
of designing
through
algorithms
proofs.
and describes their creation. If you read only one chapter
algorithms,
examples of simple
structures.
data
to
basic mathematical
to read the introduction
in
There are
book
the
can start
read Chapter
book,
this
5.
basic
two
according
geometric algorithms.
Even
Another
Chapters
organization.
way
of
the emphasis
though
familiar
are
who
is to
divide the
this
book
is on
through
present
for
book
graph algorithms,
example,
to design
according
techniques.
chosenthe
areas:
four
in
divide
way is to
One
algorithms.
former
for
algorithms
sequences
sets
and
algorithms
convex hull,
multiplication,
sorting,
(e.g.,
easier to follow.
10
Chapter
earlier
chapters,
to reductions. Although
the subject is unique
and
important
is devoted
own. This
of algorithm
part
theory.
and
completeness
the
algorithms.
of parallel
computation.
The material
which leaves
include
parts
book is
this
of Chapters
for the
more
parts
course,
of these
and can
appear
of its
a chapter
the
with
deals
in
essential
an
NP-
about
than
instructor.
3, 5, 6, 7, and
advanced
for a first
several
contains
choices
many
in
It
enough to warrant
to Chapter 11, which
complexity
designs
for proving
techniques
to parallel
who
Anyone
act
of reductions
examples
8 in
can
A
be covered
first
some depth,
chapters, along
not
although
with
be used as a basisfor
a one-semester
in
in algorithm
course
Chapters
a more
course,
design
should
all of
necessarily
9, 10,
advanced
11, and
course.
12,
Acknowledgments
and
First
here
foremost
on which
methodology
\342\200\224
more
I thank
this
throughout
important
adventure.
than anything
me
in more
in
the
suggestions,
I could
ways
than
corrections,
not have
list
I can
development
done it
of
the
and
without
her.
Preface
viii
of Iowa),
of Washington),
Victor Klee (University
H. Kingston (University
of
of New
Michael
J.
Martel
California,
Davis),
(University
Quinn
(University
Charles
M.
Diane
Madison
and
(
James
Spresser
University).
Hampshire),
who failed to supply
I thank the people at Addison-Wesley
pie with any examples
authors
are
so
fond
of
that
were
stories
of horror
very helpful and
telling.
They
In
I
and understanding.
thank
particular,
my
production
supervisor
patient
incredibly
Jefferey
Tektronix,
Award,
a Linotronic
300
Jim
do
DeWolf,
and AT&T,
for
Digital
and
way
my copy editor
even when they
financial
Lyn
Hewlett
Corporation,
Equipment
who
not only
thank
I also
better.
a Presidential
through
support,
Dupr6,
knew
Young
Packard,
and
funds.
designed
at
my
things
Foundation
for matching
let me
Science
National
Investigator
also
but
the
typeset by me. It
of Computer
Department
and
was
formatted
Science,
University
in troff, and
of
printed
Arizona.
Ralph Griswold for his advice, and John Luiten, Allen Peckham, and Andrey
\342\200\22
the typesetting.
The figures were preparedwith gremlin
for technical help with
Yeatts
for Fig. 12.22, which
at the University of California,Berkeley\342\200\224
was
except
developed
was
drawn
Townsend.
The
index
and
with
the
of
a
by
Gregg
compiled
designed
help
I
thank
and
Brian
for
me
the
Kernighan [1988].
Kernighan
system by Bentley
supplying
within
minutes after I (indirectly) requestedit. The cover
code
was done by Marshall
Henrichs, basedon an idea by the author.
I must
was prepared by the typesetter.
stress, however, that the final manuscript
He was the one who decided to overlook many
comments
and suggestions
of the people
listed here. And he is the one who should bear the consequences.
thank
Tucson,
Udi Manber
Arizona
(Internet
address:
udi@arizona.edu.)
CONTENTS
Chapter 1
Introduction
Chapter2
Mathematical
2.1
Introduction
2.2
Three
2.3
2.4
2.5
More
Simple
2.7
Euler's
2.8
2.9
Gray Codes
2.10
Finding
2.11
Arithmetic
Summation
Complicated
Problem
16
17
Inequality
Formula
in Graph
Problem
Paths
Geometric
versus
18
20
23
Theory
Edge-Disjoint
2.14
11
13
14
15
Examples
Simple
2.6
2.13
Induction
Converting
in a Graph
Mean Theorem
a Decimal
Number to
24
26
Binary
28
Errors
Common
29
Summary
30
Reading
31
Exercises
Chapter
Analysis
3.1
Introduction
3.2
The
3.3
3.4
3.5
3.6
3.7
37
of Algorithms
0 Notation
Recurrence
Relations
3.5.1
Intelligent
3.5.2
Divide and
3.5.3
Recurrence
Guesses
Conquer Relations
Relations
with
Full History
Useful Facts
Summary
Bibliographic
Exercises
Notes
37
39
42
43
46
47
50
51
53
55
55
56
IX
Contents
Chapter
61
Data Structures
4.1
4.2
4.3
61
Introduction
4.2.1
Elements
4.2.2
Arrays
4.2.3
Records
4.2.4
Linked
Lists
Trees
of Trees
4.3.1
Representation
4.3.2
Heaps
4.3.3
4.3.4
AVL
71
Trees
4.4
Hashing
4.5
The Union-Find
4.6
4.7
62
62
63
63
64
66
67
68
Structures
Data
Elementary
75
78
Problem
80
83
Graphs
84
Summary
85
Notes
Bibliographic
86
Exercises
91
Induction
by
91
92
95
96
98
Introduction
5.2
Evaluating
5.3
Maximal InducedSubgraph
5.4
Finding
5.5
The Celebrity
5.6
A Divide-and-Conquer
5.7
5.8
Computing BalanceFactors
Polynomials
One-to-One
the
Finding
Mappings
Problem
Strengthening
5.10
5.11
Dynamic Programming:
Common Errors
5.12
Summary
the
Induction
Skyline
102
104
Problem
Trees
in Binary
Consecutive
Maximum
5.9
The
Algorithm:
106
Subsequence
107
Hypothesis
The
Problem
Knapsack
108
111
112
Bibliographic
113
Notes
114
Exercises
Chapter 6
6.1
6.2
6.3
6.4
and
Sequences
Involving
Algorithms
Sets
119
119
Introduction
120
125
127
127
Sort and
6.4.1
Bucket
6.4.2
Insertion
6.4.3
Mergesort
Sort
and
Radix
Sort
Selection
Sort
130
130
xi
Contents
6.4.4
131
Quicksort
137
6.4.5 Heapsort
6.5
6.4.6
Order
Statistics
141
for Sorting
Bound
Lower
143
6.5.1
Maximum
6.5.2
Finding
Minimum
Elements
143
fcth-Smallest
Element
144
and
the
6.6
Data Compression
145
6.7
String Matching
148
6.8
SequenceComparisons
155
6.9
Probabilistic
158
Algorithms
6.9.1
Random
6.9.2
A Coloring Problem
6.9.3
6.11
6.12
161
Probabilistic
for Transforming
Technique
into
Algorithms
6.10
160
Numbers
Deterministic
161
Algorithms
164
a Majority
Finding
Three ProblemsExhibiting
6.11.1 LongestIncreasing
6.11.2
Finding
6.11.3
Computing the
Interesting
167
Techniques
167
Subsequence
Two
the
Proof
Largest
Elements
of a
Mode
in
169
a Set
171
Multiset
173
Summary
Notes and
Bibliographic
Further
173
Reading
175
Exercises
Chapter 7
Graph Algorithms
185
7.1
Introduction
185
7.2
Eulerian
187
7.3
Graph Traversal
Graphs
189
7.3.1 Depth-First
7.3.2
7.4
190
Search
198
Breadth-First Search
199
Sorting
Topological
7.6
7.7
All
7.8
Transitive Closure
7.5
7.9
Shortest
7.9.2
7.9.3
7.10
208
212
Paths
of
Decompositions
7.9.1
201
214
217
Graphs
217
Biconnected Components
Connected
Strongly
Examples of the
Use
Components
of Graph
226
Decomposition
7.10.1
Perfect
7.10.2
Bipartite
Matching
Matching
7.11
Network Flows
7.12
Hamiltonian
7.12.1
230
234
Matching
Very
Dense
Graphs
.234
235
238
243
Tours
Reversed
in
Induction
244
xii
Contents
7.12.2
7.13
Hamiltonian
Finding
Cycles
in Very
Dense Graphs
Summary
Notes
Bibliographic
Exercises
Chapter 8 GeometricAlgorithms
8.1
Introduction
8.2
Determining
8.3
Constructing
8.4
Convex
8.5
265
a Point
Whether
265
266
270
Is Insidea Polygon
Polygons
Simple
273
Hulls
8.4.1
A Straightforward
8.4.2
Gift
8.4.3
Graham's
Approach
273
274
Wrapping
Scan
275
Closest Pair
278
8.6 Intersectionsof
8.7
and Vertical
Horizontal
281
Line Segments
285
286
287
Summary
Notes
Bibliographic
Exercises
Chapter 9
Numeric
and
Algebraic
293
Algorithms
9.1
Introduction
293
9.2
Exponentiation
294
9.3
9.4
Euclid's Algorithm
Polynomial Multiplication
297
Matrix
Multiplication
301
9.5
9.6
9.7
298
Algorithm
301
Algorithm
301
9.5.1
Winograd's
9.5.2
Strassen's
9.5.3
Boolean Matrices
304
Transform
309
The
Fourier
Fast
316
Summary
Bibliographic Notesand
Further
316
Reading
317
Exercises
Chapter
244
246
247
248
321
10 Reductions
10.1
Introduction
10.2
Examples
10.2.1
10.3
321
323
of Reductions
A
Simple
String-Matching
Problem
323
323
324
325
327
327
329
Contents
10.4
10.4.1 A
for Finding
Bound
Lower
10.4.2
Reductions
Simple
331
Bounds
for Lower
Reductions
331
Simple Polygons
333
Matrices
Involving
334
10.6
336
Summary
Notes and
Bibliographic
Further
336
Reading
337
Exercises
Chapter 11
341
NP-Completeness
11.1
Introduction
11.2
Polynomial-Time
11.3
Nondeterminism
341
342
Reductions
Cook's
and
11.4 Examplesof
Theorem
344
Proofs
347
NP-Completeness
348
11.4.2 Dominating
348
11.4.3
Set
350
3SAT
351
11.4.4 Clique
11.4.5
352
3-Coloring
11.5
11.4.6
General
11.4.7
More
355
Observations
356
Problems
NP-Complete
Problems
Techniques For Dealingwith NP-Complete
11.5.1 Backtracking and Branch-and-Bound
11.5.2 Approximation
with
Algorithms
357
358
Guaranteed
363
Performance
11.6
368
Summary
Notes and
Bibliographic
Further
368
Reading
370
Exercises
Chapter 12
xiii
Parallel Algorithms
12.1
Introduction
12.2
Models
12.3
375
375
of Parallel
for
Algorithms
376
Computation
378
379
380
382
Machines
Shared-Memory
Algorithms
Maximum-Finding
12.4
Ranks
12.3.4
Finding
12.3.5
The Euler's
in
for Interconnection
Algorithms
Lists
Linked
385
Tour Technique
12.4.1
Sorting
on
12.4.2
Sorting
Networks
12.4.3
Finding the
12.4.4
Matrix
12.4.5
Routing
387
Networks
389
390
an Array
Multiplication
in a
393
fcth-SmallestElement
Hypercube
on the
Mesh
on
a Tree
396
398
401
xiv
Contents
12.5
SystolicComputation
404
Matrix-Vector
12.5.1
12.6
Sequence
404
Multiplication
405
Problem
407
Comparisons
409
409
Summary
Bibliographic Notesand
Further
Reading
411
Exercises
tions
to Selected
Exercises
417
References
445
Index
465
CHAPTER
INTRODUCTION
Great
of
has been
importance
necessary
and
For a
claim
condition
sufficient
sciences.
exact
of
no
Necessary,
be useful
to serve as a
but
doubt,
to
construction
to this process
to see in it the
the progress of the
attached
rightly
some
and
\"construction,\"
and
not
mere
not
to higher
stepping-stone
a kind
the
sufficient!
waste of
enabling us
juxtaposition
of its
of unity
elements.
Henri
The Webster's
solving a
of steps
number
New
Ninth
mathematical
by-stepprocedure
starting
fires,
frequently
for
building
solving
1902
Collegiate
problem
that
Poincare,
design
pyramids,
methods
better
finding
or
the
sorting
to achieve
mail.
their
The study
goals,
stick
will
People
whether
to
have
those be
of computer algorithms
is
to sound
some reason,
via
instructions
of people.
sounding
well-defined,
an alarm
limited
primitive
to a language that
is now performed
is
always
operations.
a computer
on one
easy).
A computer
It is a difficult
understands.
level or
another
This
by
Introduction
understood
instructions
to devise totally
to a
\"talk\"
what
possible.
with
dozens,
perform
of
used to
We are
counterintuitive.
of
millions
this order
learning
magnitude
is a tendency when
result,
and feel.
that works very
well
approach
straightforward
that work well for small problemsmay
algorithms
and inefficiency
of the complexity
easy to lose sight
in
designing
terrible
be
of
need
weird
is knowing
it
performed
was
than
by
ever
of items, and, at
or even trillions,
instructions per
primitive)
is something
new. It is in
terms of things
we can see
an algorithm
to use the
small
for
more
the
hundreds
with
(their
thinking
there
As
cases, we
to program;
much
maybe
deal
can
Computers
on
algorithms
Designing
respects
many
instructions.
and can
information,
of
bits
second.
of
thousands
past dealt
the
of
Algorithms
with
most,
not just
hard
well-
translating
just
In most
operations
only
it
than
is
enormous
their
with
that makes
computer
humans;
a problem. It
for solving
methods
new
in which we
language
of
however,
requires more
computer,
a computer can understand.
to a language
Programming
an
billions,
problems. Unfortunately,
for large problems. It is
algorithm when applied to
large-scalecomputations.
too
not
to expend
efficient and
of
the problem
the
algorithm.
perfect
grocery
it,
we perform
algorithms
It is
too often.
performed
unpacking
is organized.
kitchen
The
problem.
not
ways of doing
efficient
more
the
way
to develop
effort
of
lot
consider
example,
the
aspect to this
and are
complicated
is another
There
life are
depending
in
our
daily
worthwhile
usually
not
even
thinking
about
this
other
problem,
These
algorithms
learning
always
less
much
and
\342\200\224
the
issues
two
need
this subject.
the best. It is
for
complexities of
First, we must realize
the possible
important
to continue
methods.
it.
the
On
approaches to large-scale
counterintuitive
these algorithms
that
\342\200\224
straightforward
book
surveys
point
difficulties in
methods are not
to the
intuitive
methods. To do that,
and
illustrates
we
numerous
just as
it
is not
enough
Introduction
house.
of a
construction
the basic
with
start
We
concepts, based on
the
for
requirements
the house.
It
is the
the engineer's
in
which
They usually require modifications to the design,
phases of the construction.
another
of
and
of
turn require
costs,
feasibility
proof, adjustment
change implementation.
This book concentrates on the first step, the design of algorithms.
our
Following
been
entitled
The Architecture of Algorithms.
However,
analogy, the book could have
has a different
so using this term would
be confusing.
architecture
meaning,
computer
A
all
the
other
discussion
of
The book does not, however,
correctness,
aspects.
ignore
\342\200\224
in detail
the description of most algorithms
for
follows
analysis, and implementation
for others.
The emphasis is on methods of design.
some algorithms, briefly
to be a good architect
and
to be able to
to learn many algorithms
It is not enough
behind
the design. We
One must understand the principles
design new algorithms.
of explaining
employ a different
way
algorithms in this book. First, we try to lead the
we strongly
believe that the best way to learn how
reader to find his or her own solution;
to create something
is to try to create it.
and more important, we follow a
Second,
for
methodology
The methodology,
process.
algorithms that helps this creative
the
intuitive
framework
for explaining
an
[1988], provides
elegant
the
to approach
in
more
It also provides a unified
depth.
way
and their
methods
that are encompassed by this methodology,
designing
in Manber
introduced
of algorithms
design
variations,
are
possible
for a
however,
great
analogy between
the
possible
of the
majority
The methodology
methods
does
methodology
mathematical
process
of
ways
algorithms
is based on
intellectual
of choosing
same technique. The process
becomes more methodical.This
of the
instances
in
this
designing
book.
induction.
of proving
algorithms.
The
heart of
It is
it
lies
useful,
in an
designing
induction
The
two
(Houghton
is that a
wonderful
Mifflin,
books
1985),
by Tracy Kidder,
inspired this analogy.
Machine
(Little
Brown, 1981),
and
of
that
for
House
Introduction
Gries
done by several people, including Dijkstra [1976], Manna
[1980],
others.
This
and
Paull
book
Dershowitz
[1988],
[1983],
among
complements
[1981],
but of course whenever something
Our goal is mainly
books.
other
these
pedagogical,
we
better.
is
understood
it
better
can be explained
Among the proof techniques
usually
has been
This
the
induction
discuss are strengthening
sequence
hypothesis,
choosing the induction
is
The significance of our approach
and
reverse
induction.
wisely, double induction,
one
of algorithm design under
different
two-fold.
First, we collect seemingly
techniques
known
mathematical
second, we utilize
umbrella;
proof techniques for algorithm
design.
it opens the door to the
of powerful
since
use
The latter is especially important,
in
for
another
have
been
that
developed
discipline.
many
years
techniques
One notable weaknessof this approach is that it is not a universal approach. Not
the principle
be designed
with induction in mind.
can or should
all algorithms
However,
in
it is worthwhile
to
is so prevalent
the
of algorithms that
of induction
design
concentrate on it. The other principles are not ignored in this book. A common
criticism
of almost any new methodology
is that, although
an interesting way to
it may present
that
were
is
in
This is a valid
it
of
no
them.
created,
explain
things
help
already
creating
since only the future
will
is and how
tell how effective a certain methodology
criticism,
used
it
becomes.
I
is
another
tool for
believe
that
induction
not
widely
strongly
only just
is
in
but
it
order
to
understand
them.
explaining algorithms,
necessary
Personally, even
I
though
methodology,
algorithms
a good
had
I found
more
experience
it
quickly
and,
helpful,
(Manber
and
in
algorithms
developing
at least
McVoy
in
two
[1988],
cases,
Manber
without
following
this
me to develop new
and Myers [1989]).
it led
addition
development,
including
to
we
the
describing
also include
programs
is to
algorithms
pseudocodes for
enhance
the
through
many
creative
algorithms.
process
The
of
purpose
their
of
the
We
have not
additional
which distracts from the main
ideas of the
complexity,
in detail how we translate
the algorithmic
ideas
algorithm. We sometimesdo not explain
into a program. Such translations
The
sometimes
are obvious and sometimes are not.
in this book, as we mentioned,
is on the principles of algorithm
emphasis
design.
For the most part,
we use a Pascal-like
In
language (sometimes even pure Pascal).
into a table,\" or \"check
as \"insert
many cases, we include high-level descriptions(such
the set is empty\") inside a Pascal code to make
whether
it more readable. One notable
exception we make to the rules of Pascal is the use of begin and end to encompass
because
Exercises 5
at the beginning
and end of the programs,
and
This
convention
saves
without
separate
space
causing
We usually do not include
declarations
of variables and data types
precise
ambiguities.
in cases where such declarations are clear (e.g.,we may say that G is a graph, or that T is
let the
these statements
include
We
blocks.
only
blocks.
the
indentation
a tree).
Exercises
whose
Exercises
numbers
by a star
marked
are judged
by the
solutions at
author to be substantially
the back
more
of the
difficult
The exercisesin
chapters.
1.1
Write
1.2
Write
about
the numbers
down
them in
100 each
to
on a
separate
card.
Shuffle
the cards
and
rearrange
order again.
and
sort
the cards.
Think
the differences
49472 113292253
1.3
Consider
possible
such
erased).
9 44 32
12 7 42 34 92 35 37 41 8 20 27 83 6461 28 39 93 29 17 13 1455
72 23 73 99 1 2 88 77 3 65 83 84 62 5 11 74 68 76 78 67 75 69 70 22
25 26
21 66
7124
1 -4
Solve Exercise
1-5
Suppose
that
8 cents
1.3, such
in a
67
strange
the remaining
that
Find
(1808 cents).You
a combination
have
decreasing
order.
there
country
(all cents).
numbers are in
of 15, 23,
of 18 dollars
Introduction
The
list of
is a
input
waiting
it
particular,
k will
answer
cannot
problem is to
(any
of integers
pairs
an answer from
for
y.
any questions
a sequence
find
do). If such a
since everyone
is waiting
pencil
of pairs
of
a piece
such
proceed,
make
and
paper,
creating
however,
tables);
unrelated to
figures,
some k >
can
else.
for someone
and
x |), for
No one
is a deadlock.
there
for
waiting
x is
and, in
it. The
else,
anything
be
\342\200\242
\342\200\242
\342\200\242,
(**_, xk\\ (xk
x^),
(x | jc2), (x2
is that
of a pair (jt, y)
do
that may
others
from
sequenceexists, then
(e.g., comparisons,
numbers
When
this
a problem.)
24 1, 25 10,35
39 41,
50,
23 47,
12 20,
9, 42 3,
14 25,41
7, 36
43 5,43
28,511,48
26 10,2644,2616,26
39,2617
two-dimensional 15 by 15 table given in Fig. 1.1. The /th row and the /th
correspond to the same place. Each entry in the table indicates the direct
between the places in the corresponding
distance
row and column. The \"-\" symbol
indicates that there is no direct link between the two places. The direct
distance
may not be
the shortest
distance.
There may be a shorter path between
two places going through
a third
1 and 6 is through 5 and
the shortest route between
place (or several places). For example,
12. Find the shortest route
1 and
between
15, between 4 and 3, and between 15 and 8.
input is the
The
column
Consider
(for any /)
the table
in
Fig.
shortest route
between
5 and
all
other
places.
Consider the graph shown in Fig. 1.2. Find a closed route along the edges of the graph
which includes every vertex exactly
of a
once.
(This graph corresponds to the edges
Sir William
R.
dodecahedron; this puzzle was first described by the Irish mathematician
Hamilton, and we discuss it further in Section 7.12.)
The
numeric
(3,2) (6,6)(7,0) (2,8)(5,9) (8,4) (2,4) (0,8) (1,3) (6,3) (9,3)(1,9)(3,0) (3,7)
(4,2) (7,8) (2,2) (4,5) (5,6)(10,5)(6,2)
(4,0) (7,5) (7,9) (8,1) (5,7)
(6,10)
(4,4)(8,7)
a.
b.
Find
a path from
Find
a shortest
obstacles.
(9,2)
(10,9) (2,6)
target
to
target
that does
any
not
of the
include
obstacles.
any
of the
Exercises
3
3
10
12
13
14
15
9
-
0
-
4
?
3
2
10
12
13
14
7
2
15
11
3
2
8
2
9
8
0
2
1.11
Find
the
of two
1.12
greatest
for Exercises
integers is the
Compute the
value
of 2M. Try
Figure
to find
1.2
a way
Hamilton's puzzle.
divisor
Introduction
1.13
list represents
following
election
discussed
in
Section
5.10.)
Alabama
Alaska
Arizona
Arkansas
California
47
Colorado
Connecticut
6
8
Delaware
Florida
21
Georgia
12
Hawaii
Idaho
Illinois
24
Indiana
4
8
Kansas
Maine
Michigan
20
Minnesota
Missouri
11
Montana
Nevada
South Carolina
4
5
3
7
8
Texas
29
Utah
Virginia
12 Washington
6 Wisconsin
New Mexico
Dakota
North
Oregon
West
Virginia
12
7
8
Iowa
9
10
10
4
4
New York
36
North
Ohio
23
Oklahoma
13
8
Kentucky
Maryland
New
Hampshire
Pennsylvania
South
Dakota
Louisiana
10
Massachusetts
13
Mississippi
Nebraska
New Jersey
16
Carolina
25
Rhode Island
Tennessee
11
Vermont
10
Washington,
11
Wyoming
D.C.
CHAPTER
INDUCTION
MATHEMATICAL
No
Anon
is always the
Obviousness
enemy
of correctness.
Introduction
2.1
in the
see
will
We
In this
design.
The
examples.
proofs may
this
find
T be
value
whose
a theorem
can be
of proving
Instead
natural
any
directly
that
Readers
difficult.
algorithm
through
not
We claim
are
algorithms
constructing
role in
induction
who have
similar,
seen
the
that
and
thus
helpful.
powerful
technique.
proof
It usually works
as
we
that
plays a major
to mathematical
to be relatively hard.
chapter
follows. Let
induction
induction
many
that
chapters
following
conditions:
1. T
2.
holds for
For every
reason these
that T holds for n
The
and
so on.
two
= 2.
= 1
> 1,
if T
conditions
If T
holds for n
- 1,
are sufficient
holds for
= 2,
then T holdsfor
is clear.
Conditions1 and
then condition
2 implies that
it is
usually
2 imply
T holds
directly
for n = 3,
it
Induction
Mathematical
10
is stated as an
is usually
an example.
with
away
in the
axiom
Theorem
\342\226\241
2.1
For all
numbers
natural
x and
n, xn
by x
is divisible
- 1.
is trivially
induction on n. The theorem
true for n = 1. We
assume that the theorem is true for n 1; namely, we assume that xn~x - 1 is divisible by
x. We now have to prove that xn -\\ is divisible by x- 1.
numbers
x- 1 for all natural
The idea is to try to write the expression xn -\\ using xn~{ -1, which,
by the induction
- 1:
x
divisible
is
by
hypothesis,
Proof: The
is by
proof
+ (x-\\).
xn-\\=x(xn-l-\\)
But
just
jc
is divisible
left term
the
by
hypothesis, and
induction
the
by
the
term
right
\342\200\224
1.
The
If a
is
\342\226\2
induction
statement
P,
the truth of
is
principle
with a
n - 1
P for
as follows:
defined
thus
parameter
implies
n,
for n =
is true
its truth
for
n,
then
\\,
and
P is
every
if for
true for
> 1,
all natural
numbers.
of using
Instead
-1
and
P,
with a
we
\302\253,
use
sometimes
n and
/i
which
+ l,
is completely
equivalent:
If a statement
the
of P
truth
for
parameter
implies
n,
its truth
for n =
is true
for
+ \\, then
\\,
and
P is
every n>\\,
if for
true
for
all natural
numbers.
The proof of
Theorem
2.1 illustrates
a simple application of induction.
have
been developed.
For example, the
many variations of induction
called strong induction, is very common.
If a statement
P,
the truth of
true
for
P for all
all natural
in
proving
Another
with a
parameter
natural
n,
numbers
is true
< n
for n =
\\,
implies
its
and
truth
if for
for
Over
following
the years,
variation,
every n > 1,
n, then P is
numbers.
can use
the statement
Three
2.2
If a
statement
a parameter
with
P,
the truth
every n>2,
all natural numbers.
for
for
This variation
step imply
for all even
is true
for n =
implies
for all
numbers.
If a
such
for
This variation
and
truth
its
for
for
''works\"
P
for
ofP
n,
n-2
the induction
induction
step
imply P
n>
\\,
1, and
of P
if,
every
for
Three
2.2
The
+ 2
the
to
S(l)=l = l-(1+
the
of
the
first
n natural
first
is by
proof
We
l)/2.
induction on n. If n-
now assume
that
the
sum
1, then
of the
the
first
is true
claim
(n +
because
numbers
n natural
numbers
natural
theorem.
following
2.2
Proof: The
expression
We prove
\342\226\241
Theorem
is
the
find
writing
Examples
\342\200\242
\342\200\242
\342\200\242
+n.
by
k (starting
parameter
Simple
is
problem
S(n)=
from
follows
induction for
out the
n = 1 and
n =
are
11
Simple Examples
n +
first
S(n)
natural
and
S(n) that
therefore
we
wanted
of
prove.
to
\342\226\
We
compute
continue
the
sum
with a slightly
more
sum. Suppose
complicated
= $ + 13+ 18+ 23+ \342\200\242\342\200\242\342\200\242
sum
+(3 + 5\302\253). The
T(n)
that
we
in the
want
to
previous
of the elements
to n2l2 + nl2. Each
in the current
is
example
the corresponding element in the previous
Hence,
example.
it is reasonable
to guess that T(n)
is also a quadratic expression. Let's try the implicit
That
the parameters c,, c2, and c3, and
guess G(n) = cln2 +
is, we introduce
it is convenient
we can determine the
to do so. For example,
determine their values when
example,S(\302\253), is
slightly
more
than
equal
f\\\\e
times
c2n+c3.
Induction
Mathematical
12
(2) 4-c, +
If we
We
expression.
verified
+ l)
We have
now
+ l)2
77^
series
5\302\253w
fl/f/je
18 + 23+
end this
Ifn is a
+3
5.5\302\253+5.5
1).
We have to
example.
simple
+jc>
0, then
(2.1)
+ (n
prove that
+ 1)jc +
assumption
+\302\253jc
for
(1 +
(by induction)
nx2
> 1 + (n
multiply
that
the
l+*>0.
= 1,
all x
jc)/,+i >
>
+JC)\302\253
were able to
clearly nonnegative.
induction on n. If
that (1 +jc)\"> 1
(1 +xy?+l = (i +JC)(i
that we
and
is by
proof
+ 1.
another
with
1+fljt.
assume
the
right
already
to prove that
\342\226\2
We
because of
the
is
have
theorem.
number
natural
Proof: The
Notice
try
that
2.4
(\\+x)n>
case of n
we
and
+ 5(/i+l)
+
+ 5\302\253
+ 2.5
2.5\302\2532
We
implies
\342\200\242\342\200\242\342\200\242
+ 5\302\253)
+(3
section
\342\226\241
Theorem
+*.
For
+ 5.5\302\253.
is2.5\302\2532
induction.
by
C(/i)
induction)
following
2.3
8+ 13+
to
l) + 3 = (by
= 2.5(/i
Theorem
\342\226\241
= T(n)
G(n) = T(n),
that
+ 5.5\302\253+5\302\253
+ 8
the
guess
assume
We
(2), we get
that G(n)
from
that G(n)
prove
= 2.5a?2
proved
We
therefore
to
5(/? +
7\302\273 +
it
subtract
We
try
a base case.
= 7,(/i + l):
+ l)
7,(/?
0.
2 and
by
and c2 = 5.5.
Cj=2.5,
C(/?
= 13 +
2-c2
(1)
multiply
be
lc2 = 8
l-c, +
(1)
so c3 must
sum is 0,
0, the
then both
such
+ (n
that
1 )a\\
sides of (2.1)are
and
+jc>0,
for all
x such
that
equal
consider
1+
the
x> 0:
(1 +jc)(1 +\302\253jc)
1 )jc.
inequality
(implied
by
the
induction)
possible
because
by (1 +jc)
nx2 is
\342\226\2
2.3
2.3
Counting
Regions in the
Counting
Regions
in the
Plane
13
Plane
Guess:
Adding
increasesthe
n-\\ lines in
is true
guess
general
position
We
can now
in the
plane
by n.
of regions
for
<3.
use
the
guess
as our
total
the
number
number
of
of regions, but
can a new line
to prove
regions
this
that adding
by n +1.
will be
part
straightforward.
increase the
Figure 2.1
number
+ 1 lines
of regions?
in general
position.
Mathematical Induction
14
region
lines
two
the
one.
of just adding
meet at only one point.
instead
regions)
of the
presence
nth
but
line,
R is
Furthermore,
the
Hence,
it adds
n+
only
line
1th
the
with
+ 1 regions
the
so affected,
region
adds n
A?th
regions
and
line,
without
since
the
the proof is
complete.
2.5
Theorem
\342\226\241
Proof:We
have
2 +2+
1+2
3 +
+ 3+
4 + 5+
+ l)/2;
+n=n(n
therefore,
the
line
total
n more
of
number
seen
We
\342\200\242\342\200\242\342\200\242+/!.
have
\342\200\242
\342\200\242
nth
total
the
position is
in general
n lines
by
that the
hence,
regions;
formed
plane
proved
already
two
introduces
line
the
regions
Comments
of the
the
with
2.4
A Simple
We are
again
n distinct
in
neighboring
we
lines
assigning
were
Problem
a plane,
in
colors
have different
regions
points
interesting
Coloring
interested
Consider
two
function
to
colors
this time
the
(two
regions
regions
not
necessarily
in general
position.
formed
that
and
edge
say
possible to color\" the
follow this rule, and
of
colors
a valid coloring.
assignment
In general,
it is possible to color any planar
colors
of this fact
with
four
(the
proof
map
has occupied mathematicians
for about
a hundred years, and was found only recently).
The regionsformed
as is shown
lines, however, have specialcharacteristics,
by (infinite)
if they have
if we can
regions
only
in
the
next
in common).
an
We will
we call the
that
\"it is
theorem.
\342\226\241
Theorem
2.6
It is possible to color
with only two colors.
the regions
formed
by
any
number
of lines
in
the
plane
2.5
Proof: We
Induction hypothesis:It is
lines in the
It is
clear
plane
15
Problem
to color
possible
the regions
formed by
<
two colors.
only
are necessary
colors
two
that
with
Summation
Complicated
induction hypothesis.
the natural
use
More
n=
for
sufficient
and
1.
the
Assume
induction
to modify
and consider
the coloring
the nth line is added.
when
side
according to which
the same
as before, and
of the nth line they lie. Leave all regions on one sidecolored
To prove that this is a valid coloring,
the colors of all regions on the other
side.
reverse
R
If
both are on the same side of the A?th
and
we considertwo neighboring
\\
R2.
regions
line was added (by the induction
then
were
colored
before
the
line,
they
differently
the
reverse
but
have
If the edge
colors,
They may
they are still different.
hypothesis).
between them is part of the nth line, then they belonged to the same region before the line
colored
Since the color of one region
was reversed, they are now
was added.
differently.
lines.
hypothesis,
\342\226\2
The
Comments
flexibility,
as much
as
given a
was
general
or for more
coloring,
the
to handle
used
degrees
of
of new
formation
next
example
idea is
The
by the
regions
complicated. Considerthe
is more
following
=
The
of
is to
the
64
21+23+25+27+29
125
the
rows
seem
This idea
11
of the
regular
ith
row,
pattern;
cubes.
Induction
The
27
expression
for
an
search
the hypothesis
idea was that,
triangle.
13+15+17+19
find
sums of
the
stretch
=8
7 + 9
problem
usually
to
added line.
The
is
example
Summation Problem
this
in
illustrated
freedom.
in order to get the most out of it. In this case, the key
we can reverse all colors and still have a valid coloring.
possible
valid
method
problem
problem
to be
vaguely
hypothesis:
i in
the triangle
is i3.
Therefore,
the
problem
we
will
make
accordingly.
some
16
Induction
Mathematical
is the difference betweenthe first number in row i + 1 and the first number in
the numbers are the odd numbers in order and there are i of them in row i,
Since
/?
row
the second number in row i + 1
between
the difference is 2/. This is also the difference
third
the
fourth
in
row
the
number
number,
i,
number, and so on. Overall,
and the second
is
also
the
of
size
2i.
There
last
element at the end of row
each
are / differences,
there
What
i+l,
to
matched
not
is
which
in the
number
any
play.
We prove
element.
of
the problem
reduced
have
We
differences.
i+
and
[ i2 + 3/
But
we
and
1 ]
at
problem,
this
illustrates
difference
starting
final
the
with
Thisis a very
the
sum
is i3
comes to
an
finding
+ 3/
is i2
+ 1.
by induction, to check
between the last number
1 ]
= 2/
the
only
in row
+ 2.
in row i +
numbers
corresponding
established.
that we should
policy to advance
again
from
l. Since
in row
equal to
a good
is
It
proof also
the
illustrates
a simpler
problem
try to
always
in
achieve
long as
as
stages,
the
we are
method
problem
and simplify
method
common
not
and
our
working
it by
(not only
in
reducing
way toward
the
to simpler
and
is clearly
true
it
mathematics).
Inequality
we prove
section,
sufficient,
the
- 1)+
been
of
we start
A Simple
it is
difference betweenany
thus
step.
Instead
simpler problems.
In
+ 3(i
1f
This
progress.
a proof.
2.6
This proof
in one
proof
final
[ (i
Now,
i is
row
in
Comments
making
1.
in row i
number
last
already
is 2i.
whole
for i =
is true
claim
The
finding
last
again by induction.
statement
last
the
number
i+
problem of
to a
sum
the
the
difference
the
row
in
between
the
following
inequality.
2.7
\342\226\241
Theorem
foralln>\\}
Proof:
for
we
information
'This
assume
get
to prove
that
(2.2)
from the
is usually written
of series; this formulation
inequality
knowledge
want
We
We
n-\\.
the theorem by
is true for
induction
as a
fact
hypothesis
about
is completely
convergence
finite.
The
induction.
and
\302\253,
is that
of
we
theorem
consider
the sum of
infinite
series,
but
/i
the
+ l.
first
The
only
is
n terms
we do not assume
any
less than
extend
can we
How
1.
to
it
the n +
include
1th
term?
the last
by
^fl
+
+ \342\200\224
the
left
apply
the
+l
the induction
is to
terms:
1
1
\342\200\224
\342\200\224
here
*y\\
we look at
l/2\"+l to
Adding
The trick
hand
17
Formula
Eulers
2.7
hypothesis.
now
But
2+4+I+
we can
2\"
and get
sides
the expression
\342\226\
(2.2)for\302\253+l.
It is
Comments
the
induction
instances where
some
The
planar
outside
next
it are
If
is easier
the
run
you
to consider
l)th element in
There are other
be a special element
be flexible, and consider
problems,
following
(n +
element.
element
+ l)th
(\302\253
into
as the
first
the
notion
satisfying
as many
further.
Formula
is for
proof
with
map
it
to let
better
The
can.
Eulers
The
faces.)
map
by traversing
connected.
are
said
E edges,
as one
map
to be
edges of the
Intuitively,
as
Euler's
and F
faces.
known
theorem
vertices,
is counted
region
is
it
to consider
necessary
properties.
special
options as you
2.7
not
Sometimes
proof.
in
map.
A map
face
11 vertices,
it
is possible
is called
a map is connectedif
is an
Consider a connected
enclosed region. The
has
a square
example,
2.2 has
connected if
Formula.
(A
to go from
connectedif
it consists
vertices
four
19edges,and
10 faces.
one vertex to
every
two
of one part.
2.8
\342\226\241
Theorem
Figure
2.2
A planar
map
with
11 vertices,
19 edges,
and
four
Two
10 faces.
vertices
the
in
Induction
Mathematical
18
Proof: We
will
The
induction.
proceeds
of induction
a variation
by
on the
first
as
known
double
and then on
vertices
of
number
the
of faces.
number
Consider first
because, otherwise,
a map
the
cycle
A connected
face.
another
all trees, V+
l=\302\243+
map
base case is
is
a cycle
without
does not
a map
a cycle
contain
outside
first
form
that, for
would
prove
2.
induction hypothesis:
First
The
this theorem
prove
induction
with
n vertices
have n
edges.
edges,
and consider
one edge.
if
the tree
and
we
traverse
Otherwise,
edges
then
we
are
to
vertex
from
to
return
a
vertex,
any
guaranteed
along the edge, starting
is a
stuck.
But this means that there
is a cycle, which
getting
already visited without
v along with
the edge
to it. The
connected
We can remove the vertex
contradiction.
But it has one less vertex and one
thus, it is still a tree.
map is still connected;
resulting
trees
with
There
n + 1 vertices.
less edge,which
Considera map
not
with
and
one
cycle.
of
less edge
has E edgesand
one
separates /
from
the
outside.
follows.
\342\226\2
used
on
induction
on
(the
required another induction
another
parameter (the number of vertices). The proof shows that we have to be careful
about
the right sequence of induction.
the induction switches from
Sometimes,
choosing
one parameter to another;
it is based on a combined value
of
several
sometimes,
and sometimes, it is applied to two different
at the same time.
parameters;
parameters
Choosingthe right sequence can make a big differencein the difficulty of the proof. As
we will see in the
the right sequence of induction
can also
following
chapters,
choosing
make a big difference
in efficiency of algorithms.
2.8
number
of faces),
but
is a
which
Removing
Comments
to only
of faces.
number
with n faces
have a face/,
by a
We removeone
one parameter
connected
+ 2.
It must
faces.
on the
map
planar
Any
= E
+ n
is surrounded
the map.
disconnect
We now
n +
a face, it
Since /is
face.
that
such
induction
an
induction hypothesis:
vertices
two
least
at
vertex
one
the claim.
implies
least
at
are connected to
vertices
all
if
must be
vertices has
with n
tree
trees
that
Assume
trivial.
the
base
case
in Graph Theory
A Problem
We
first
need to introduce some basic concepts of graph
(these concepts are
theory
discussedin detail in Chapter 7). A graph G = (V, E) consistsof a set V of vertices and a
set E of edges.
Each edge corresponds to a pair of distinct
A graph can be
vertices.
directed or undirected. The edges in a directed graph are ordered pairs: The order
between
as an
the two
arrow
pointing
vertices the
from
edge
one
is important.
(the tail) to another
connects
vertex
In
this
(the
case,
head).
we draw an
The edges
edge
in
an
2.8
A Problem
in
19
Theory
Graph
undirected
of
are
graph
unordered
v is
a vertex
2.9
Theorem
\342\226\241
Let G = (V,
E)
be a
directed graph.
in
can
on the
induction
The theorem
hypothesis:
Thereexistsan
be reached
is
from a
number of
true
for
set S(G)
independent
in
vertex
in
S(G)
by a
with
< n
vertices.
all directed
graphs
vertices.
The theorem is
trivial
for
n <3.
Let
be
vertex
an arbitrary
in
V.
= {v|
Let N(v)
V | (v,
hypothesis.
1.
u {v| is independent.
S(H)
2.
induction
induction
cases.
in N(v)
are reachable
N(v)
induction
^j
set of
the
In
this
case,
we can
is reachable from
from a
vertex
in
S(H)
with
set 5(G)
distance
with distance
be
to
S(H)
1. The
at
most
{v|,
vertices not
2 by
the
hypothesis.
be a vertex w e S(H)
that
In this case, there must
u {v| is not independent.
which implies that
is adjacent to v. Now, we S(H) implies that we V-/V(v),
w is adjacent
to v, (w, v)
of G. But, since we assumedthat
(v, w) is not an edge
in N(v)
can be reached
must be an edge of G. In that case, however, every vertex
w (through
from
v) with distance at most 2. We can set 5(G) to be S(H)
{j {vv|,
S(H)
which
completes
the proof.
of
\342\226\2
fixed.
in this proof was not
That is, we
\"reduction\"
on
instance
the
from
to
number
n
a
smaller
depending
problem
of
the problem. Furthermore, the smaller
of smaller
not
was
an
arbitrary
problem
problem
size. It depended
on the particular
We removed just enough
heavily
problem.
larger
vertices
to make the proof feasible.
There is a very
fine balance
in such proofs between
too
which
the
in
case
is
too
too
weak, and removing
removing
hypothesis
many vertices,
few vertices, in which
case the hypothesis is too strong. Finding
in
this balance
is,
many
Notice also that we used the strong induction
the heart of the induction
cases,
proof.
for all instances of smaller size.
it was required to assume the theorem
because
principle,
The amount
Comments
reduced the
size
of the
20
Mathematical
2.9
Gray
are
We
Induction
Codes
of n
a set
given
objects
a unique
scheme.
We deal
circular
in a
names
with
only
such
list
we want to
name them.
is represented by
for
a
may
many
objectives
\"good\" naming
We would like to arrange the
one objective in this example.
from
that each name can be obtained
the previous name by
and
Each
name
different
be
several
are
one bit. Such a scheme is called a Gray code.2There
sensor
scan
some
to
For
a
is
better
It
codes.
of
may
example,
objects.
Gray
applications
from
one
to
the
of
next.
The
be able to change representations
quickly
object
purpose
it is possible to construct a Gray
code
for any number
is to find out whether
section
this
of objects. The objects themselves
play no part in the problem; we care only about their
exactly
changing
number.
the relationship between the names
is by using graphs.
two
vertices
of
the
and
names
are
connectedif they
graph,
correspond
A Gray code corresponds to a cycle containing
all the vertices.
differ by only one bit.
What
We start by trying small values of n. The casesof n = 1 and n = 2 are trivial.
=
3. If
3? It is not hard to see that it is impossible to find a Gray code of length
about n
we either get the same
we start with any string and change one bit twice,
string or another
with
a two-bit difference; we cannot get the same
string after three changes. In
string
is
that
it
to
construct
a Gray code of any odd
this observation
fact,
impossible
implies
is
a?
=4?
The
a
code
of
4: 00, 01, 11, 10. The
What
about
following
Gray
length
length.
is
course
a
We
now
for
our
first
of
are
square.
ready
corresponding
graph
attempt.
A good
visualize
to
way
The names
to the
2.10
\342\226\241
Theorem
code
The proof is
by
of length
induction
Ojj, ljj,
the
Although
of
length
is
it
size
each
possible
n with
fewer
in
than
n/2
any
on k.
The case
2(k +
consider
objects
bits?
of
1). Let
k.
= 1 is trivial.
Assume
s \\ ,j2\302\273\342\200\224\302\273^2it
correspond
that
to
\\s2,
0s2,
0s3,
0j4,
..., 0s2k-
with
|~log2rtl bits.
is not
number
I-,
very
of
satisfactory. The
objects. In general,
To achieve a logarithmic
of objects is doubled. Let's assume
we know how to
that
construct Gray codes for all even numbers 2k, such that k < n. Given 2n objects, we try
to construct
the code from two smaller
codes each of size n.
We
run into a problem. Although
is a Gray
there
2n is even, and thus
immediately
code of that size, n may be odd, and there is no odd-sizeGray code.
we
Consequently,
one
bit
whenever
the number
integer
positive
proof
the
string
to represent
2k for
to the
case where
2.9
Codes
Gray
21
05,
IS,
o_
052
lj2
2.3
Figure
able
be
not
may
to values
ourselves
code of
a Gray
Constructing
whenever
that
assume
size Ik.
n is
odd. Let's
we know
restrict
to construct
how
(we
will
see
later
how
Let j
n.
consider
!,l5,, lj\342\200\236/2,
l^\342\200\236/2-l ls2l52,052,053,...,0^/2,05
For example,
extendthe
we can
The
two
with
sequences
are
000,
Gray
code
001, 011,
for n =4 to
010, and
100,
110,111.We
sequenceis 101,001,011,010,000,100,
string
will
be
bit
than
we used
for
the
Gray
code
101,
Hence,
the
log2w.
Figure
2.4
Constructing
a Gray
code from
two
smaller
ones.
as follows.
The combined
a Gray
constructed
for nil.
n =8
111, 110.
code for
length
of each
How do
Looking
sufficient
exactly
two
be
for
the general
it may
to
construction
this
extend
we
even
any
value
Recall
of nl
that
the
with constructing odd Gray codes was that it was impossible to close the cycle.
it
back at Fig. 2.4, we can seethat it is not necessary to have two closed
cycles;
we
construct
If
can
an
code
two
have
to
open
Gray
open sequences.
problem
is
Induction
Mathematical
22
sufficient
than
more
by
We now
construction.
have
one bit) of
two cases.
odd length,
then
2.11
Theorem
\342\226\241
There exist Gray codes of length \\\\og2k] for any positive integer k. The
Gray codesfor the even values of k are closed,and the Gray codes for odd
values ofk are open.
Proof: We
both
prove
cases
one
with
induction
stronger
exist
Gray
the code
hypothesis.
codes of length
\\\\og2k\\
is closed: ifk is odd, then
all
for
code
the
is open.
The base of
the
We
is trivial.
induction
a Gray
construct
now
code of
size n.
are
There
two cases:
1.
a?
is even:
The reduction
a power
was
of 2.
By
is similar to
case
this
in
induction
the
the
reduction
there exists
hypothesis,
for the
a Gray
case where
code
of length
one with
2.
riog2(2*)l
= log2(2*),
and
+ 1)].
1 =[log2(2\302\243
log2(2*)+
\342\226\2
thing to
and
to include
both
climbing
benefits
induction
two
with
from
the
the steps
problem
in one
induction
induction
not
by
the
such that
it
covers
included
the
plan
taken
merely
hypothesis.
hypothesis
We do
feet.
hypothesis
generalized
them
of
other
concerning
steps of
foot.
a more
two
cases.
It
is
solution
It
of
is much
better
general problem.
In the next
section,
In
this
to define
example,
we present
one
like
foot
the
the
an
2.10
Figure 2.5
example where
more
with
the
that
tradeoff.
for n
statement
But,
the proof.
the
section
next
values
less than
2.10
the
The advantage to
hypothesis is stronger and can
We need to prove the induction
Paths in
a Graph
23
Gray code.
open
an extended
by solving
problem
structures.
general
induction
obvious
an
Constructing
to solve
is easier
it
Finding Edge-Disjoint
on
working
a more
problem dealing
general problem is
Paths in a
Edge-Disjoint
Finding
be used
Graph
be a
E)
G=(V,
disjoint, if
do not
2.12
\342\226\241
Theorem
Let G =(V,
vertices
find
E) be a
odd
with
paths
edge-disjoint
The proof is
Proof:
true for m =
undirected
connected
degrees.
by
We can
vertices
connecting
induction
graph,
divide
on the
the
in
and let
vertices
each
number of
O be
in O
the
set
into pairs
pair.
edges. The
theorem
1.
Induction
graphs
hypothesis:
with
< m
edges.
The theorem
is
true
for
of
and
all connected
undirected
is clearly
Mathematical Induction
24
m edges,
and let O be the set of odd
undirected
G with
connected
graph
is
true.
then
the
theorem
take any two
is
O
If
Otherwise,
trivially
empty,
vertices.
there
is
a
them.
is
Remove
the whole
G
Since
O.
in
connected,
path connecting
vertices
We
would
induction
has
fewer
like
to
use
the
The
edges.
graph
remaining
path from G.
odd
the
rest
of
the
and
to
the
for
to find the paths
vertices,
proof.
complete
hypothesis,
the path, we may
the
have
disconnected
however, is that, by removing
The problem,
to
connected
to be very
We
have
induction
The
only
hypothesis
applied
graphs.
graph.
Considera
careful
about
this case
in
now
We
way
ingenious
The problem
it.
induction
the
using
an
graphs having
< m
with
induction
following
induction
Revised
was
encountered
we
have the
hypothesis correctly. We
change the hypothesis
\342\200\224
we will
the
can
and
avoid
adapt
requirement.
connectivity
this
in
difficulty
needs!
Let's remove
it to our
hypothesis:
hypothesis: The
theorem
is
true
for
all undirected
edges.
is simpler.
Consider
This is obviously a stronger theorem.Its proof, on the other hand,
not be
m
and
as
before.
The
with
O
undirected
an
may
edges,
graph
graph
again
connected components.
We
into several
connected. In this case, the graph is partitioned
is
Since
each
a
will
two odd vertices from the
same
take
component.
component
of
odd
vertices.
if
it
must
have
an
even
number
there
connected
Hence,
by itself,
graph
we can find two of them in the same component.
So now we are
are any odd vertices,
we can
vertices are in the same component,
basically done. Since the two chosen
connect them by a path. We then remove the path. The graph has now less than m
because
it does not require
edges, and we can use the induction
connectivity.
hypothesis
in
we
can
the
odd
vertices
We can
Thus, in the remaining
pair
paths.
graph,
edge disjoint
then add the path we removed and complete the proof.
We actually proved a stronger theorem
than
the one we sought! We proved that
the connectivity
is unnecessary. And the proof was easier.
\342\226\2
requirement
Comments
the
induction
is very
Arithmetic
next
geometric
an
of a very
example
It is
similar in
some
important.
2.11
The
hypothesis.
versus
Geometric
example
Mean Theorem
to Cauchy,
of the arithmetic
nonstandard use of induction,
versus
which
we
2.11
2.13
Theorem
\342\226\241
*-
(jr,jr2
values of n to one
principle:
If a statement
truth
values of
for larger
is true
for an
n implies
for
n-\\,
for
fact
an assumption
for smaller
the following reversed induction
extending
use
subset
infinite
truth
its
n,
the
then
we
is identical to
hypothesis
that the induction
proceeds
induction
from
case and
a base
proving
(2-3)
\302\253.
The
proof comes
of the
part
of
Instead
backward.
on
induction
by
then
-\342\226\240
The proof is
its
numbers,
\342\200\242\342\200\242\342\200\242*\342\200\236)\"
Proof:
25
Theorem
Mean
Geometric
versus
Arithmetic
of the
then P
is
true
natural
all
for
numbers.
holds because
This principle
that for
guarantees
We
to
will
x\\
steps.
step, we
The claim
of n that
becomes
statement
holds for
a greaternumber
from
In the
of n that are
values
second
the
In
=2.
Considers
for
only
two
in
Consider
n.
all
\\.
need.
set we
infinite
k, there is
number
the theorem
prove
the theorem
prove
for
n=
every
the reversed
use
then
the
that
fact
the
natural
an
set
infinite
the set; we
can
k.
m to
first
m in
step,
we use
powers of 2.
The
regular
powers
use reversedinduction
to prove
are powers of 2. The theorem
induction
of 2
is
the
the theorem
is trivial
for
+*2
^[x^x~2~<
now
that (2.3) is
we can verify
by squaring both sides. Assume
easily
=2*+l.
We
rewrite
the
left-hand
side
of
and consider 2n
(2.3) as follows:
which
n =2*,
/I
JL
\"
(x\\x2\"
We
^2
can
'x2n)
x2n)1'\"-
for
theorem
the
use
'''
The
n=2
with
y{
(2.4) becomes
expression
+:v2
y\\
\"-x2n)2n
(x{x2
But, by the
induction
hypothesis
*1
y\\
and the
'''
+*2 +
2
for
+xn
a?,
we
\"
have
xn + \\ +xn+2
+^2
=^y~2<
+ x2n
for
(2.4)
+ \\xn+2\"'x2n)n-
=y(x\\x2'''xn)n(xn
now
(x,,+\\xn+2
JL
true
= (x{x2
'''xn)xin,
and
Induction
Mathematical
26
We are now
that (2.3) is
Assume
Z =
That
the theorem
prove
- 1.
consider
for
all
n.
Define
be
for
true
\"
in
it is
particular,
\"
chosen
so,
numbers,
n positive
any
true
is,
Ul*2-\"*i.-l*)\"
was
to
induction
and
a?,
n-\\
forjC!,Jc2,..;X\342\200\236-\\*z-
we
an arbitrary
for
(z
use reversed
to
ready
true
of this
side
right-hand
Z-
expression.) Hence,
have
<z,
(xxx2'-xn_xz)n
-'X\342\200\236-XZ<Zn,
and
l
(xXX2'\"Xn.x)n
the same
is exactly
which
2.12
n-\\
as (2.3) for
Invariants:
Loop
<Z =
- 1.
\342\226\
Number
a Decimal
Converting
to Binary
a loop
result
of
the
relationships
algorithm
in
Fig.
represented by
the
Algorithm
variables
the
array
ignoring
is
b (which
Convert_to_Binary
otherwise). The
the
The
during
third
is an
of
statement
initially
induction
the
hypothesis
Such
the
use of
decimal number
loop
into
reflect
When
ofn
invariants
a binary
the
array
b.
with
the
number
The first
consists of one loop with three statements.
index to the array b. The second statement computes
1 if t is odd, and 0
the division of t by 2 (namely,
division (namely,
divides t by 2, using
an
integer
Binary
ConvertJo
in
the
zero).
2.14
Algorithm
is stored
the
an induction
fractions).
\342\226\241
Theorem
that the
on
induction
should
execution.
loop
illustrate
use
that
program
to prove
want
We
We can
result.
intended
a loop invariant. We
converts a
2.6, which
called
is
hypothesis
indeed
is executed.
loop
between
a certain value.
to compute
the loop is
executing
Consider a
of algorithms.
correctness
proving
is supposed
that
of times
number
for
useful
is very
Induction
contains
terminates,
the
binary
representation
Convert
Algorithm
Input:
Output:
Jo _Binary
(a positive
integer).
b (an
of bits
Converting a Decimal
Loop Invariants:
2.12
array
27
to Binary
Number
(n);
to
corresponding
the
binary
of n).
representation
begin
t
{ we use a new
n ;
:=
t to
variable
preserve
0 ;
k :=
0 do
t>
while
+ 1;
k:=k
:=tmod2
b[k]
t :=tdiv2
end
Figure 2.6 Algorithm
Proof: The
proof is by
Convert
Jo _Binary.
The induction
Induction
be
not have to
If m is
hypothesis:
on k,
induction
does
hypothesis
the
the
number
the
same
integer
of
times
as the
represented
by the
binary array
b[\\..k],then n=t-2k+m.
The
algorithm.
number
k of
t,
The
shifted
the
the
loop
binary
by k,
To prove the
heart
states
hypothesis
of times
the loop,
when
is the
t-2k+m
expression
is executed.
array
represents
corresponds to the
the
of
that
the
of
the
of
the
value
It captures
the
idea
behind
bits
the algorithm.
of
a?,
and
At
step
the value
of
(1)
algorithm, we have to prove three conditions:
at step k
of the loop, (2) the truth of the hypothesis
implies its truth for step k + 1, and (3) when the loop terminates, the hypothesis
implies
the correctness of the algorithm.
At the beginning of the loop, k = 0, m = 0 (by definition,
since the array is empty), and n=t.
2k +m at the start
of the \302\243th
Assume
that n =t \342\200\242
loop,
and consider the corresponding values at the end of the kth loop.
There
are two cases.
assume
that t is even at the start of the kih loop.
In this case, t mod 2 is 0. Thus,
First,
there is no contribution
to the array (namely, m is unchanged), / is divided by 2, and k is
is still true. Second, assume that
incremented. Hence, the hypothesis
m is odd.
In this
contributes
2k
which
b
+
to
/
is
set
to
to
is
w,
case, [k 1]
1,
(/- l)/2, and k is
changed
the
incremented.
So, at the end of the &th loop,
corresponding
expression is
=
=
+
/w+2*
is
which
+w+2*
t-2k+m=n,
(r-l)-2*
(f-l)/2-2A+I
exactly what we
the hypothesis is
true
correctness
at the
of this
beginning
Mathematical
28
Induction
the
loop
terminates
when
= 0,
which implies,
by
the
hypothesis,
\342\226\
thatA?=0-2*+w=w.
fall
If
finish
We
Errors
Common
2.13
this chapter
with
a few
warnings
into
by using induction
in
one believes strongly
trivial
following
hypothesis
assumptions.
way
around.
Another
dangerous trap
of the form
involves exceptionsto
the
theorem.
It is
common to have
The induction
30.\"
or \"a? is not a prime less than
exceptions
for a? =2 from the hypothesis for
principle depends on the ability to imply the hypothesis
n = l, the
hypothesis for a? =3 from the hypothesis for a? =2, and so on. If even one of
these
of this trap. The first
two examples
steps fails, the whole proof fails. We present
the second example is a more serious one.
anecdote;
example is a simple amusing
minor
Consider
the
following
n > 3,
claim.
lines
in the
have
one point
plane, no
in
two
common.
of
which
are parallel
2.14
29
Summary
is obviously true
This claim is clearlywrong,
but let's look at a ''proof\" of it. The claim
more careful and consider two
the claim is still
for one line. Let's even be a little
lines;
Assume
that the claim is true
n lines, and consider n +1
for
true.
lines.
By the
n
lines
the
first
in
common.
have
a
also
the
the last
But,
by
hypothesis,
point
hypothesis,
+
in
the
have
a
common.
The
common
n lines
(n l)th line)
point
(including
point of the
n
the
n
lines
lines and
last
must be common to all n + 1 lines, because lines having
first
in common are equal. But,
in that case, the (n + l)th
line passes
two
points
through the
same point, and the claim is proven.
is wrong
with this proof? Actually
What
The only wrong
little.
is that
very
step
the
fact
n
the proof unintentionally
that
must
(or in this case very intentionally)
ignores
to work.
That is, the claim is true for n - 1, n = 2, and also,
be at least 3 for the argument
if
true for
it is
from
step
a?
a?
to
=2
claim
in this
miss.
Let's look
/? =
(The
\\
+ (/?
the
that
another
example
following
claim:
- 1) \\
goes to
expression
show
at
the
Consider
expression
correct).
expression
n+
1. If we squareboth
=
+ n
Rearranging
that is
Vl +
(a?
+2)
proof of
is a
converges
If n = 1, then (2.5)
a?,
that
so
becomes
+Wl
+(/i-l)\\l
\342\200\242
\342\200\242
\342\200\242.
The
5,
is
problem
only
the
the
to
so obvious
not
for all
sides
+ 1 = 4,
+ 1)VT+
(a?
Here
infinity.)
in parenthesis
the
n2
wrong.
very
true for
it is
This
=3.
a?
case,
\342\200\242
\342\200\242
then
\342\200\242,
4,
=3,
\342\200\242
\342\200\242
\342\200\242.
(2.5) by
induction.
the claim is
1 = \\1
+0(
for
omit
We
meaningful.
\342\226\240
\342\200\242
which
\342\200\242),
is correct
(2.5)
we have to
First,
is true
a?,
(since
consider
and
+ (/?+2)---.
l)Vl
terms, we get
-5-^i = n
A7-1
+ 1 =
\\1 + n
Vl + (a?
+ 1 Wl
+2)
(a?
^,
is exactly
Or is it? The only
(2.5) for n + 1. The proof is now complete.
wrong
was
n
1
this
that
is
not
0. But, n - 1 =0
when
without
value
step
dividing by
verifying
n = 1, which
is the first step in the induction!
works
one
for
except
Again, everything
which
\342\200\224
the one
implication
whole
In this
proof.
2.14
case, by
the
way,
= 1 to a? =2
\342\200\224
and
the claim is
this is
correct, but
the
enough to invalidate
proof is not that
the
easy.
Summary
Mathematical induction
and explored
hypothesis.
is a
some of
the
We have to
cases, there is
only
one
rich technique. We
for using it.
methods
decide to
parameter,
which
and the
parameter
The
variations
step is to define
we apply the induction.
first
choice is clear.
In
other
cases,
of induction,
the
induction
In
many
however,
we
30
MathematicalInduction
have
a fair
introduced
always
amount of flexibility.
especially for the proof.
the
from
directly
follow
defined
one,
may be even a newly
not
does
seen, the induction
hypothesis
in
theorem statement. Sometimes,we apply
induction
The
parameter
As we have
to the
such that
introduced
in this
chapter
and
Notes
of
discovery
closer
a stronger
it
Bibliographic
The
us
the
several
algorithm
design techniques.
and Further
mathematical
induction
Reading
principle
is
attributed
to
the
Italian
induction
to note that a
mathematician
is described
in
Bussey
The
[1917]
of mathematical
31
Exercises
Lovasz
[1979].
found
be
introduced
(see,
Cauchy
[1953].
Gray
by
The
[1986].
Hamming
theorem is due to
for
versus
[1972]
invariants
can
geometric
or Beckenbach
Bellman
us by
codes were
Gray
in
7.
More on
mean
and
loop
to
Darrah Chavey.
induction can
mathematical
on
material
Further
course,
be
found
be found
can
examples
in
Polya's
in
Sominskii
wonderful
[1963],
this book.
throughout
Exercises
2.1
2.2
\342\200\242
**
a, +tf2+
2.3
Find
all natural
to
general
prove
numbers jc, y
(x *y),
sums.
arithmetic
and n.
That is,
sum
the
find
are constants.
andc,,c2
c1/i+c,2,
sum and
2.2
for
=
#\342\200\236
where
+an,
1-2 + 2-3+
2.4
in Section
solution
the
Extend
is divisible byx-y
x\" -y\"
that
Prove
your claim:
\342\200\242\342\200\242\342\200\242
+
l).
+/i(rt
the following
claim:
111
1
\342\200\224
+ \342\200\242\342\200\242\342\200\242
+ \342\200\224
+ \342\200\224
+ \342\200\224
2
2.5
Find the
2.6
Prove
sum of the
of the
squares
+ 32 -42
a set of n
Given
are two
2.8
2\"
first
n natural
claim.
that
l2 -22
2.7
\342\200\242
\342\200\242
\342\200\242
+
(-I)*\"1*2
numbers
numbers in
Let a, b, and
n be
the set,
= (-l)*-'*(*
out of the
one of which
first
In
+ l)/2.
numbers l,2,...,2/i, prove
natural
that
there
the other.
divides
V-\\an+bn)>{a+b)n.
2.9
2.10
Prove
by
induction
only
if the
Find
an
sum of
representation,
is divisible
by 3 if
and
sum of the ith row of the following triangle, which is called the
The sides of the triangle
are Is,
prove the correctness of your claim.
is the sum of the two entries directly above it.
for the
expression
Pascal triangle,
and each other
its
and
entry
1
1
1
13
14
32
2.11
Induction
Mathematical
an
for the
correctness
expression
of your
above it
(a nonexisting
Find
claim.
sum of
Each
/th
the
in the
entry
12
2.12
for all
that,
Prove
'
_L_
+...+_L
for all n
i +
can
k_
2.15 Considerthe
first
as a sum
be written
10
16
19 16
10 4
'
is an
even number.
series,
following
series,
13
2.14 Considerthe
In
3
k is an
where
> 1,
i+...+i..
> 1,
n+2
n+\\
5 terms
of distinct
starts as
that
an
arithmetic
any positive
integer
starts as an arithmetic
series, 1, 2, 3, 6, 12,24, 54, 84, 114,..., which
terms it becomes a geometric series, and then, after 3 more terms, it
If it
becomes an arithmetic series again.
Does your proof of Exercise 2.14 fit this problem?
be written
as a sum of distinct
does, find the error in it since, for example, 81 cannot
numbers
this series. What
is the subtle point in the proof of Exercise 2.14?
series, after
2.16 Considern
following
first 3
the
lines in
>3
general
position
plane. Prove
in the
that
one of
at least
the
regions
2.17
Considern
n
2.18
lines
>3
- 2
triangles.
a set of n
Given
2.19
that
2.20
any
Prove
can
2.21
that
Prove
position
the regions
neighboring
in the
that
plane.
Prove
these
any
in a
that
lines
contained
two
form
at least
in a
unit-size
colors
such
be colored
2.22
in general
regions formed
by a
that
planar map
no two
all
of whose
neighboring regions
be
Prove
common.
33
Exercises
Figure 2.7
*
Circleswith
one
chord.
2.24
We
encoding for
any
even
value
Figure
2.8
the
we
following
can try
of objects
such
A simple polygon
way. Instead of
to
that
maximize
each
on the
it.
minimizing
Is it
possible to design
strings differ
two consecutive
lattice points.
the difference
an
by
34
Mathematical Induction
etc.)?If
2.25
is k is
(where
bits
it is
possible,
there is at
prove that
common,
n >2,
2.26 Let dx, d2,...,d\342\200\236,
2.27
there exists a
segment. Assume
regions formed by
2.28 Let
T = {V,
tree with
undirected
an
the following
principle
are
n boxes,
inside
principle
meet
inside the
circle,
in
simplest variation)
one box will
that
function
dn.
vertices to vertices,
maps
either (f (v),/(w)) is
either
maps an edge to an edge, or
exists either a vertex v in V such that
an edge
in \302\243,
then
(in
other words,
there is
states
at least
d2,...,
dx,
contain
A complete binary
tree of height
h
+
1 consists
tree
of
of
complete
binary
height
two complete binary
trees of height h whose
roots are connected to a new root. Let T be a
tree of height h. The height
of a node in T is h minus the node's distance
complete
binary
from the root (e.g., the root has height /?, whereas
a leaf has height
0). Prove that the sum of
the heights
of all the nodes in T is 2/,+l -h-2.
F(/i) = F(/i-1)
and
m be
n2 -m(n
(Hint: Use a
either
forward
number,
the induction
Strengthen
integers such
\\)
+ 2n
\"two
that
+ m2 <
n2
or
in the
\\<m<n.
defined
is
which
for
n>2.
Prove
hypothesis by
section on Gray
Prove
by
inductively as follows:
that F(n)2 + F(/i + l)2
proving
induction
two seemingly
separate
codes.)
that
+ n.
on m. Prove two
backward from m = n.)
induction
sided\"
from m =
+F(/i-2),
at the
theorems
Let
(Hint:
as follows.
inductively
is the root.
Fibonacci
nth
the
= F(2)=1.
+ 1).
is defined
which
node
be
Let F(n)
F(2n
2.33
(in its
then
tree
binary
of
consists
F(1)
2.32
3,
induction.
by
A complete
2.31
k-
connect
2.30
(or k -2,
bits
if
n vertices
an
put
that
line segments
these
E) be
satisfies
which
one vertex
n points
Put
How about
string)?
construction.
T such
least
be positive
each
'\342\200\242\342\200\242
+d\342\200\236=2n-2,
di+d2+
then
in
an efficient
find
T and k subtrees of
a tree
Given
of bits
number
the
base
cases,
m =
and
m =/i, and
go
Let
graph is an edge whose removal disconnects the graph.
be a connected
undirected
G has the
graph without a bridge. Prove that
\"ear decomposition\" (see Fig. 2.9). The edges of G can be partitioned
into
such that \302\243,
sets
is a cycle, and, for each
is a path
...,\302\243*\302\273
z, 1 <i<k,
disjoint
\302\243|,\302\2432,
\302\243,
whose endpoints are vertices that already
appear in a previous \302\243,,
j < /, and its other
vertices (if any) have not appeared in previous
path may be a closed one, in
\302\243,s.(The
which
case it includes only one previous vertex.)
A
bridge
G=(V,E)
following
in an undirected
Exercises
denote
2.34 Let K\342\200\236
the complete
2.35 Given
common.
a graph
In
with
decomposition.
undirected graph
with
number.
Prove
(A spanning
tree
even
an undirected
vertex in
An ear
2.9
Figure
that
is a
matching
is one in
and n2 edges such
vertices
35
which
that
matching.
2.36
Let a,,
the
a2,..., an
geometric
(Hint:
numbers
that
- \342\200\242
= \\. Prove,
a\342\200\236
a2
without
using
that
\342\200\242\342\200\242\342\200\242
(l+a\342\200\236)>2\\
(l+fli)(l+fl2)
2.37
be positive
versus
arithmetic
another
variable
that
two specially
replaces
chosen
Considerthe
recurrence
this
for Fibonacci
relation
recurrence,
G(/i + l):
G(/i
What
= G(/i) + G(/i-1)+1
+ l)
is wrong
= F(/i)-1+F(/i-1)-1
+1
= F(/i
+ l)-l.
is another
versus
The
mean inequality.
proof of the arithmetic
geometric
which
makes it incomplete in general.
Describe this weakness
proof has a major weakness,
and
then define the restrictions on the theorem that are needed to make this proof correct.
Let S =xt
numbers
\342\200\242
\342\200\242 To find
+x\342\200\236.
+x2+ \342\200\242
whose sum is
5 and
whose
a contradiction
geometric
to
mean is
the
larger
theorem,
than
we need to exhibit
n
Sin. It makes sense to
36
Induction
Mathematical
look for
a set of
In other
words, we
because
I
Xi +X>>
X\\X2<
with equality holding only if jc, =x2. If all the numbers are
this is a contradiction to the maximality
assumption
Otherwise,
2.39
2.40
equal,
then
of the
the theorem
Prove the
ConvertJo _Binary
number.
correctness
holds.
set.
algorithm
is an array
algorithm
a number given in
(see Fig. 2.6) such that it converts
The input is an array of base-6 digits, and the output is an array
of your algorithm by using a loop invariant.
CHAPTER
ANALYSIS
OF
It
ALGORITHMS
not depend
does
on size,or a cow
Proverb
German
Pennsylvania
rabbit.
catch a
would
Introduction
3.1
The purpose
of algorithm
of an algorithm
without
It
so
are
clear.
doing
it is
are too
characteristics of the
most
the
implementation
On
perfect.
algorithms
the
the
behavior,
computer
underlying
factors.
influencing
attach
predict the
Instead, we
every
a
Furthermore,
algorithms.
for each
of
the
It would
to
extract
be
an
the main
We define
algorithm.
Most
algorithm.
to determine
the
the efficiency
system changes.
alternatives
to
test
and
''small\"
different
impossible
usually
many
important
and
time,
the running
especially
the algorithm
implement
for
the
mileage claims,
running
the
work
Unfortunately,
There
algorithm.
about
to
than
in
complicated
the
The
it on a specific computer.
advantages
implementing
more convenient to have simple measures for
is much
of an algorithm
time a certain parameter
too much
is to predict
analysis
efficiency
are
Rufus
Curtius
Quintus
important,
using
our
only
an
approximation',
can yield
analysis, we
We can
purposes.
this
significant
it
not
is
information
car
\342\200\224
your
ti
38
of
Analysis
In this
times
running
this
algorithm as the
goes to
algorithm consists of
steps,
predicting
The
algorithms.
on
concentrate
For
infinity.
\\00n
for
and
factors
constant
of the input
size
if the
and
\302\253,
we ignore
is that
approach
size
Algorithms
example,
the approximate
main feature of
we ignore the
will introduce precise
then
of the
behavior
the
is
if the input
an
array
of
100 and
constant
that
however,
for all n
running
n is
if
>50. On the
100\302\2531,8.
however,
than
2\302\2532+50.
their
will
times.
running
most
well
works
The result
even
Thus,
is sufficient
behavior
time
slower
than the
though
in practice.
as a first
approximation
In
the
approach
asymptotic
in
the
cases,
be smaller
to
expression
can be
looking at only
and indication of efficiency.
most
one
algorithm was
than n2. In this
algorithms
first
of the first
other
hand, suppose
running
first algorithm seems better, since nx% is smaller
have
to be approximately 300,000,000 for 100\302\25318
Fortunately,
sometimes,it
is indeed
that the
the
Again,
case,
The secondalgorithm
large enough.
the
of
misleading
asymptotic
expected to run
the
for
of
definition
our
of
analysis
a particular
size
will
Unless specifiedotherwise,
Given
the running
time\"
for all
a problem
time of the
be
will
given
be
the
We
straightforward.
size will be denoted by
will
see
some
examples
shortly.
n.
of size, we want
to find an expression that
gives
'
of
relative
to the size. (The precisedefinition
algorithm
'running
in Section 3.3.) As we said earlier, there is usually
not just one value
and
a definition
of the same
all inputs
Consequently, we must choose, among
to use as our indicator.
The most common choice is the worstcase input. This may seem peculiar.
or the average input?
not use the best input,
Why
The best input is usually ruled out because, in most cases, it is not representative;
there is usually an input
for which the problem is trivial.
The
average-case
input may be
a goodchoice,but it is sometimes very hard to measure effectively. First, it is generally
not clear what
an \"average\"
over
input is. We can average
many different parameters in
not
If
we
are
different
the
cases
contain many
that
can
careful,
many
ways.
average
never occur in practice,
thus making this measure irrelevant.
serious
Another
problem
case is the mathematical
with
the average
in analyzing average-case
taking
difficulty
performance. We are still very far from having
easy-to-use
relatively
comprehensive,
size, the
inputs
input
of equal
size.
we want
3.2 The
O Notation
39
We
will
discuss
average-case
analysis for few
resort
worst-case
to
the
worst input as
mainly
analysis. Choosing
problems,
indicator
turns out to be very
useful.
In some cases, the worst
is
an
input
very close to
and
to
observations.
In
other
even
the
cases,
average
input
experimental
though the
is
from
the
different
the
that
worst
achieves
input
substantially
average input,
algorithm
the best performancefor the worst input also performs very well for all cases. Unless
we will use worst-case analysis throughout
this book.
specifiedotherwise,
In summary, both
and worst-case
asymptotic
analysis
analysis are only
for average-case
techniques
do not
definitely
They
running time
of the
approximations
analysis.
will
we
but
give
the
of a
whole
story.
particular
a particular
under
algorithm
good
very
input.
indicators
in
most cases.
The O Notation
3.2
to
said, our approach will be to ignore constant factors when
trying
time
of a particular algorithm.
To do that effectively
we need
We
is 0(f(n))
for another
function
g(n)
say that a function
f(n)
special notation.
if
or
sometimes
there
exist
constants
c and /V,
\"Oh,\"
Oh,\"
\"Big
of/(\302\253)),
(pronounced
In other
for large enough
such
words,
a?, the
that, for all n >N, we have g(n)<cf(n).
is
more
than
a
constant
times
the
The
function
no
function
g (n) may
g (n)
function/ (n).
it only from above. For
be less than cf (\302\253),
even
bounds
less; the O notation
substantially
= 0{n2),
At
since 5n2 + \\5<6n2 for
n>4.
the same time,
+15
example, 5\302\2532
5n2 + 15= 0(\302\2533), since 5n2 + 15<n3 for all n >6.
we
can
The O notation allows us to ignore constants conveniently. Although
We always
write
include constants within
the O notation,
there is no reason to do that.
we write
without specifying the
0(n) instead of, say, 0(5n+4). Similarly,
0(\\ogn)
As
have
we
already
the
evaluate
running
of the
With
simple.
relatively
The
cases.
most
monotonically growing
functions
rule
if
the
simple rules, we can cover
We say
is the following:
n j >n2
that/(a?
implies
of (but
majority
that
function
not
f(n)
all)
is
,)>/(\302\2532)-
3.1
\342\226\241
Theorem
For all
some
useful
constants c > 0
and
>
1, and
for all
growing
monotonically
f(n),
(f(n)Y=0(a^n)).
In
other
polynomial
words,
function.
an
exponential
function
grows
faster
than
does a
\342\226\2
40
of Algorithms
Analysis
to compare many
functions.
for
all
constants
that,
get
be used
3.1, we
For
example,
a > 1,
we
if
n for
substitute
c > 0 and
(3.1)
n(=0(a\.")
comes
example
Another
from substituting
for f(n).
log\342\200\236A?
c>0
and
a>\\
= 0(n).
(\\ogany=0(aloSl\")
We
can
2) If fin)
Proof:
largest
Since
= 0(s(n))andg(n)= 0(r(n))then
= 0(s(n))
and g(n) = 0(r(n)) then
By
of N,
the
N2
O notation
divide.
or
subtract
and
can
are
there
definition,
for a?>/V,,
f(n)<C\\s(n)
3.1,
importance
which contains
algorithms
using
the
rules.
following
consume
constants
c\\,
corresponds to
That
of
+ g(n) = 0(s(n) +
= 0(s(n)r(n)).
fin)g(n)
fin)
N\\,
The
c2,
largest
r(n)).
such
and
N2,
of c,
that
be used
is,
it
is
that
g(n) = 0(r(n)) imply
f(n)-g(n)
I
Exercises
3.15and
0(s(n)
r{n)) (see
The
O notation
the
with
3.2
Lemma
\342\226\241
/. If fin)
(3.2)
concentrating
several
typical
for a problem
the
\342\226\2
to
\"<\"
not
it is not
however,
relation,
possible
that
true in general
f(n) = 0(s(n))
= 0(s(n)-r(n))
or that f(n)/g(n)
and
=
3.16).
the asymptotic
behavior is
times
and the time
running
on
of size n
= 1000
for different
illustrated
the
in Table
corresponding
differ by a constant
1000
of 2 from column to column, from
steps per second to
8000 stepsper second.We can clearly see the improvements we gain by speeding up the
we gain by
computer (or the algorithm)
by a constant factor versus the improvements
to
a
faster
An
the
changing
table).
(i.e., going up
exponential
asymptotic
algorithm
speeds
of years) to handle
astronomical
time (billions and
billions
the
base is very close to 1).
(unless
The O notation
is used to denote upper bounds
on the running times of algorithms;
All the algorithms in this book, for
however,
using only upper bounds is not sufficient.
more than
not require
times
of 0(2\.")
That is, they
do
example, have running
time.
of these
for
most
bound
crude
exponential
However,
0(2\") is a very
upper
\342\200\224
are
in
not
much
interested
faster
than
that.
We
are
algorithms
they
upper bounds,
only
but
also
in an expression that
as
time
is as close to the
actual
possible. In cases
running
where it is too difficult
to find the exact expression, we would
like to find at least upper
and
lower
bounds
for it. Obtaining lower bounds
is more difficult than is obtaining
upper
bounds.
An upper bound
time of an algorithm
on the running
only that there
implies
exists some algorithm
that does not use more time
A lower bound
must
than
indicated.
a
better
achieve
that
no
can
bound
It is impossible, of
for the problem.
imply
algorithm
to
consider
all
one
one.
We need mechanisms to model
course,
possible algorithms
by
that
a
in
enables
and
us
to
bounds
way
problems algorithms
prove lowerbounds. Lower
6.4.6.
There is a similar
are discussed further in Section
to handle lower bounds
notation
algorithm
n
= 1000
will
require
time |
2000
10
32
log2\302\253
\342\200\236..5
1,000,000
n3
1.1\"
Table 3.1
0.005
0.003
0.001
0.5
0.25
0.125
(in seconds)
1.25
2.5
8
250
250,000
1039
times
Running
8000 steps/sec
16
500
500,000
1039
time
steps/sec
4000
1,000
n2
time
steps/sec
0.010
1
1 log2\302\253
time
1000steps/sec
times
running
41
125
125,000
1038
1038
under
different
(n=1000).
assumptions
while
number
logarithm
part need
by
only
The 0,
need
notation
a constant
used to
the O
prove
part
and
the
the same.
be
not
Q,
and
corresponding
we
Sometimes
\"=\".
=
(pronounced
o(g(n))
tt/(A7)islittleohofg(A?),,)if
= o.
lim^
For
g(n)
but n/\\0*o(n).
n/\\og2n=o(n),
example,
Similarly, we say
f (n)
= (o(g(n))
if
= o(f(n)).
We can
Theorem
strengthen
3.1 by
o:
little
3.3
\342\226\241
Theorem
The
that
growing
monotonically
words,
an exponential
\342\226\2
OO Symbol
The O notation
has
of course,
in
convenient
notation
that
received
reality
to forget
gives
only
a lot
constants
a first
is essential
it
serves
to remember
a useful
purpose,
that
and
the
its
O
use
Analysis of
42
has
was also
of complexity
development
efficiency.
to distinguish
important
however,
is,
are
notation
by
It
measures.
all
light on
sheds
which
theory,
book a
in this
large and
is efficient
new
in
It is
symbol.
where
case
the
not meant to be
constants
the
they are
this
make
To
practice.
where
case
the
between
prohibitively
algorithm
corresponding
introduce
are practical
that
algorithms
many
of algorithm
by the
ignored
the
the
in
instrumental
aspects
It
of
the development
prompted
many
Algorithms
small and
we
distinction,
a precisemathematically
(or at least
only to replace some prose that accompanies
whose
as
measured
times,
should
running
by the O
algorithms
accompany)
value
We
to
denote
theoretical
of
are
notation,
only.
suggest
by CO(f(n)) (pronounced
that is O (/(\302\253)), but with constants that are too large for
\"Oh Oh of/(\302\253)\") a function
should
be easy to remember since it resembles
a big
uses. (This notation
most
practical
meant
\342\200\224
it is
notation
defined
some
oo.)
The use of
not a certain
of
intention
the
reader
to tighten our
of
symbol is to stressthat
Time
3.3
How
do
need to
we
count
and Space
example, a
The
not
the
whole
of
main
the
Another
We have no
is to indicate to
purpose
goal
or
Whether
writer.
defined.
well
not
the
this
in introducing
story.
Complexity
analyze an algorithm's
the number of steps the
judgment
use\" is
concise form.
is
O notation
to the
definition.
in a
writer
the
the
left
a \"practical
to
leads
attempting
opinion
should be
CD notation
the
constant
steps, and
time
running
algorithm
each
may
without
performs.
require
We
of time.
For
amount
different
are
may
take
longer
dependency.
Instead of counting
all steps,
to us to
we focus on the one type of step that seems
be the major step. For example, if we are analyzing a sorting
then we choose
algorithm,
of sorting;
comparisons as the major step. Intuitively, comparing elements is the essence
all the rest can be regarded
that
sure
as overhead.
Of course, we have to make
indeed constitute
comparisons
constant
factors anyway, it
the
suffices
major
part of
to check that
the
algorithm.
the
number
Since
we
will
ignore
operations is
is a bound
for
and if 0(f(n))
for the total number of steps.
of
all other
the
number
3.4 Summations
spacecomplexity
43
is used
different
straightforward, and,
in
most
cases,
we
will
for the
algorithms
the input/output
case
not
cannot
we
Also,
it.
discuss
Summations
3.4
an
algorithm
of
number
total
is
operations
\342\200\242
\342\200\242
\342\200\242
+\302\253. We
1+2+
thus
denote
sums
with
the
term
/, where
sigma
sum is
above
The
notation.
goes from 1 to
As
\302\253.\"
can comparethis
observe that, by
sum
as
written
which
\302\243/,
seen in Section
case where each
we have
to the
time of the
the running
cutting
2.2,
\"sum of
means
step requires
/th
the
exactly
(n +
We
and we
l)/2.
n operations,
from n to /, we
step
save a factor
of
2.
about
3.1
\342\226\241
Example
now
Consider
other
words,
the
of executing
case
we are looking for the
a loop in
which
the /th
summation
S2(/i)=ii2.
It is
step.
are
clear that
guess
induction
that
since
S2(n)<n3,
P (n
n3 is equal to running
we can guess that
the
the
for n2
loop
differences
(since coefficientsof
^-(a\302\2533+/7\302\2532+c\302\253+^)
the
same
power of
must
operations
= \302\2532
+ 2\302\253
+ l,
be equal)
in
each
S2(n) and n3
by induction. We
and the
P(\\)=\\
between
= P(n)
+ l)3+/?(\302\253
a(\302\253
which implies
example,
that
Analysis of
44
3a
the coefficient of n2
= 1
+b-b
3a +
Algorithms
2b + c -
c =2
the
coefficient
of
the
coefficient
of 1 .
\302\253
,
and
= 1
a+b+c+d-d
that a = 1 /3, b =
These equations imply
initial condition (P (1) = 1),which
implies
all
S2(n) = T
it is
There
that by reducing
the
size
(3.3)
of the
ith
/2, we
n2 to
from
step
is another
\342\226\
at expression
to arrive
way
is a
S2(n)
of
such
(3.3).
is a
It
general
technique
we
that
third-degree polynomial,
We then arrive at the solution
then
we
can
polynomials.
involving
and
it
other
Consider
polynomials.
explicit
sum
(3.4)
S3(n)=i,i\\
We
the
3.
of about
use several
\342\200\242
gnote
to
interesting
savea factor
will
c = 1 /6.
that a+b+c+d=\\.
we get
the terms,
Again,
1/2, and
will
first
S3(n) =
write
(3.4)
in
a different
i,i3 = X(/-l
the
way:
X0' +
l)3
X(/3+3/2
summation,
in Fig.
is illustrated
l)3 =
3.1.
We
can
goes
now
3/+l).
from
equate
(3.5)
0 to
the
left
- 1 instead of
side
expand:
X/3=I(/3+3/2+3/+l).
The /
terms
canceled.We
(0+1)3
(3.6)
/ ranging from 1 to
then
write an equation
for
the
and
- 1 are
common to
the
involving
I3
23
(l+l)3
(2+1)3 +
Figure
3.1
Computing
rest of
a summation
both
sides
(/i-D3
(\302\253-l+l)3
by
shifting.
and can
of (3.6),
sides.
both
n3
be
3.4
45
Summations
n-\\
+
\342\200\2363=03
3/ +
(3i2 +
l).
n-\\
n-\\
We
know
already
difference is
n3 =
n3
the
in
l)/2,
clear that
it is
and
(the
\302\243/2=S2(n)-n2
only
Hence,
\302\253th
term).
solve for
- l)/2
(n
+ /i.
S2(n):
-3n(n-\\)/2-n=3(S2(n)-n2),
which implies
r/,
that
\302\2533
3\302\253(\302\253
-1)/2
S2(\302\253)
which
= n(n-
J)/
3(S2(n)-n2) + 3/i
now
can
We
that
-\302\253
2
+ \"2 =
is, of
The main
example) in two
same
\302\2533
3\302\2532
+
\302\253
-T
/i(/i
as (3.3).
was to use a
+ 1)(2/i
+ 1)
\342\200\242
expression
this derivation
sum
in this
(S3(n)
each other. Many
other
the difference between a sum
exhibit the same behavior. If we consider
sums
'' '
f\\ +/2 +
+//i and a shifted sum f2+fi+ '' ' +/\342\200\236+i,we see that most of the
terms are left. We presentthree
cancel each other. Only the boundary
more
coefficients
of this technique.
examples
different
We want to
compute the
/r(/i) =
\302\2432'\"
would like to
and by canceling
2F(\302\253)
= 2
+ 4+
1+2
an
particular
cancel
to
another
whole
terms
F(n) by shifting
consecutive terms in F(n) is a
will allow
us to shift):
by 2 (which
expression
difference
The
expression
involving
between
\342\200\242\342\200\242\342\200\242
+2,?+2\"+l.
= 2\"+l
implies that
involving
F(n)
F(n):
-1.
= 2n+[
- 1.
\342\226\
3.3
\342\226\241
Example
now
the
following
slightly
mostly
+2\".
the
expression
2F(n)-F(n)
Consider
they
\342\200\242\342\200\242\342\200\242
compare F(n)
most of them.
+ 4 + 8+
now get
But, this
that
sum:
following
factor of 2, so let'smultiply
We can
such
ways,
3.2
\342\226\241
Example
We
in
trick
apply
the
same
technique:
more difficult
3-23+
sum:
\342\200\242\342\200\242\342\200\242
+/i-2\\
Analysis of
46
Algorithms
incremented
(we simply
By
subtracting
-(1-21
+ 1'22+
power).
= n-2n+l
= 2G(n)-G(n)
Finally, we
of heapsort:
-(2\"+I -2)
consider the
G(\302\253)
can apply
sum,
following
+2-2\"-2
\302\243i2#\"\"=l-2#|-1
the
\342\200\242\342\200\242\342\200\242
+1-2\
which
will
+ 3-2',-3+
in Section
appear
6.4.5
in
the
analysis
\342\200\242\342\200\242\342\200\242
+/i-2\302\260.
\342\200\242\342\200\242\342\200\242
+\302\253-2!.
Probably
we eliminate
expressions,
= 2n
+ l-2\"-2 +
\\'2n-1
/ factor:
\342\200\242\342\200\242\342\200\242
+ 1-21 -/i-2\302\260.
Relations
relation
recurrence
function.
two
= 2G(n)-G(n)
Recurrence
3.5
we eliminate
expressions,
by subtracting
G(n)
two
= (n-\\)2n+l+2.
+3-2\"-2+
2G(\302\253)=l\342\200\2422',+2\342\200\2422',-,
Again,
the
3.4
Example
We
the
+ n-2n^
i factor:
= n-2n+{
\342\200\242\342\200\242\342\200\242
+ 2-23+3-24+
2G(\302\253)=l-22
is a
the most
to define
a function by
famous recurrence relation
way
an
involving
one defining the
the same
expression
is the
Fibonacci
numbers
F(n) = F(n-\\)
expression
uniquely defines the function.
value of the
function
at every
number
F (4) = F (3)+F (2)= 3, and so on. However,
We can
This
the definition,
we
convenient to have an explicit
us to compute F(n) quickly,
called solving the recurrence
recurrence.
following
would
need k-2
(or closed-form)
and
relation.
compute from
For example,
k.
if
(3.7)
we
compute
the value
of the
the
expression
F(3)= /r(2)+
to compare
this
= 2,
F(l)
function
is much
would
by
more
enable
functions.
This
is
relation
simply
in the analysis
of algorithms. We briefly
a useful technique for solving recurrence relations, and present
general
solutions of two classes
of recurrences
that are among the most common
recurrences
These recurrences will be used later in the book.
involved in analyzing
algorithms.
discuss
here
Recurrence
3.5
Relations
47
like a nonscientific
a wide class of recurrence
may seem
a solution
Guessing
well for
very
to find
is
useful
guessing
bound. Consider
are trying
but, keeping
method,
It works
relations.
the exact
not
is that
the
than
values
computing
the
of n that are
powers of 2:
7(2/i)
1.
7(2)=
+2/i-l,
<27(/i)
(3.8)
rather
than equality. This is consistentwith
bound
and with
(in the form of the O notation),
finding
only
We want to find a function
side represents the worst
case.
the fact that the right-hand
but we also want to make sure that /(/?) is not too far
/(/?) such that T(n) = 0(f(n)),
from the actual
T(n).
Given a guess for / (n), say f (n) = n2, we prove that 7(n) = O (/(/?)) by induction
on n. First, we check the base of the induction. In this case, 7(2) = 1 </ (2)= 4. We then
We
our
prove
The
wrote
modest
this
an
goal of
implies that
T(n)<n2
implies
proof is
as follows:
< 27
inequality
an upper
that T(n)<f(n)
7 (2/2)
as
recurrence
(n) +
2/2
1,
- 1,
<
7(2/2)
/ (2/2).
need
We
to prove
that
f.
(by
the
definition
of the
(by
the
induction
hypothesis)
recurrence)
<(2/2)2,
which is exactly what we wanted to prove. Thus, T(n) = 0(n2). Is n2 a good estimate
for T{n)l In the last step of the proof, 2/22 + 2/2 - 1 was substituted
by the greater An2.
two
which gives us
But there
these
between
is a substantial
2n2)
expressions,
gap (about
a hint that maybe n2 is a high estimate
for T(n).
Let's try a smaller
estimate,
say, f(n) = cn for some constant c. It is clear,
is no
than
does, since c2n=2cn, and there
however, that en grows more slowly
T(n)
room for the extra 2/2-1.
Hence, T(n) is somewhere betweenen and n2.
Let's
try
now
\\og2n. Clearly,
T(n)<n
that
T(n)<n
log2/2,
7(2/2)< 27(n)+
< 2/2
log2/2
- 1,
2/2
2/2
- 1,
(by
the
definition
of the
(by
the
induction
hypothesis)
recurrence)
< 2/2(log22/2),
which
is exactly
what we
Later,
we
wanted
will
The leeway
that this is actually
to prove.
prove
in
the
the
proof
exact
is only 1 now, so we
solution to within
a
Analysis of
48
a similar
define
can
relation
recurrence
The
We
Algorithms
The recurrence
of
of
that
monotonically
increasing
2k is
where
the
< T(n)
7(2*_l)
power
< 7(2*). We
that
proved
to assume
sufficient
2, T(n) = O
implies
c.
Hence,
cxn\\og2n.
that
T(n)
power of
is a
that
7(2*)<c2*log22*
<
c\\, which
constant
another
as (3.8).
same
the
exactly
T(n)<c2k\\og22k<c(2n)\\og2(2n)
for
for integers.)
only
it is defined for all values
same
the
first
2, (3.9) is
powers of
for values of n
are
that
way:
defined
relation
values
for
but,
\302\253,
powers of 2.
(3.9)
the
that
(Notice
of n that are
the following
7(2)=1.
+ /i-l,
r(/i)<2r(L/i/2j)
values
= O(n
are
we
when
It
is
usually
asymptotic
expression.
Let's summarize
relation. Suppose that
the
to a recurrence
proof of a solution
of the following
form:
recurrence relation
in an inductive
used
steps
a general
have
we
T(g(n)) = E(T9n)9
(3.10)
(3.11)
f(g(n))>E(f,n).
For example,
in
(2/i)(log2(2/i))
> 2(/i
than\"
We are
following.
if
we
substitute
trying
g(n)
repeated
times
several
reasonably
for
the
notation
even
through
though
the
solution
in
prove
to
try
that f
prove
intuitive,
that/(\302\253)
in /(\302\253), we
T(n).
But,
(n) = n
we
thus,
\\og2n;
had to show
'
that
the opposite \342\200\224
is, to replace
to
and
remember,
explanation
easy
more quickly than T(n) does.
grows
should
T(g(n)) = E(T, n)
\302\243(/,n).
than
(this
is
This
process
(guesses) until
the
'greater
is
the
Hence,
we get by
the recurrence
what
exactly
proof
that
may have to
of the inequality
be
tight.
Another common
guess that
An
to
is to
than.\"
\"less
with
guessed
\\og2n) + 2/i - 1.
mistake
common
we
(3.8)
mistake
is O
(f
is
to use
and
(\302\253)),
the O notation
to substitute
That is, we
However, the
the O notation is that,
using
end, we cannot ignore them
of (3.8) is O (\302\253),
solution
by
when
we try
0(f(n))
with
the
the
guessing.
for n.
for
O (n)
substituting
+ 2n
< 2T(n)
T(2n)
< O (n) + 2n
get the
we
\302\253,
- 1,
- 1,
49
Relations
Recurrence
3.5
(by
the
definition
of the
(by
the
induction
hypothesis)
recurrence)
= 0(/i),
as we
is wrong,
which
that
Let's
F{n) = F{n-\\)
is
that F(n)
Substituting
c, and
fact
the
in
correct
The
''proof.\"
to
that the
guess
namely,
approach is
is 0(f(n)),
solution
determine
by guessing.
we
Again,
(3.12)
F(2)=l.
previous values, a reasonable
it is approximately 2\".
Let's
of two
sum
the
the
want
relation
+ F{n-2), F(l)=l,
of F(n)
value
the
Since
stages of
When we
explicitly.
lies
error
The
at different
ignored)
constants
the
we guess
try
be
would
guess
F(n) = c2n.
c2\"=c2\"-1 + c2n~2.
and the left side is
impossible, since c is canceled
the
that c2n is too large, and
that
side. So we learned
constant c plays no role in the induction
step.
is clearly
This
equality
than
the right
The
next
could
attempt
Instead of guessingdifferent
its value
compute
a\" in
Substituting
through
verification.
the
will
try
base.
base and
a is a constant.
as a
a parameter
F(n)
multiplicative
a smaller
with
but
greater
always
= a\", where
to
(3.12), we get
an=an-1 +an~\\
which
that
implies
+ l.
a2=a
The
now
find
that
know
F(n)
c such
a constant
If we
more
for (3.13)
solutions
two
values
(3.13)
want
to
that
find
carefully.
combination of them
does.
c(a
{)n is
greater
Since
given
F(n), we
{ax)n
and
general
solution
both
So the
the
than
values for
will
{a2)n solve
of the
we
in particular,
and we can easily
= 1 and n = 2.
So,
the
to consider
need
the
recurrence,
recurrence
any
initial
linear
is
c,(a,r+c2(a2r.
We
and
exact
need
F (2).
to compute
a simple exerciseto
of the Fibonacci relation
It is
solution
the values of c \\
and
that
verify
is
so that
c, =
the expression
1
/\\5,
and c2
fits
the
values
of F (1)
the
of Algorithms
Analysis
50
1+
i-V?
V?
F(\302\273)=-p
^5
V5
The equation
+ 1, which
we encountered
relation (3.12), is called the characteristic
is the basis for solving
relation. The same technique
in our
= a
a2
recurrence
F{n) = b{F{n-\\) +
for a
In
and
Divide
\342\200\242\342\200\242\342\200\242
+ bkF{n-k)
b2F(n-2)+
Relations
Conquer
algorithm,
a divide-and-conquer
recursively,and a combine
that there are a subproblems,
Assume
algorithm
for someconstants
T(n) =
We
0, b,
c, and k.
The running
time
simplicity,
that n =bm,
to expand
try
+ c(n/b)k)
T(n) = a(aT(n/b2)
T(n)
if we
so
assume
that
T(\\)
= c
runs
original
problem,
original
in time
cnk,
satisfies
thus
algorithm
an integer
to get
(b is
an
integer
+ c(n/b)k)+
cnk.
= 1, we get
to nlbm
\342\200\242
\342\200\242
+ c:/!*.
\342\200\242)
would
value
different
times
+ c(n/b2)k)
+ cnk=a(a(aT(n/b3)
way
(a
is always
nib
that
(3.14) a coupleof
\342\200\242
\342\200\242
T(n) = a(a( \342\200\242
T{nlbm)^c{nlbm^)k)
Let's
solve the
subproblems
of the
each
subproblems,
(3.14)
We first
for
than 1).
In general,
smaller
aT(n/b) + cnk.
assume,
greater
to combine
used
into
algorithm is used to
each
of size Mb of the
subproblem is solved
problem.
recurrence
form
of the
recurrence
any
the
to
solution
of the
k.
constant
3.5.2
search for a
equation
change
the end
result
only
by
constant). Then,
T(n) = cam
T{n)
c^am-lbik
this
Case 1: a
this
constant
am = a
of
+cftm*,
both
fok
cam%(\342\200\224)
i
\342\200\242
a simple
is
is less
(bk/a)
/=o
/=o
In
\342\200\242\342\200\242\342\200\242
implies that
which
But,
+cam-2b2k
+cam-[bk
three
cases,
than
1, so
depending
on
whether
> bk
=nlog\"a
sides).
Thus,
T(n) = 0(n^ha).
series
is less
Therefore,
can be easily
T(n)
= 0(am).
proven
converges to a
the series
Since
by taking
\\oghn,
logarithm of
we get
base b
3.5 Recurrence
Relations
In
a=bk
2:
Case
this
of the
the factor
case,
a = bk
51
that
implies
is 1, and
series
geometric
= kand
m-0(logn).
\\ogha
thus
m). Noticethat
= 0(am
T(n)
Thus,
T(n) = 0(nk\\ogn).
Case3: a < bk
In
this
case,
for
expression
first
the
T(n)
pm
_ i
+ l
am\302\243
F -1
are summarized
cases
three
These
= 0((bk)m)
-=0(amFm)
the
in
1.
by F
(F is a
= 0((bm)k) = 0(nk).
theorem.
following
The solution
relation 7(n) = aT(nlb)+
of the recurrence
b are integer constants,a > 1,b > 2, and c and k are positive
<j
standard
constant). Since
3.4
Theorem
\342\226\241
T(n)=
use the
We
than
0(n^a)
if
0(nk\\ogn)
ifa=bk.
0(nk)
ifa<bk
cnk,
a and
where
constants,
is
a>bk
\342\226\
of Theorem
result
The
3.4 applies to
result
used
time.
is also very
many
helpful
Generalizations
in
the
of
this
design
formula
It
algorithms.
divide-and-conquer
stage, since
are
it
be
can
in the
given
exercises.
Recurrence
3.5.3
not just
function,
relation
recurrence
full-history
on a few
of them.
is one
One
that
of the
on all
depends
simplest
full-history
recurrence
of the
relations
is
r(/!) = c
a constant
c is
where
(3.15)
and
we used to
method
7(1)
history.)
We
We can solve
will
try to write
this
the
recurrence
recurrence
by using the
in such a way
T(n + \\)
If we
is given.
compute sums.
of the
most
which
+ \302\243r(i),
subtract
clearly
= c
+ ZT(i).
(3.15
) from
implies
that
same
that
of
(3.16)
(3.16), we get T(n + \\)-T(n) = T(n). So,T(n + 1) = 27(\302\253),
is true for 7(1), and, by
claim
T(n + l) = r(l)2\". (This
of Algorithms
Analysis
52
claim is
if the
induction,
This argument
7(1)= 1 and
be
should
induction
an
through
does
very
= 7(l)
7(2)
that
+
7(\302\253
For
example, set
of carelessly
example
The error results from the fact
another
at
case
but
we
it
is very
in
discuss
= 0.
7(1)
(for\302\253>2).
(3.17)
to
most of
cancel
the
T(i)
+ 1):
(n>2).
-^yX7(/)
we multiply
want
for T(n
expression
corresponding
l)-l
We
terms technique.
canceling
the
+
(\302\253
convenience,
so simple,
of quicksort which
is not
is
relation
\302\2437(1),
one
i=\\
and
shifting
l) =
case.
is very
/i-1 + -
Let's look
terms.
the
double
canceled
7(1) is not necessarily
by c. One
when a parameter (c in this case) that appears in the expression
final solution.
the
To solve this problem
we note that
correctly,
and that the proof above is correct
for all n>2.
Hence,
definition),
use the
This is
base
proof ignoring
for T(2), since
in
important. It appears
Section 6.4.4. The recurrence
7(/i) =
7(2)
We can, for
incorrect!
is
it
= 6*27(1).
the
work
not
recurrence
This
We
but
+1), since we
for T(n
+ c)2''-1.
= (7(l)
+ l)
7(\342\200\236
''clear,\"
is true
it
suspicious
not appear in
+ c* (by
does
be
may
5, and see
c =
the proof
that
then
time.)
value every
going
T{n),
for
true
(3.18)
and
\302\253,
both
sides of
(3.18) by
n +
1:
n-\\
(/i +
+2
= n(n-\\)
nT(n)
l)r(/i + l) = (/i
(n>2).
\302\2437(/)
+ l)/i+2
now
(n +
which
(3.19)
\\)T{n +1)
(3.20), and
from
=
nT(n)
(n +
(3.20)
- n
\\)n
obtain
(n
- 1)+
2T(n)
= 2n
+ 2T{n)
(n
>2),
implies that
+
r(\302\253
This
subtract
(n>2).
2;r(i)
/=
We can
(3.19)
l) =
recurrence
+
^7(\302\253)
n+\\
is easier to
-^n+\\
solve.
(n>2).
First,
we
substitute
2 for
\302\253
+ l
approximation:
+
r(\302\253
If we
l)<-^_7(\302\253)
n +1
+ 2
(n>2).
(3.21)
3.6
\302\253
+ l
7\302\273<2
= 2 1 +
=2
1+
+ \342\200\224
+
+ l)(//(\302\253 + 1)-1.5),
1+
without
n-2
(1)
for T(n)
+ y+0(\\/n),
series
has
where
is
\\ogn).
several
proof,
Harmonic
equalities
and
inequalities
that
are
algorithms.
analyzing
series
Arithmetic
1+2 + 3
will
we
is the
= O(n
we present,
n-\\
n-\\
+7- 1.5) + O
section,
in
solution
Facts
n-2
constant. Hence,the
+ l)(ln/i
< 2(/i
\342\200\242
\342\200\242
1/2+ 1/3+ \342\200\242
+ \\ln
which
n + \\
\342\200\242
\342\200\242
\342\200\242
\342\226\240
1
\342\200\242
\342\200\242
\342\200\242
\342\200\224
n-\\
n +1
Useful
this
n-2
n-\\
+ l
\302\253
\342\200\242
\342\200\242
\342\200\242
\342\200\224\342\200\224-
111
= 2(/i
7\302\273
In
n-\\
n-\\
+ l
\302\253
+ l)
a simple approximation,
7=0.577.. is Euler's
useful
+ l
\302\253
= 2(/i
where H(n)=
3.6
n-\\
4_
n-2
n + \\
n-\\
2+
n-\\
\302\253+l
\302\253+l
+ l
\302\253
2+
53
Facts
Useful
More generally, if an
=
\302\253
/!(/! +1)
(3.22)
+c\\ where
=an_\\
c is a constant, then
n(an+a{)
\342\200\242\342\200\242\342\200\242
=
+0\342\200\236
ax +02+03+
1+2 + 4+
generally,
\342\200\242\342\200\242\342\200\242
+2\"=2\"+1-l.
if
an
<c
< 1,then
=can_\\,
the
sum
of the
(3.24)
c*\\ is
where
'*' +an=a\\
ai+a2+fl3+
If 0
(3.23)
series
Geometric
More
a constant,
then
cn-\\
infinite
(3.25)
c-\\
geometric
series
is
Analysis of
54
2>
/
Algorithms
(3
-TT-
=1
of squares
Sum
\"
+ 1)(2h
h(h
.,
+ 1)
(3
h\"
Harmonic series
=
//\342\200\236
\302\243
j
= ln/i+Y
+ 0(l/\302\273),
is Euler's
constant.
y= 0.577..
where
(3
\\ogha
(3
\\ogab
loga* =
log/,*
(3
>og/,a
(3
(3
of logarithms
Sum
ILlog2'J
+
=(\302\273
!)Llog2\"J
-2Llog!\"J
+ 1
+2
(3
0(wlog\302\253).
1=1
a summation
Bounding
If
is a
f(x)
integral
\342\200\236
function,
then
.v=n+l
(3
J /(*)</*.
I/cos
/=!
,v=l
approximation
Stirling's
n! =
by an
\\2tw
In particular,
(3
(1+0(1//i)).
log2(w!)
= 0(w
logn).
BibliographicNotes
3.7
Further
and
55
Reading
Summary
Niels Bohr
that
said
once
of an
it is
but
algorithm,
We
characteristics
important
predict; especiallythe
not that
The main
It is
future.\"
from
far
being easy.
to extract only the
many details and attempt
of the algorithm.
The O notation is useful
in that respect,
is
a
it
first
On the other hand,
the
only
approximation.
hard to
is very
\"it
the behavior
to predict
hard
ignore
to get
designer
from
efficiency
of an
algorithm
of
indication
some
least
at
the
deter
not
should
difficulty
the
algorithm.
In
The first
This will
many
cases,
thing
we should
us
give
The first
do with
some idea of the
terms
few
when recursion is
especially
help
in
of the
behavior
relation,
but
relation.
a recurrence
get
is to look
the
making
used, we
relation
a recurrence
at
first
the
few terms.
by no means enough.
a solution.
Another useful
it is
Section
3.5.2.
Guessing and
good technique for solving recurrence relations, but it is usually just a first
\342\200\224
We must be careful not to \"overguess\"
that
is, to try an upper bound that is
step.
There are many
other
most
correct, but too pessimistic.
techniques.
Fortunately,
in
lead
to
one
of
a
few
of
classes
recurrence
that
practice
very
appear
algorithms
is to
step
expand
verifying
is a
relations,
most
a first
several
are described
of which
that n
step,
recurrence
the
has a specialform
this
It is
chapter.
Further
to assume,
sufficient
usually
that n is
particular,
and
Notes
Bibliographic
in
\342\200\224
in
a powerof
as
2.
Reading
analysis was promoted in the early 1970's, and it was met with
There are
efficiency.
by now the major measure for algorithm
\342\200\224
on
and
that
cover
several books \342\200\224
discrete
mathematics
combinatorics
mainly
recurrence
and
other
needed
for
summations,
relations,
expressions
techniques
evaluating
The
of asymptotic
idea
some resistance.
for
analyzing
Knuth,
and
It
is
Brualdi
algorithms.
Patashnik
[1989]
devoted entirely
to algorithm
Additional books and survey
Purdom
and
Knuth
Brown
[1976]
[1985a],
discusses
[1977], Bavel
are just
analysis.
papers
latter
Greene
include
contains
[1982],Roberts
[1984],
and
in
[1987],
the
Lueker
the solutions to
and
Knuth
[1982],
and Hofri
notation.
[1980],
Lueker
Graham,
that are
material.
[1980],
[1987].
techniques for
and Bentley, Haken, and
Additional
3.24.)
Tarjan
56
of
Analysis
Algorithms
Exercises
Drill
(n) is a polynomial
3.1
Prove
that,
if P
3.2
Prove
that,
if f(n)
3.3
Prove, by
using
= o(g(n)),
in /i, then
then
Theorem
3.1,
that
Theorem
3.1,
that
O (log (P (/i)))
Is the
= 0(g(n)).
f(n)
= O
(log n).
opposite
true?
/i(log3/i)5=0(/i12).
3.4
Prove,
by
pairs of functions
Compare the following
whether f(n) = 0(g(n)),f(n) = Q(g (n)),
in
terms
of order
100/1+log/i
b.
log
n +
n2
(log /i)2
n (log
(log/i)log\"
nL
(log/i)5
f.
n 2\"
3\"
recurrence relation.
T(\\)=
Give
an exact
solution.
Give
an exact
solution.
1.
T(n),
which
is defined
+ 2/ilog2/T,
7,(/7)= 27,(L/i/2j)
satisfies T(n) = 0(n
The following
matrix
)2
log /i
e.
3.9
case, say
/i
d.
3.8
In each
log (/i2)
log/i
3.7
magnitude.
S(\
a.
c.
of
= Q(g (n)).
and/or f(n)
fin)
3-6
>0
constants a, b
for all
= 0(nb).
(\\og2ny
3.5
using
multiplication
= 4.
7(2)
the recurrence
by
T(2)
relation
= 4,
log2/?).
recurrence
([Pan
relation
describes
1978]). What
is the
asymptotic
of a
running
recursive
time of this
algorithm
algorithm?
for
57
Exercises
Creative
T(\\)=\\.
Let A
works
on complete
depth). Suppose that
k is a parameter that has to do with
A performs
0(k) steps for each leaf in the tree, where
of the tree), and
the amount of information stored in the leafs (but is otherwise
independent
node.
We claim that
the total running time of the
time c per each internal
constant
trees
binary
the
in
following
binary trees
(namely,
analysis.
in
which
be an
algorithm
are at
that
the same
is 0(k).
algorithm
'
is by induction on /i, the number of nodes in the tree. If n = 1,
'proof\"
Assume that the claim is true for all
number of steps is obviously
0(k).
<
Such a tree consists
and
consider
a
tree
with
with
n
n nodes.
trees
nodes,
complete binary
the
induction
of a root and two subtrees, each of size (n - l)/2.
By
hypothesis, the running
time for the tree is 0(k) + 0(k) + c.
time for the two subtrees is O(k). Hence,the running
But
this is equal to O (&), and the proof is complete.
Wrong proof: The
then
total
the
3.11 Solvethe
= max
inO),
7\302\273
where 7(1)=
3.12
Solve
recurrence relation:
full-history
following
1.
the following
n-\\
r(/i)
/i
+ Sr(/)f
where T(\\)=
3.13 Use(3.34)
2i*
1.
to prove
positive
integer
&,
= 0(/i*+I).
/=i
3.14
Use (3.34)
to prove that,
for every
positive integer k,
= 0(nk+l\\ogn).
Zik\\og2i
i = \\
Creative
3.15
Find
Exercises
that f(n)-g(n)
3.16
Find
that f
f(n) = 0(s(n))
and
g(n)
= 0(r(n))
imply
f(n) = 0(s(n))
and
g(n)
= 0(r(n))
imply
a counterexample
a counterexample
(n) I g(n)
= 0{s{n)I r{n)).
3.17
Find two
functions/(n)
and g(n)*0(f(n)).
and
g (n),
both monotonically
increasing,
such
that f(n)*0(g(n))
58
3.18
of
Analysis
Algorithms
Consider the
relation
recurrence
T(n) = 2T(n/2)+\\,
T(2)=\\.
to prove
try
that
= 0(n)
T(n)
3.19
where m and
3.20
c are constant
our
the
why
satisfies
(The solution
solution for
that
(see
1,
5(2)=
parameters.
the asymptotic
that
Prove
which
S (mn)<
limit
constant
behavior of S (n),
the asymptotic
Find
(we
unknown)
should be a function
of /i, m,
and c.)
relation
the recurrence
T(n) = 2T(n-c) + k,
both c and k
where
3.21
The
are integer
relation
recurrence
following
problem is divided
into
is T(n) = O
constants,
unequal
(dn) for
some
in divide-and-conquer
appears
constant
d.
algorithms
in
which
the
size parts:
it
T(n)
+ cn.
2,aiT(n/bi)
i = \\
All
the
tf,s
6,s are
and
constants, and
satisfy
they
i-2>iA->\302\2601=1
Find
3.22
the asymptotic
behavior of this
relation
recurrence
recurrence relations.
It is
sufficient to
find
the
verifying).
asymptotic
behavior
of T(n).
a. r(/i) =
b.
(Hint:
3.23
Prove
+ l;
4rf[^l]
= 27\"
7\"(/i)
H-2/i; T(2)=
rr^TJ]
Substitute
another
the solution
that
T(n) = kT(n/2)+f(n\\
T(2)= 1.
1.
of the
recurrence
relation
T(\\) = c
is
T(n)
nlogk(c+g(2)
+ g(4)+
\342\226\240\342\226\240\342\200\242
+*(/!\302\273,
where
You can assume that n is a power of 2. (This
is a
g(m) is defined as /(m)/wlogA.
more general solution
than the one given in Section
to any function
3.5.2, since it applies
/(\\342\200\242)
3.24
solution of the
recurrence
relation
Exercises
Creative
T{n) = kT{nld)+f{n\\
= c,
T(\\)
3.25
3.26
can
Find
the asymptotic
7>) = 7>/2)
+ Vrt~,
power of d.
by
the recurrence
relation
by
the recurrence
relation
by
the recurrence
relation
1, T(2) = 2.
are powers
that
defined
n is a
function
T(n)
of 2.
defined
= 1.
T(\\)
of n that
values
= T(n/2) +
T(n)
T(\\)=
behavior of the
the asymptotic
Find
. You
function
behavior of the
+ n,
only values of n
consider
You
+ g(n)\\
behavior of the
the asymptotic
Find
\342\200\242\342\200\242\342\200\242
og
/ (m)lm
as
59
are powers
function
T(n)
defined
T(2)= 2.
T(\\)= 1,
+ n (n>2),
T\\
of 2.
\\og2n
3.28
You
can
consider
only values of n
that
Find
the
solution
of the following
recurrence
behavior of
satisfies
T(n)
2T\\
are powers
of 2.
evidence
T(\\)= 1, T(2)
(n>2),
It is
relation.
give convincing
sufficient to
that
the
find
function
the
asymptotic
f(n)
you
find
= 2.
log2fl
3.29
Although
is not
in
it is
general
the case.
always
T(n/2)+
T(n) = 2T((n-\\)/2)
with
that
the
solution
powers of 2, T (n)
b.
Show
that,
= O
3.30
this
if
of 2, that
of this
recurrence
prove
relative
of
behavior
that
for
powers
of 2 is
T(2k) = k+
(namely,
for
(log n)).
recurrence.
Use (3.34)to
Compute
for powers
relation:
odd,
s(/o=i;riog2(/i//)i
3.31
is even
n is
if n
following
1.
T(\\)=
a. Prove
to evaluate
sufficient
Consider the
sum precisely:
values
Analysis of
Algorithms
5(/i)=snog2(/i/oi.
You can
assume that
The Fibonacci
and so
numbers
F(/i+2)
definition:
n is a
Prove that
G(/i)
to solve
can be extended
+l) + F(/i), and F(l)=l,
F(n)
= F(/i
suggest a way
power of 2.
defined
it.
= (-l)\"+,F(/0.
as F(-n).
to
negative
F(0)
=0
values of
using
(e.g., F(-l)=l,
Write a recurrence
relation
for
the same
F(-2)=
-l,
G{n),
and
CHAPTER
TO
INTRODUCTION
BRIEF
STRUCTURES
DATA
is nothing
Science
but trained
and
common
organized
sense.
T.H.Huxley, 1878
/ hate
they are
intellectuals;
I am from
bottom
the
Frank Lloyd
4.1
the
top down;
Wright
(1869-1959)
Introduction
Data structures
are
is like a design of
most
about
effective
the building
a building.
for the
functionality,
intended
efficiency,
not
techniques. Putting
Other ideas
possible.
design of
an
algorithm
must
of the
form,
construction
is
from
up.
and
put
all
the rooms
building. To
beauty.
in midair may
but too
possible,
a room
may
be
One
be based on a
thorough
of an algorithm
that is the
A design
together
do that,
needs
it is
a way
not enough to
desired
In
know
of
knowledge
thorough
achieve the
expensive.
understanding
in
the
of
effect,
same
data
but
it
way, a
structure
book.
\302\243^k
Structures
Data
62
useful
notion
in the
we
write a
when
Normally,
is that of an abstract
data type.
study of data structures
program, we have to specify the data type (e.g., integers,
reals, characters). But, in some cases, the data type is not important for the design of the
first-out
(FIFO)
queue of
algorithm. For example, we may want to maintain a first-in
into
the
of
of
items
and
are
insertions
removals
queue,
items. The required
operations
the
items
must
be
in
the
same
removed
order
In
case
of
the queue.
removals,
from
items
the
convenient
and more general to design
which
in
they were inserted. It is more
the data type of the items.
We specify
without
these
for
specifying
operations
algorithms
data type that supports these operations
We call the abstract
operations.
only the required
of
an
abstract
data type is a list of operations
that
The most important
a FIFOqueue.
part
data type is a queue in which
the
Another example of an abstract
we want to support.
have priorities.
items
according to the
highest
priority
queue.
Again,
to
have
are totally
The removals
That
priorities.
the items
among
we
do not
the data
specify
in
specify
of
type
and that
ordered
are
is, the
the
the
first
item
queue.
data
to the
according
to be removed
not
type
the priorities;
but
insertions,
is
item
of
the
step
of
order
each
in
type
is called
a priority
of the
order.)
on
By concentrating
operational
nature
of a data
for
4.2
the
Data Structures
Elementary
4.2.1 Elements
We
use
the
notion
of
an
element
throughout
this
book as
generic
name
for
an
or
can be an integer, a set of integers,
a file of text,
unspecified data type. An element
data structure. We use this term whenever
the discussion is independent
of the
the algorithm
If the only
type of data. Consider,for example,
sorting
steps
algorithms.
takes are comparing
elements
and moving them around,
then
the same algorithm can be
used for sorting integers or names (strings of characters). The implementation
(that is,
the program) may be slightly different, but the ideas are the same. Since we often
concentrate
on the ideas rather
to ignore the
than
on the implementation, it is reasonable
of
the
elements.
types
The only assumptions
we make about elements are the following:
another
1.
Elements
2.
Elements
3.
Elements
can
be copied.
is possible
not
concerned
a valid
total order.
4.2 Elementary
Data Structures 63
All
to
the
size
4.2.2
operations
of the
Arrays
An array is
elementsin
a row
of
of the
elements
This size
must
modified
if
of
is the number
and
are of the same type,
the amount
of memory that should
be allocated to
all the elements
are names with
8
store the array is known a priori. For example, if the elements
if
each
character
1
of
if
of
and
the
size
the
is
characters
each,
requires
byte
storage,
array
to store
the array. The storage for an array is always
100, then 800 bytes are required
then the fcth
consecutive. If the first byte of the array is stored at location x in memory,
of the array is stored at location
x + k- 1. Consequently,it is easy to compute the
byte
in the array. In our example,
if the
of the storage of each
element
location
starting
of
the
is
at
then
the
55th
name
starts
at
the
location
433rd
10000,
starting
array
byte,
which
at location 10432, assuming
locations
is stored
are numbered
by bytes. (This
that
calculationcan
can
be
are
Arrays
array
array.
easily
very
and
efficient
be accessed
in
constant
are numbered
locations
very
time.
common
data
The algorithm
array
is fixed,
differently.)
Every element
structures.
designer
who
uses
of an
a high-level
calculations \342\200\224
are done by the compiler.
they
for
As
a rule of thumb, arrays
should
be used whenever possible. The main
drawbacks
cannot
to
elements
of
different
are
their
restrictions.
be
used
store
using arrays
Arrays
We deal with
types (or sizes), and the size of an array cannot be changeddynamically.
these two restrictions
in the following subsections.
language is
4.2.3
Records
rarely
with
concerned
location
Records
are
similar
to arrays,
except
that
we
do not
assume
that
all
elements
are of
the
A record is
thus
a list
of elements
compiler.
consist
of 2 integers, 3 arrays
of 20 integers
each, 4
may
each containing 12 characters. (Note
that the two array types
in the record
are considered now to be elements by themselves.) This recordis defined
in
with the record contains the starting
stored
relative
locations of all the
Fig. 4.1. The array
elements.
in the
Thus, if each integer is stored in 4 bytes, Int6, which is the ninth element
Since the sizes of all the
record, starts at byte number 261 (2-4 + 3-20-4 + 3-4+1).
to compute the location
it is possible
of each element
in the records are known,
elements
Data Structures
64
recordexample!
begin
: integer
Intl
Int2 : integer;
[1 ..20]
Arl
: array
Ar2
: array
[1..20]
Ar3
: array
[1..20]
lnt3
: integer;
Int4
: integer;
Int5
: integer
Int6
: integer
Namel
: array
Name2
: array
of integer ;
of integer ;
of integer ;
[1..12] of character
[I. .12] of character
end
Figure
in
constant
similarly,
4.2.4
time.
it is
not possible to
Linked
Lists
applications
the algorithm
progresses.
4.1
arrays,
Definition of a record.
the
storage
for a
add elementsdynamically.
as
the number of elementsis changing
dynamically
as arrays (or records)
to define all the elements
to ensure
a good
sufficient storage space. This is often
solution, but, of
large enough
to the worst case (and, in
course, it is not very efficient to demand storage according
there are cases where there is a
cases, the worst case is unknown).
Furthermore,
many
need for insertions
and deletions in the middle of the list.
If we use arrays
and we need to
insert
an element
in the middle, we have to shift all other elements.
This inefficiency is
inherent
in the
insertions
and
consecutive
thus,
representation of arrays;
arbitrary
in which
It
is possible
with
them
large
data
arrays.
structures
need
throughout
data
dynamic
this
book;
is essential.
of
4.2
are
There
more
the
space.
30th
two
major drawbacks to
There is one
additional
the
Data Structures
Elementary
linked-list
First,
representation.
it
Second, if we want
beginning and look at
per element.
pointer
65
requires
to
look
at
for example,
element,
pointer to point
(see Fig. 4.3).
The
complicated.
the
at
31st
the
pointer
two
details
important
stored
that
tend
to make
problem is
The main
the
implementation
to
how
detect
lists
the end
of
the
ignored several
lists a little
more
has
linked
of
list.
a special
Usually,
30th
N
ooo
30th
M
Figure
4.2
Inserting
-1
OOO
new
element
into a
>
linked list.
30th
1
\342\200\224
n
ooo
\342\200\224H
30th
\342\200\242I1\342\200\224\342\200\224M
I J
Figure
The
Deleting
\342\200\224H
to talk about
the new 31st and the old 31st
system is consecutive in nature. It is thus confusing
uses \"30a\" to denote an insertion after 30. This notation causes many problems.
If we
(30aa?). This is a good example of the need for dynamic
again after the 30th, we may run out of notation
element.
insert
4.3
OOO
number
We often
data structures.
66
Data Structures
is a pointer
to nowhere;
it can be
introduce a regular
but
record,
that the search will
end
there.
This additional
to include in it a key that will guarantee
makes the program simpler, since there
are
record,
record, sometimescalled a dummy
a
are
for
of
data
structures.
records
useful
cases.
fewer special
variety
Dummy
address,called
used to
4.3
The
they
indicate
nil,
such
is provided,
an end
of a list.
a nil
that
Another
pointer
is to
possibility
Trees
only
structure
represent.
that
There
lists
can capture is
arrays and linked
are numerous applications that
require
They
can
structures.
also serve
In this
as a
the
more
section, we
will
order
more
efficient
be
of the
structure.
data
concerned
elements
Trees
structure
with only
is a set of
as rooted
trees or arborescences. A rooted
tree
with
a
set
of
that
connect
the
nodes
elements,
(or vertices),together
edges
One
node
the
in
is
root
of
the
tree
of
a
the
elements
(the top
way (see Fig. 4.4).
special
The root is connected to other nodes, which
are at level 1 of the
hierarchy).
hierarchy;
at level 2, and so on. All the connections are thus
they, in turn, are connectedto nodes
and their direct unique
between
nodes
called parents after
\"supervisors\"
(usually
the
root
has
no
The
main
of trees is that
trees).
Only
genealogical
parent.
property
they
do not have cycles. As a result, there is a unique
between
two
nodes of a tree.
path
any
A node
is connected
to its parent
and to several underlings
the
(again,
following
we
will
call
the
latter
The
maximal
number
of
children).
genealogical terminology,
children of any node in the tree is called the degree
of the tree. We usually
order
the
children of every node, then identify
them by their index
in that order (the first,
second,
and so on). In the special
case of trees of degree2, called
we identify the
trees,
binary
children by left (for first) and right
A node with no children
is called a leaf
(for second).
the terminology
comes from real trees). A node that is not a leaf is called an
(this
time,
internal
node.
The height of a tree
level of the tree, namely,
is the maximal
the
which
we call
Figure 4.4
rooted
tree.
Trees
4.3
67
distance
between
the root and a leaf. Each node has a key, which
comes
from a
set
We
ordered
a
number
or
an
will
real
(for
totally
example,
integer).
interchangeably
no confusion
can arise. For convenience,
refer to the key and the node as the same when
assume
that we deal with
we can link together
we
all the
Otherwise,
keys.
unique
list and have one node with
a pointer
to that list.
elements with the same key in a linked
has a data
field containing
the data (or a pointer
to the data) that
Each node usually
is
with the node. The data
field
on the application, and
we
will
associated
depends
generally not deal with it.
In this section, we concentrate on two
uses
of trees: search trees, and
In
heaps.
with
of the representation
trees
are used. We start
a discussion
of trees
both cases, binary
maximal
in memory.
4.3.1
There
are
two
main representations
representation. In
is done by a pointer.
(In
of Trees
Representation
some
the
A node
applications,
of trees, an
representation,
explicit
with
k children
also contains a
a node
representation
implicit
to
tree
an
its parent.)
and an explicit
to another
node
array
It is
of k
usually
pointers.
more
to have all
the
nodes
Figure 4.5
Binary
of the
representation
of a nonbinary
tree.
68
Data Structures
from
left
level
to right,
root is storedat
is stored
in
A [1]
A[2i],
can define
We
level.
by
the
representation
node v
(1) The
by induction:
that is stored
The
in
A [i]
of
advantage
this
saves
On the other hand,
if
no pointers are required, which
storage.
representation
much
from
if
some
leaves
are
farther
the
root
than
is
unbalanced,
away
namely,
the tree
nodes must be represented. An unbalanced
tree is
others are, then
nonexisting
many
in
node
indicate
its
the
An
below
each
shown in Fig. 4.6; the numbers
position
array.
or
8 nodes. The implicit representation thus
may
array of size 30 is neededto represent
is that
may
not save
operations
storage, depending
the
in
can be reasonably
on
of the
middle
if they
supported
Also, since
tree.
the
other
the
On
are
used,
dynamic
hand, dynamic operations
to the end of the
correspond
arrays
array.
4.3.2
Heaps
A heap is a binary
tree
whose
keys satisfy
node is greater
the
than
property:
heap
following
to
or equal
the
of
of any
key
its
children.
By the transitivity
or equal
to
law,
than
a priority
implementing
two
the
the
of
all
queue,
which
keys
node's
that
is an
data
abstract
type
in
by the following
defined
operations:
insert a key x
Insert(x):
Heaps
will
implies
property
heap
use
into
largest
the
structure.
data
key from
the data
structure.
implicit
the implicit
We assume
elements
the
the
that
heap
is
array
will ever
Figure 4.6
i4[l..\302\243],
where
contain (if
Implicit
an
representation
k is an
upper
upper
bound
of an
tree
representation.
the heaps
that
bound
is not
be
will
on
known,
unbalanced tree.
the
then
We
balanced.
number
of
a linked
4.3 Trees 69
Let n denote
required).
A
[\\..n
] is of interest
array
is
representation
only
how to implement insert
namely,
the
We
key
largest
from
The
root.
the
first
[n ],
:= A
We
the leaf
take
two
have
still
decrement
and
heaps
separate
we
property,
now
jc
propagate
This is done by
maximum.
the
proceed
the heap;
to describe
heaps.
the node
property,
heap
in
always
operation
the
with
removes
the key
the
property
heap
on top, which
or may not satisfy
the
may
the
at
this
is
if
the
satisfy
heap property
point
was
contained
the value jc.) To restore
the heap
a value
plus
whole
By
use of
now
heap
the
with
is to restore
problem
We now
been deleted.
We
efficiently
operation.
is the root,
a heap
in
Remove
the
with
start
remove
and
We
moment.
at any
of elements
number
current
the
jc
jc
to
the tree,
down
jc
comparing
a subtree for
it reaches
until
with
the
values
of
its
which
children
two
it is
04 [2]
and
A [3])
remains unchanged,
the subtree
about
continue
key
jc
again
thus
rooted
at
inductively
is now
if
jc
is
in
the
it also
A [2]
same
satisfies the
now it
Assume that
(because
way.
We have to
property.
heap
has
we
jc
in
its root).
But
only
worry
now
we
continue
atA[j].
compare
exchange
and
A[2j+\\]
either
(if they
when jc
exist),
becomes
can
the
We
and
the
number
of comparisons
a leaf. The maximal
of a subtree,or when
it reaches
tree.
The algorithm
the
of
the
for
which
is
twice
a deletionis 2|\"log2\302\253~|,
required
height
for removing
a heap is given in Fig. 4.7.
a maximum element from
the new key
n by one, and insert
An Insert operation is similar. We first increment
as the new leaf A [n ]. We then compare
and exchange if the
the new leaf with
its parent,
new
of its subtree
the new key is the maximal
leaf is larger than
its parent.
At this point,
it was
found to be larger). We
was the maximal and
(since the parent
assume,
and
A
the
tree
satisfies
that
the
rooted
condition,
[n ])
heap
?xA[j] (initially
inductively,
that if we remove
We
continue
the
this tree the rest of the heap satisfies
heap
property.
this
its
the new key up the tree, until the new key is not larger than
process,
promoting
The
it reaches
the root). At this point, the whole tree is a valid
parent
(or until
heap.
maximal number of comparisonsrequired
is riog2\302\253l, which is the height
for an insertion
of the tree. The algorithm for inserting
in Fig. 4.8.
into a heap is given
an element
in time
of
and
Insert
Remove
Overall, we can perform
sequence
operations
any
O (\\ogn)
On
is
the
other
it
other
not
to
hand,
per operation.
possible
operations
perform
a heap. For example, if we want
with
to search
for a given key, the hierarchy
efficiently
A Heap is a good exampleof an implementation
of an
given by the heap is not useful.
a limited
abstract data type. A heap supports
number of specific operations
very
we need these particular
we can impose the
Whenever
operations,
efficiently.
heap
structure
on the data whatever its type is.
maximal
70
Data Structures
(A, n);
Remove_MaxJrom_Heap
Algorithm
Input:
Output:
of size
array
(an
maximal
Top_of_the_Heap (the
heap), and
new
(the
begin
ifn = 0 then
size of
the
is
\"the heap
a heap).
element
representing
if n
heap;
of the
= 0, then
heap), A
the
heap
empty\"
else
:=A[\\];
Top_of_the_Heap
A[\\]:=A[n];
n := n -1 ;
parent :=
1;
2 ;
:=
child
-1 do
child := child + /
;
then
> A[parent]
ifA[child]
swap(A[parent],
parent := child;
child := 2*child
child :=
else
then
< A[child+J]
[child])
the loop
stop
{to
end
Figure 4.7 Algorithm
_Heap (A,n,x);
Insert Jo
Algorithm
Input: A
Output:
of size
(an
RemoveJAaxJromJieap.
array
A (the new
heap),
representing
and
n (the
a heap), and x (a
new size of the heap).
number).
begin
n :=
A [n ] :=
while
assume that
the
array
does not
overflow
x;
child := n
parent
; { we
+ 1
:= n div 2
parent
> 1
;
do
A[child]);
[parent],
swap(A
child
then
:=
parent
{ see alsoExercise4.6}
2 ;
the
loop }
end
Figure 4.8
Algorithm
Insert
jojieap.
(the
new
is empty).
4.3
4.3.3
trees
search
Binary
implement
delete(x):deletethe
data
Abstract
insertions
holds
(or to
nil).
are
leaves
node
be removed
may
simplicity,
Binary
search
of
may be
pointers
assume
we
the
and
dictionaries.
called
as well
in the
already there).
is there.
if it
structure
efficiently,
limit ourselves to
key
it is
(unless
are important
deletions
and
structure
use the
will
We
operations.
the data
dictionaries
implement
operations:
following
the data
x from
key
that handle
types
search trees
x into
key
data
the
in
will assume
we
insert
insert(x):
the
efficiently
search(x)\\
node
71
Trees
binary
in
manipulated
in heaps
trees,
other ways.
because
only
search
many
any
For
are distinct.
Search
tree
is a structure to facilitate searching. The structure
search
that we have a key x
Assume
search procedureis understood.
and we want to know
whether
it is currently a key of a node in the tree, and if it is, we
x against the
want to find that node. This operation is called a search. We first compare
=
If
x
root
we
of the tree, whose value
If
x
then
are
done.
r.
<r, then we
is, say,
r,
name suggests,a
As its
once the
becomes
clear
continue
child.
in the
keys
it.
keys satisfy
this
tree is presented in
than
the
continue
the
range
search
of the
from the
keys below
right
it:
the
and
it,
search trees.
We
A simple
condition.
divide
tree serves to
rule defines
This
we
otherwise,
child;
all smaller
are
subtree
left
greater than
search
the
in
key
left
the
in
search
a binary
4.9.
Fig.
Insertion
Insertions
for x is
into
is already
performed
(We assume that we
If x
first.
be aborted.
will
Otherwise,
be
then
value
the
that
which
we
ends
search
The tree
find
is given
remains
the
in
leaf.
Fig.
4.10.
in
do
the
Given
simple.
then
tree,
it
will
(as either a
consistent,
the new
We use this
it to
through
also quite
(unsuccessfully)
inserted
of x).
trees are
search
binary
right
since
leaf. A
child
nodes
with
containing
the
a search
insertion
same
key.)
the new key can
or a left
subsequent
node
a key x to insert,
be found and the
program
to write a
72
Data Structures
to a
(a pointer to
node
Output:
x) ;
root of a
(root,
BST_Search
Algorithm
search
binary
node
the
and x
tree),
the key
containing
jc,
(a number).
or nil
if no such
node exists).
begin
if root
= nil or
{root\"
is the
that
record
else
ifx < root\".key
then
the
:= root
then node
= x
root\".key
is pointing
root
pointer
x)
BST_Search(root\".left,
else BST_Search(root\".right,
to.}
x)
end
4.9
Figure
Input:
root
(a pointer
to a root
tree is
changed by
The
Output:
the
(root, x);
BSTJnsert
Algorithm
child;
pointer
BST_search.
Algorithm
search
inserting
a node
and x
tree),
the
with
a node with
is already
there
if
of a binary
key
begin
if root = nil
then
node pointedto
a new
create
by
child
child ;
root :=
:=
root\".key
else
node :=
:=
child
while
root;
{ to
root;
node *
nil
and
= x
if node\".key
that it is
it so
initialize
not
nil}
* nil do
child
then child
:=
nil
else
parent
:= node
then
ifx < node\".key
else node := node\".right
if child
create
nil
:= node\".left
node
;
then
a new
node pointedto
;
by
child
child\".key := x
child\".left
:=
nil;
child>ight
then
else parent\".right
:= nil;
parentleft
:= child
:= child
end
Figure 4.10
Algorithm
BSTJnsert.
jc,
(a number).
key x
then
pointed to
child
= nil.
by
Trees
4.3
73
Deletion
It is easy to delete a leaf;
we need
complicated.
be
to
nil.
It
also
not
is
hard
to
delete
a
node
that
has
pointer
change
to
the
is
the
node
to
to
its
child.
if
the
child;
However,
pointer
changed point
has
two
then
we
need
to
delete
find
a
for
to
the
two
want
children,
place
pointers.
Deletionsare
more
generally
to it
the
only to
only one
node
we
Let B
be a node with two children whose key we want to delete (see Fig. 4.11). In the first step,
we exchange the key of B with a key of another node X, such that (1) X has at most one
child, and (2) deleting X (after the exchange) will leave the tree consistent. In the second
now has the key of B which
we wanted
to delete. We can easily
step, we delete X, which
the consistency
of the tree, the
delete
X, because it has at most one child. To preserve
as
as
all
the
X
must
be
at least
in
the
left
subtree
of
of
B, and must be
large
keys
key
the
all
subtree
of
B.
Notice
that
the
of
X in Fig. 4.11
than
the
in
smaller
right
keys
key
satisfies these constraints: it is the largest among the keys in the left subtree of B. X is
X cannot have a right
of B in the tree.
it
since otherwise
called the predecessor
child,
The deletion algorithm
not have the largest key in that subtree.
in Fig.
is presented
would
4.12.
Complexity
the tree
take
The
the
and
us all the
constant
time
to
way
of search,
of the relevant
the
bottom.
the actual
(e.g.,
worst-case running
times
running
location
node.
In
insertion,
the
and
insert,
the
steps
on the shape of
the
search
would
case,
path
delete
worst
in
the
depend
algorithms
of keys in the
exchange
of a path from the
root
require only
So, the
deletion).
is
maximal length
to a leaf, which
tree is reasonably balanced(we will define balance shortly),
then its height is approximately
log2fl, where n is the number of nodes in the tree. All the
If the tree is unbalanced, then these operations
case.
are
are
efficient
this
in
operations
much less efficient.
If
a binary
the
search tree in a random
order, then the
keys are inserted into
\342\200\224
more
2ln n. In this case, the search
expected
height of the tree is 0(\\ogn)
precisely,
and
insert
of the tree can
are efficient. In the worst case, however, the height
operations
be n (when the tree is a simple linked list). Trees with long paths can result, for example,
from insertions
in a sorted, or close to sorted,order. Also,
deletions
may cause problems
even if they occur in a random order. The main reason for that is the asymmetry
of
the
If
to
a
deleted
node.
there
are
always using
predecessor
replace
frequent deletions,
the height
of
the
is the
time
tree.
If the
\302\243
Figure
4.11
Deleting
a node
with
two
children.
74
Data Structures
root (a pointer
Input:
The
Output:
(
to a
tree is
changed by
a binary
search
a node
deleting
never deleted,and
assume
We
x);
root of
(root,
BST_Delete
Algorithm
and x
tree),
the
with
that
(a number).
key x,
all keys
if
it exists.
are
distinct
begin
:= root
node
;
and
* nil
node
while
:= node
parent
node
else
if node
if node
* root
if
then
node
node\".right
then
nil
is not
print(\"x
parent\".right
else if node\".right
ifx < parent\".key
the
tree\")
; halt
then
else parent\".right
two
node! :=
\".right
:= node\".right
- nil then
:= node\".left
parent\".left
:=
node\".left
case
children
node\".left;
parent! := node
nodel
parent I
do
* nil
\".right
:= node!;
\".right ;
nodel
:=
nodel
{ now
in
then
:= node
parent\".left
while
node\".left
then
= nil
else {the
:-
then
node\".left
else
*xdo
node\".key
the actual
comes
:=
parentl\".right
node\".key :=
deletion }
nodel
\".left;
nodel \".key
end
4.12
Figure
followedby
and
deletions.
insertions,
This
have a height
of O
can be avoided if,
asymmetry
Algorithm BSTjdelete.
(\\\302\253),even
instead
of
for random
always
insertions
choosing
the
section.
of the
4.3 Trees
4.3.4
75
Trees
AVL
structures
Definition:
of
This
definition
an
a maximal
an AVL tree
h of
height
Proof: Left
theorem
This
as an
We
property.
every node,
tree
and
right subtrees is
at
most
as 0).
of O
height
(logn),
as is
in the
shown
next theorem.
<
1.4404
with
start
key
log2(>z+ 2) -0.328.
exercise.
that search in
to perform insertions
Let x be a new
bottom of the tree in
nodes satisfies
n internal
with
implies
is how
problem
left
its
4.1
Theorem
\342\226\241
The
empty
guarantees
search
binary
the heights of
tree is defined
between
(the height
is a
tree
AVL
An
difference
the
\342\226\2
AVL
an
and
insertions;
that
the usual
again,
we wish to
way.
we assume that
into
insert
all
tree.
an AVL
the insertion,
after
If,
tree requires
and
deletions
the
tree
First,
AVL
the
maintain
are distinct.
we
remains
insert
x at the
an AVL
tree,
we
binary
h + 2,
are done.
search
which
(Fig. 4.14).
same as the height
property
is the
The
height
is now
As a
of the
not
the new
(Fig. 4.15). Again,
so no more balancing is required. An
rotation
(single or double) is always
is required
one,
will
the
critical
node.
It is the root of the
subtree as a result
of the insertion.
To perform
the
critical
node
find
the
and
we
to
to
have
determine
of
which
the
cases is
insertion,
in each node a balance factor,
is equal to the difference
involved. We maintain
which
For AVL trees, the
between
the heights of the left and right subtrees of this node.
smallest subtree
that
becomes
a non-AVL
Data
76
Structures
h-I
h-l
/\302\273+/
new
new
(a)
4.13
Figure
balance factor of
rebalancing if
of
the
(b)
Insertions
node
each
balance
that
invalidate
1, -1, or 0. An
was either 1 or -1, and
is
factor
the AVL
insertion
the
property.
a subtree
into
increases
insertion
requires
the height
a subtree
nonzero
after
h-l
h+1
h+l
h-I
new
(b)
(a)
Figure
4.14
single
rotation:
(a) Before,
(b) After.
4.3
77
Trees
new
(b)
(a)
Figure 4.15
double
rotation:
(a) Before,
(b) After.
insertion
the old heights from the critical
Hence,
node).
(recall that the balancing retains
the critical node is the lowest ancestor of the new node with nonzero balance factor. On
the
one.
down the tree, we look at the balance
factors, remembering the last nonzero
way
to the \"right\"
or
When we reach the leaf, we can easily determine whether
we insert
\342\200\22
or
down
bottom
We
then
another
direction.
make
top
up
pass (either
\"wrong\"
less nodes), readjust the balance
bottom
involves
factors,
preferably
up, since that usually
and perform a rotation
if necessary.
We omit the details.
It is no longer true that the tree can be
more
Deletions
are, as usual,
complicated.
There are cases
after
a deletion.
rebalancedwith only one single or double rotation
where
rotations
are required, where n is the number of nodes in the tree.
0(\\ogn)
number of steps; thus, the worst-case
each rotation
Fortunately,
requires
only constant
time
of
a
deletion
is
still
we omit the details.
O
running
(log n). Again,
Comments
worst
in
the
trees
AVL
case, requiring
case.
average
at
most
form
45%
Empirical
an efficient data
structure.
more comparisons
studies have
They
than
shown
optimal
the
average
perform
trees,
well
in
the
search
time to be
main
[1973], pp. 460]). The
and the
factors,
disadvantages
storage for the balance
fact that the program that implements them is rather
complicated.
Many other schemes
been
for balanced-search trees have
2-3
trees, B-trees, weightproposed,
including
balance
trees, and red-black trees.
approximately
log2fl+0.25
of AVL
comparisons
need
(see
for extra
Knuth
78
4.4
Structures
Data
Hashing
is
Hashing
algorithms.
can also be
for
structure
one
of the
most (if
not
used for
storing
The
deletions.
data
with
keys
the
most)
idea behind
numbered
structures
is
hashing
from 1 to
computer
of
it
a data
Designing
The data can be stored
simple.
is easy:
for
some variations
array of
size n,
such
that
key / is
stored at
data
useful
insertions
for
location
(there are
digits of
the
billion
numbers,
possible
social-security
in which
79
4.4 Hashing
since it
is much
same year
in the
social-security
the
size
than
it is
that
number.
Functions
Hash
We assumethat
the
the table
cannot
size that
is
keys
function
hash
effective
and
were born
the
be
easily
adjusted
a power
of 2, for
) mod m,
= (xmodp
to be
a prime
example),then
of the
where m
(it
the
is a
hash
prime
is convenient
hash
following
is m. A simple
table
number. If the size of
sometimes to have a
function can be used:
be sufficiently
larger
(x)
where
p is
Handling Collisions
The
way
simplest
in
entry
hashed
the
into
that
called
collisions is to use a method
chaining.
separate
all the keys
serves as a head of a linked list containing
search
To access a key, we hash
it and then perform linear
to handle
hash table
entry.
Each
that are
on the
will
fail.
Another
are
no pointers.
The
hash
function
determines
size
the place
of the
of
the
key
80
Data
Structures
then
the first empty place after
it is
place is already occupied, that is, if a collision occurs,
is
table
considered
the
the
same
A
for
follows
search
(The
instead.
procedure.
key
taken
order; if the last place is reached and it is full, then the first place is considered
in a cyclic
is
thus ends at the first empty place. When the table
search
unsuccessful
An
next.)
the
is
there
will
works
well.
If
table
method
this
relatively full,
simple
relatively empty,
that are caused by keys with different
are collisions
collisions, which
be many secondary
with
that have the same hash
avoid collisions
function,
hash values. We cannot
keys
such keys are mapped into the same place. We should, however, try to minimize
because
Suppose that the /th place is full and that
secondary collisions. Let's look at an example.
A new key, which is mapped to /, will
cause
a collision, and
the (/ + l)th place is empty.
the collision
is resolved with
This case is efficient,
since
into /+1.
be inserted
will
to / + 1, there will be a secondary
effort.
However, if a new key is now mapped
minimal
it
is
not
full
become
full
collision and /' + 2 will
(if
already).
Any new key mapped to /, to
/ + 1, or to /+ 2 will not only encounter secondary collisions, but will also increase the
effect
is called
This
size of the full segment, causing more secondary collisionslater.
clustering. When the table is almost full, the number of secondary collisions with linear
to linear search.
will be very high, and the search will degrade
probing
with
linear probing. If an insertion
Deletions cannot be implemented
efficiently
if
on
its
to
then a
a
an
and
that key is later
deleted,
key
way
empty slot,
4'passes\"
through
will
future search
be unsuccessful,
since it will stop in the new empty
slot.
If deletions
are required, we must have a collision-resolution
scheme using pointers.
The clustering
effect can be reduced with double
When a collision
hashing.
occurs, a second hash value h2{x) is computed. Instead of searchingin a linear order,
namely, / +1, /' + 2, and so on, we searchthe places i + h2{x), i + 2h2{x),and so on (all in
a cyclic order). When
another
will be
key y is mapped to, say, / + h2(x), the next attempt
at
i + h2(x) + h2(y), instead of at i + 2h2(x).
of h2(y),
then
If h2{x) is independent
is
eliminated. We must be careful, however, to choose the second hash value
clustering
such that the sequence
/ + h2{x), i + 2h2(x)...,i + nh2(x)
the whole table (which
spans
will happen
if the numbers h2{x) and n are relatively prime).
The main disadvantage
of double hashing is that
it requires
more computation
of a second hash
for
the
One
to save extra
search.
(namely, the selection
value)
way
is
to
select
a
second
hash
is
not
of the
value
that
computation
completely independent
first hash value, but that still reduces clustering. One such method
is to set h2{x) = 1 if
= 0, and
that m is prime and that
assume
h\\(x)
h2(x) = m-hx{x) otherwise
(we
=xmodm).
h{(x)
The
4.5
The
the
union-find
use
Union-Find
(also
problem
of nonstraightforward
Problem
known as the
data structures
problem)
equivalence
to
The
problem
divided
into
is a good
example of
improve
kinds
4.5 The
find
(/):
returns
goal
is to design
operations as efficiently
is
group
A with
distinct
from
an
ahead
X[\\..n]
array
solving
X[i].
that
takes more time. Assume
is necessary to change all the
We
now
xt
group B to form
names
will
do).
will support
any
sequence
a new
the other
that
structure
to allocate
possible
data
that contains
group
group
with
these
of
two
as possible.
of the
the name
81
Union-FindProblem
present
union(/l,
entries
a different
identity
\342\200\224
we
of time
(and
for them.
of the
group
this
the
containing
B) results in a combined
Bio A.
containing
problem.
indexed
from 1 to
n),
method
of
straightforward
look at the
simply
approach to
are
they
The
array.
/th element
A union
group called
Instead
in
operation
A.
of making
it
Then,
the
find
operation
Each
entry
in
the
nil
nil
11
Figure
4.16
the
union-find
problem.
Data Structures
82
of
the
store
We
way.
following
group,
when a
Balancing:
group is set to
The
arbitrarily).
the
appropriate
shown
as is
log2\302\253,
size
of the
utilizes
the
in
the name
only
the
the
combined group
is also
the smaller
of
pointer
(ties are
group
larger
broken
and placed
computed
in
root.
the
in
of
record
not
it.
is performed,
operation
the
to
field
union operation
If the
union
point
a root
that corresponds to
record
each
with
but also
of
the height
then
balancing,
more
is never
trees
the
than
theorem.
following
4.2
Theorem
\342\226\241
// balancing is
used, then
tree
any
of height
at least
contain
must
2h
elements.
Denote
and
h(A)
by
elements;
twice as
hence,
ZTs
find
union
final
At that point,
final destination,
destination.
stations
about
traverse
path to
point
Traversing
the
complexity
time a find
every
of a
both
steps
the
log2\302\253pointers.
would
be
so
they
that
If several
analogy.
address to
a good idea
can forward
until
another
to
the
all
notify
changes of
it reaches the
forwarding
After
we
on
the
number
is performed.
the worst case
of steps;
The
therefore,
the
the same.
remains
which
theorem,
following
asymptotic
compression
we
will
not
complexity.
4.3
balancing
in
operation
operation
\342\226\241
Theorem
//
it
only doubles
find
than
\342\226\2
again
path
time
prove,
to
directly
the
the
its
Consider
idea.
following
and
is satisfied.
more
follows
never
operation
than ,4),
smaller
be
constant
take
addressesoccur, then
height.
original
that a
4.2 implies
union operation always
Theorem
or
obviously
as ZTs tree
more than
is one
height
tree
combined
theorem
the
heights
the combined
of
elements
many
the
h(B)
The height
respectively.
is
h(A)
larger, then the
worst
and
path compression
case for any sequence
is 0(m\\og*n),
union)
defined below.
where log*n is
the
operations
iterated
total
number
(either
logarithm
find
of
or
function,
\342\226\2
4.6
(b)
(a)
Figure 4.17
function
The
\\og*n
log*\302\253is
defined
1 +log*([log2\302\253]).
= 1+log*
log* 60000
all
virtually
find
operation
0(n
\\og*n)
is almost
finds
We
as follows,
log*
(b)
1 =
4.
For
any
we have
number
such
\302\253,
<5.
log*2 =
\\ogn
that
for this
any
in
each
O(n)
2,
analysis,
an
open
sequence
particular
of them require
involves
which
step separately.
problem is still
n >
log*4 = 3, and
a?<265536, which covers
1. For
14 = 1 +
Thus,
may
steps.
After.
than
counting all steps together rather
time
possible to design a linear
algorithm
4.6
(a) Before,
compression:
recursively
purposes,
practical
and
of unions
Path
For
16 =
83
Graphs
Whether
it
is
problem.
Graphs
chapter (Chapter 7) to
a whole
devote
used to
algorithms.
graph
section, we discuss
of a set V of
consists
E)
In this
data
vertices
structures
G =(V,
(also called
In
the two vertices
the edge connects is important.
arrow pointing
from
one vertex (the tail) to another
(the
The edges in an undirected
head).
graph are unordered pairs. Trees are simple
a hierarchy
to indicate
of graphs. If we want
in a tree,
we can orient
all the
examples
trees
Such
are
root.
the
sometimes
called
from
it
rooted
since
edges to point
trees,
away
is enough
of all the edges. We can also
to specify the root in order to define the direction
do not correspond to a
which
consider undirected trees (sometimescalled free
trees),
pairs
\342\200\224
the
case we
specify
ordered
this
hierarchy.
order
an
edge
between
as an
84
Structures
Data
use
will
We
representation
G is an
matrix of
the matrix is
example,
of edges
number
the
size n2
space of
require
they
which
no
a tree
in
(depending
pointers per vertex
With adjacency matrices, each vertex
most
if the number of edges is small,
tree).
words,
if (v,,
only
the
in
The
|V|=\302\253.
v7-)
yth position
The
e E.
first
adjacency
of
row
/th
if there is
an
edge
or two
one
= 1 if and
has a 1
The
book.
this
in
Let
otherwise.
a 0
and
that a^
such
graphs
of a graph.
matrix
of size
array
v7-,
of
representations
adjacency
n matrix
an
from v, to
leading
the
n x
thus
main
two
uses
on
we
whether
has
entries
of the
in
to
want
the
or
go up
array of
an associated
other
In
matrix
adjacency
the
down
size n.
be
will
Os.
Instead
of
an
having
representation
explicit
Is (representing
of
in a
link the
can
we
be
one
pointer
the adjacency list. In the adjacency-list
each vertex is associatedwith a linked list consisting of all the edges
representation,
vertex. This list is usually sorted according to the labels of the heads of
this
to
adjacent
number
actual
per edge.
the
the
edges)
This secondrepresentation
The whole
edges.
corresponding
in the
array includes
its list
of edges.
label
the
the
If
is called
graph
by an array of
a pointer to
is represented
graph
and
or
insertions
deletions
of
beginning
are allowed
4.7
Data
can
structures
we start
They
represented by
arrays
in the
Summary
structures.
efficient.
can be
The size
using
it,
Linked
can support
and
be divided
into
of an
or at
and
dynamic
least a good
structures.
on
bound
it, has
static
are
Arrays
to be
known
before
it cannot
lists are
any
array,
static
size
Data structures
can
multidimensional structures.
structure
they represent is the
also
Arrays
possible
be
divided
and
order
into
linked
among
one-dimensional
structures
one-dimensional. The
the elements. Trees represent
a
lists are
and
only
little
Bibliographic
1
1
(b)
(a)
13
12
Reading 85
13
10 11 12
(c)
4.18
Figure
more than
more
even
Graphs can
hierarchy.
represent
they
Of course, we
structures.
elaborate
\342\200\224
structure
one-dimensional
Graph representations.
multidimensional
represent
or
arrays
The concept
operations
are
required
dictionaries, priority
If we need only
best option.
the explicit key of
hash table, then the
is the
The
study
and
Knuth
include
to
whole
table
We
structures
data
a result,
any
structure
be used if the
now
considered
[1980],
[1984],
and
algorithms
Aho,
and
on them,
imposed
access dependson
to
the
find
the
that
for
[1983].
then
something
minimal
hashing
besides
key
in
Reading
a
basic
have
about
part of computer
been
data
science
is by Tarjan
concentrate on
data structures.
if we
wish
For example,
must still be scanned.
is
us to
allows
It
[1983], Gonnet
structures
cannot
element.
the
of
[1973b]
data without
store
Hashing
As
Standish
and union-find
queues,
Bibliographic
education.
and
structure,
on the
dependent
abstract
of
on data
86
Structures
Data
study of many
Jones's
paper also
A comparative
[1986].
Jones
The algorithms
others,
described, among
empirical study
was performed by
random deletions
comparison
Knuth
in binary
length of the
the
that
search trees
average
certain
under
that,
were
average path
[1973b]. An
deletions
and
Culberson[1985]proved
path
conditions,
A
of the average
path to be O(yn).
Schwab
in
Baer
and
is
schemes
presented
balancing
the length
cause
insertions
and
different
between
the
by
priority
trees
search
that
see
information,
who conjectured
[1983],
Eppinger
as 0(log3A?).
be as high
the effects
on
may
by Hibbard
on
bibliography
comprehensive
n random
after
length
insertions
for
queues.
includes a
was done
for priority-queues
structures
data
Sleator
in Knuth [1973b] and Tarjan
[1983].
[1977]. Balanced trees are also described
for
methods
new
maintaining self-adjusting trees.
and Tarjan [1985] present several
node to the top after
accessed
is to adjust the tree by moving the most currently
idea
The
balanced,
access.
they exhibit good performance
Although the trees are not always
every
be slow, but over a
in the amortized
sense; namely, a singleoperation
characteristics
may
is
small.
each
for
time
the
operation
average
long
period,
in Knuth
can again be found
about
More information
[1973b], and in
hashing
Gonnet
A book
[1984].
by
and
Vitter
describes
Chen [1987]
in
great
random
also
Fischer
by
[1972],
in Theorem
mentioned
0{ma{my a?)),where
than
log*a?.
compression
7 and
Hopcroft
a(n) is the
achieve
the same
variations
graphs,
and
result
of path
see Chapter
Exercises
4.1
Write
a program
to delete an
4.2
Write
should
a program
all point
to reverse
backward.
Convert
the
simple
element
the
from a linked
direction
recursive search
list.
list.
of a given linked
procedure
for
binary
In
other
words,
search trees
to
procedure.
4.4
the
its bibliography.
Drill
4.3
[1964],
[1973]
(who
time to
[1975] improved the running
function, which grows even slower
Tarjan
Ackerman's
inverse
Fischer
and
obtained
Ullman
and
others.
among
by Galler
studied
Tarjan
that
and
4.3),
first
was
structure
functions
Design
an algorithm
to list
in
order
all the
keys in
a given
the pointers
a nonrecursive
Creative Exercises 87
^5
4.6
Let
to Heap may
at most one
Insert
Algorithm
so
algorithm
that
is
What
heap
many times up
(0(log/i)
elements
swap
swap
be
will
the
performed
Modify the
heap.
comparisons are
still
allowed).
4.7
4.8
Show the
4.9
Show
tree formed
AVL
tree
AVL
an
resulting tree
the tree, specify
an AVL
be made
the
whose deletion
a node
with
cannot
the numbers
inserting
by
and explain
node,
as a priority-queue
trees
AVL
tree
the
to 20
is the
What
in order.
in a
results
non-AVL tree,
resulting
the
that
such
or double)
rotation.
tree cannot be balanced
one (single
only
by
why
data structure.
Draw
with
one
rotation.
Exercises
Creative
4.10
of an abstract data
an implementation
Design
Insert(x):
In
other
This abstract
4.11
Another
now
the
Design
from
If there
most
in
the
data
assign
it to
same
element
y.
be removed.
of Exercise
once
4.10
in
in the data
each operation
already
duplicates.
type
at
Implement
for storing
a worker
in the
the
way: Assume
following
structure.
as before,
worst case?
but
What
with
is
now
that
every
insertion
An
must
pool data type (see Exercises 4.10 and 4.11) is the following: Assume
in the range 1 to /i, and that n is small
elements are identified
by integers
can allocate memory of size O (n). Each element
can appear at most once.
for insert and remove (as defined
in Exercise
4.10) that work in 0(1)
of the
variant
enough that
do.
x is
hold
and
Modify
the complexity of
average case?
4-12
element
if
type is
data
are generated
is removed. All
duplicates.
any
will
should
New jobs
job
be performed even
the data structure should
words,
Removeiy): remove
any element
Again,
supports
should
insertion
the
structure.
that
type
you
algorithms
time.
4J_3
node
O (log
(m +
Design
of sizes n
4.14
Design
n))
in
the worst
an algorithm
What is the
complexity
case.
that
contains
given
heaps.
88
4.15
Data Structures
x into
the key
insert
lnsert(x)\\
find
Find_Next{x):
should take
operations
the data structure.
these
key
only
structure (if
0(\\ogn) time
is not already
if it
than x.
is greater
that
worst case,
in the
there.
is there).
it
structure
the data
in
supports
structure
the data
the smallest
that
type
the data
Delete(x):
All
of an abstract data
an implementation
Design
n is
where
the number of
elements in
4.16
of an abstract data
an implementation
Design
insert
Insertix):
Find_Smallest{k):
find
should take
operations
the data structure.
these
only
structure (if
the data
key
0(\\ogn) time
in the
is not already
if it
it
there.
is there).
structure.
the data
in
supports
structure
the data
Deleteix):
All
x into
the key
that
type
worst case,
n is
where
the number of
elements in
4.17
of an abstract data
an implementation
Design
Insert(x): insert
are
All
structure (if
&th \"right\"neighbor
the
that
the data
larger than
is not already
if it
it
there.
is there).
smallest
the keys
among
key
.v.
time in the
0(\\ogn)
supports
the key
Delete(x):
that
type
n is
the number of
elements in
*
algorithms
that were
implementing
presented in
Torepresent
(with
algorithms
only
4.3.4 require
Section
three
values
a slight
balanced factors
we need 2 bits.
modification)
with
only
three
with
for
a method
Suggest
1
bit per
extra
node.
4.19
take two sets, such that all the keys in one set are smaller than
set, and merges them together.
Design an algorithm to concatenate
time
should
binary search trees into one binary search tree. The worst-case running
O (/?), where h is the maximal
of
two
the
trees.
height
concatenate
the keys
4.20
Design
valid
height
in
all
operation
the other
an algorithm
to concatenate (as defined
AVL tree. The worst-case running
time
of the two trees.
4.21 Consideran
in Exercise
should
be 0(h\\
where
is the
into
two
be
one
maximal
and deletions.
a fairly random sequence of insertions
by
a
balance factor appears
the same probability
with
(namely,
Prove that the average length of the path from the
probability of 1/3 for each possibility).
critical node to the place of insertion is a constant
of the size of the tree.
independent
Assume
4.22
4.23
that
AVL
tree formed
each
possible
of the
AVL tree
formed
by
inserting
the numbers
to n
tree?
That
is, construct
an AVL tree of
height
h with
the
minimal
in
4.24
4.25
number of
nodes. Usethis
the maximal
height of an
Let 7, and
at most In
A
AVL tree
worst
AVL
tree with
Theorem
(Hint: Try a
n nodes.
having
equal to
Prove
it is
that
regarding
sufficient to apply
T2.
vertices of the
the
and D=E\\<jF
\\^jVxU (namely,
from each vertex in V to each vertex in U). Suggest
to be performed efficiently.
allows join operations
4.26
G = (V,
join
that
to prove
nodes.
89
Exercises
Creative
a good
representation
for
that
graphs
[s\\,s2,..., sm) be a very large set, and assume that S is partitioned into k blocks.
block
that you have a procedure called which
such that given an element
s,,
block works in constant time
whichblock(Sj) = number of the block that contains s,; which
to all street addressesin the United States, and the blocks may
(e.g., S may correspond
You
want to maintain a small subset of S, T, and to perform
the
to
codes).
correspond
zip
T:
on
operations
following
Let S =
Assume
Insert(Sj).
Delete(Sj).
in T
elements
should
belong
take O
(logn)
Delete block
T.
in
that
to block
time
j.
in the
worst case,
n is
where
the
(disconnects)
one of them.
every
removes
only
elements from the data structure; it need not physically remove each and
Both m and k are too large, so you cannot afford to use a table of size m or k.
relatively small, and you can use O (n) space.
n is
However,
4.27
Let A [\\..n ]
be an
array
of real
numbers.
to perform
algorithms
Design
any
of the
sequence
Add{i,
add the
Notice
4.28
that the
y to
the
/'th
number.
Partial_sum(i)\\
only
value
first
fixed
number of elements remains
values. Each operation
of size n as a work space.
are to the
changes
/ numbers,
\302\243[/'
i
].
or deletions);
steps.
can
You
the
use
Extend
deletions.
Partial
value
The
worst-case
operations.
return
sumiy):
is less
than
running
y,
the sum
of all
the elements
currently
in
the
set whose
\302\243a,.
\342\226\240V|<V
time should
still
be
0(n\\ogn)
for
any
sequence
of
O(n)
90
4.29
Data Structures
a data
Design
structure
operations
following
Find_value(x):
to
maintain
a set
find
be supported:
the value associated
a key
should
with
the
element
(nil
if x
is not
in
the
set).
Insert(x, y).
Delete(x).
Add(x,
y):
add the
value
y to
the current
value
4.30
of the
the
each
element
elements
of these
with
key x.
set.
operations.
in the
case
/' of
1:
Statement(l);
2:
Statement(2);
4:
Statement(3);
256:
Statement(4);
65535: Statement(5);
CHAPTER
ALGORITHMS
OF
DESIGN
BY INDUCTION
invention,
than
is more
Nothing
the
which
G.W.Leibniz (1646-1716)
Invention
breeds
invention.
R.W.Emerson
5.1
In
(1803-1882)
Introduction
this
we introduce
chapter,
our
approach
to algorithm
design
using
the
analogy
to
We
include
relatively simple examples, and present the basic
method
is based. The analogous induction
the
techniques on which
have
been described in Chapter
When appropriate, we repeat the
2.
techniques
discussion
here to make this chapter
self contained.
Mathematical induction
is based
we have a
on a domino principle. Imagine
that
line
of upended
and that we wish to knock down
all of them by knocking
dominoes,
down
all dominoes
the first. To make sure that
will fall down,
we need only to
only
one
and
first
that
the
that
each
we
have
domino
will
the next one as
verify
pushed
topple
it falls.
We need not collapse the whole
we
time
add
a
new
domino
to
arrangement
every
work.
will
The
can
that
the
same
new
be
to
verify
arrangement
principle
applied
mathematical
principles
algorithm
induction.
and
design.
ai
Designof
92
by Induction
Algorithms
With
in mind, we
principle
(or to a
problem
set of smaller
The
problems).
on reducing
concentrate
should
problem
possible to solve a
to every
(2) a solution
of smaller problems (the
solutions
from
to solve the
it is
is
trouble
that
it is
to facilitate
chapter, we present severaltechniques
were chosen not because
of their importance
this process. The examples
chapter
but because they are simple and yet they
(some of them have limited applicability),
we want to emphasize. We will present
numerous other examples
illustrate the principles
the
book.
of this approach
throughout
to reduce
a way
the
In this
problem.
this
in
5.2
a simple
with
start
We
Polynomials
Evaluating
\342\200\224
algebraic problem
a given
evaluating
polynomial
at
a given
point.
Given a sequenceof
Problem
The
and
+
This problem
We
problem.
it
of the
an, an_\\,...,
a0,
an inductive
candidate
for
to a
natural
a \\,
that induction
the most
with
start
to be
seem
show
will
to better
lead
that
not
may
Nevertheless,we
numbers
real
approach,
approach.
to the
very good
then find variations of
solution
solutions.
The problem
of a
problem
of evaluating
The
numbers.
n+2
involves
in terms
inductive
is to
approach
solve
this
with
the
Pn-\\(x)
This is
it
by
polynomial
+ a\\X+a0.
+tf\342\200\236-2-*\"~2+
problem,
that it has
except
we can
Therefore,
solve
induction.
Induction
by
We can
solve
the
smaller
'\342\200\242\342\200\242
an-\\xn~{
same
the
to a
solution
the
now
the base
input
use
hypothesis:
an_\\, ...,
the
case,
We
a\\,
hypothesis
which
know
a0,
how
at the
to solve
is computing
to evaluate a polynomial
point x (i.e., we know how
the
by
problem
a{); this is
trivial.
induction.
Then,
represented
to compute
First, we have
show how
we must
to
to
5.2
solve the
problem
is the
compute xn,
At this
it by
multiply
Pn{x) = Pn_x{x)
the
Although
multiplications
the
This induction
(since
to compute
x\"~l
by
xn, then
that, even
not
is
it
return to
coefficients
an,
problem becomesthe
is
an_{,..., a {, which
an is
that
(Notice
on.) So we
have
now
a new
the
evaluation
polynomial
This
hypothesis
how
the
way.
straightforward
first
coefficient,
polynomial represented
by
the
an_{
is
the
(a?
\342\200\224
2)th
and so
coefficient,
induction hypothesis.
represented
to compute
is more
by the coefficients
suited
P'n-\\
to
to compute Pn(x)
following
expression:
from
how
know
a,nan_{
to evaluate
the point
,...,#
\\ at
it
is easier
the
x (i.e.,
(x)).
our
different
\342\200\242\342\200\242\342\200\242+*!.
l)th coefficient,
(n-
It is
a better
However,
implement.
of the
need
n additions.
in yet another
to
addition
since we
strong,
and
= anx\"-{+an_{x\"-2+
P'n_{(x)
later.
point
only one
one
then
to
is easier
it
perform
an xn,
too
not
hypothesis
this
induction
using
by
The smaller
a{).
to get
It is efficient,
We discover
Reducing
it
though
We will
overall.
work
induction
to
need
We
hypothesis is
there are 2n multiplications
the
xn~\\ but
computing
induction
Overall,
the
xn~x.
to compute
(The
anyway.)
note
exists.
but
step,
requires
use
now
the value of
to compute
how
know
how
since it requires
to compute x\.")
easier
computation.
measures.
all
algorithm
We
is stronger,
to compute
the
to less
It
We
additions.
solution.
a better
hypothesis:
we know
now
is
it
to complete
it leads
and
improvement
hypothesis
extend
interesting
preceding
In a
written.
is
it
efficient.
not
is
it
multiplications
multiplication
as
left
\342\200\224
it
powers
induction
Stronger
polynomial
to
correct,
+ 1)/2
obtain
the
by
implied
to
right
is frivolous
problem
The
by including
to
in this
of our approach.
is
algorithm
differently
from
polynomial
the power
+1=a?(a?
We make the
computation:
the
\342\200\242\342\200\242\342\200\242
+
a little
case; simply
(x):
use of induction
a very simple solution. The algorithm
+n-\\+n-2
induction
to Pn_{
result
in this
smaller
the
to
solution
step is straightforward
it may
point
discussion is merely
evaluating
moment, however, we will see
n
the
aid of
the
with
+ anxn-
complicates
just
(computing Pn(x))
value of Pn_\\(x)). This
problem
original
(which
93
Polynomials
Evaluating
purposes,
only
P'n-\\(x).
one
The
because
to extend.
and one
multiplication
complete algorithm can be
addition
Clearly,
are
describedby
Design of Algorithms
94
anxn+an-xxn
This
(It
+tf1*
mentioned
also
evaluate the
\342\200\242
\342\200\242
\342\200\242
is known
algorithm
was
polynomial
as Horner's
rule after
the
English
a=a0,a^a2,
= (('-((anx+<*n-\\)x+<tn-2)'
+ tf0
1981],
mathematician
467.)
page
W.G. Horner.
The program to
(a, x);
PolynomialJEvaluation
Algorithm
Input:
Induction
by
..-,an (coefficients
of a
and x (a real
polynomial),
number).
Output:
P (the
value of the
polynomial
at x).
begin
for i := 1ton do
p :=x*P+an-i
end
Figure
5.1
Algorithm
The algorithm
Complexity
requires
the
memory location. Even though
it worthwhile
efficient, we have found
algorithm
faster
than
only
Polynomial^Evaluation.
n multiplications,
additions,
but
also
very
and one
extra
and
very
is this
simple
Not
its corresponding
only
program is
simpler.
Comments
Induction
allows
us to concentrate on extending
solutions
of smaller
to
those of larger problems.
that we want
to solve P (n), which
is a
subproblems
Suppose
on a parameter n (usually
its size). We start
with
problem P that depends
an arbitrary
instance of P(n\\ and try to solve it by using the assumption
that P(n -1) has already
been solved. There are many possible ways to define the induction
and many
hypothesis
to use it. We will
several
of
these
and
their
possible ways
will
show
methods,
survey
power in designing
algorithms.
This simple example illustrates
The
the flexibility
we have when
we use induction.
trick
that
led to Horner's rule
was merely
the
from
left
to
instead
considering
input
right,
of the intuitive
is comparing top down
versus
common
right to left. Another
possibility
bottom
is involved).
It is also possibleto go in increments of 2
up (when a tree structure
(or more) rather than 1, and there are numerous
other
Moreover, sometimes
possibilities.
the
best
induction
sequence is not the same for all inputs. It may be worthwhile to design
an algorithm
We will see examples of
just to find the best way to perform the reduction.
all
these
possibilities.
5.3
Maximal Induced
Subgraph
95
5.3
problem.
following
and
disciplines
Consider
different
decide
whom
to invite?
This problem corresponds to the
An induced
Let
be
an
undirected
G
=(V, E)
problem.
following
graph-theoretic
graph.
of
is
a
such
that
V
and
F
includes
all
G
i/c
F)
H=(U,
graph
subgraph
edges in E both of
vertices
are in U. A degree
of a vertex is the number
of vertices adjacent
whose incident
The vertices of the graph
to the scientists, and two vertices are
to that vertex.
correspond
for the two corresponding
scientists to exchangeideas.
connected if there is a potential
to
a
subset
of
An
induced
the
scientists.
subgraph
corresponds
the
How
conference.
induced
an
find
induced
subgraph
undirected
//=(\302\243/,
degree > k
(in
graph
of G
F)
//),
exists.
approach to solving
vertices are
an
subgraph
of H have
all vertices
A direct
Given
Problem
The
k,
do you
this
problem
is to
remove verticeswhose
degree
is <
k. As
reduced. When
removed. The
is a
\"proof\"
the
by
with
their
of a
degree
of removals,
induction.
adjacent
96
Design of
Induction
all of
hypothesis:
vertices
whose
Induction
by
Algorithms
We
to find
how
know
> k,
have degrees
provided
is < n.
We
that this
to prove
need
for
a base
case,
and
that
its truth
for
- 1
implies
that
the conditions of
satisfies
edges
n-\\ vertices
without
n.
for
truth
all the
then
the
affecting
Therefore,
problem.
the conditions
of
the
theorem.
we can
After
remove v
and
its adjacent
v is
and,
by the
done.
Finding
Let/be
to another
n. We
function
One-to-One
that
maps a
finite
Mappings
set
A into
elementof
A is
mapped
element
function.
of ,4).
A mapping
5.2
Figure
6,/
- 6
7^^
.7
Given a finite
itself
(both
sides represent
Mappings 97
same
the
set).
The
set
A and
maximal.
induction
straightforward
then
one-to-one,
Induction
the whole
If, on
hypothesis.
hypothesis:
We
know
how
to solve
the
for
problem
sets of
n-\\
elements.
The
base
case
itself,
and we are
which
is a
one
element
Assume
satisfies
now
in the
that
set, then
it must
we have a
the conditions
of
the
set
problem.
be mapped to
n elements
We claim
of
Design of Algorithms
98
that
words,
an element
cannot
be
in
is such
n
Otherwise,
5.)
which
elements,
say, k
therefore,
remove it from the
into itself;
/maps
the
by
is not
belong to S. (In
connected to any
elements, then
cannot
mapping
We
set.
now
other
edge,
are
If there
k elements
those
be one-to-one.
a set
A'=A-{i) with
we know
how
to
have
induction
hypothesis,
the mapping is one-to-one,
then
exists,
it cannot
which
diagram,
S has,
the
solve the
the
k-\\ elements;
we simply
/, then
an
element mappedto
right side of
if / e S and
in the
at most
mapped into
has no other
/ that
element
any
Induction
by
are
we
and
done.
remove
/. We proved that
/ cannot
is that we must
The essence of this solution
Once
we
remove
an
and
of
induction:
element
reduce
is
the
This
S.
to
strength
belong
done.
We
have
to
be
that
the
reduced
we
are
the
careful,
however,
the size of
problem,
The only condition
problem is exactlythe same (except for size) as the original problem.
is still
function
on the set A and the
/ was that / maps A into itself. This condition
that
was mapped
to /. The
for the set A-[i}, since there was nothing
maintained
when no more elements can be removed.
terminates
algorithm
step,
we found
a recursive
as
algorithm
that no other
such
element
an
the
described
We
Implementation
procedure.
element is mapped to
it,
removed
In
each
it, and
Complexity
put on
the
queue
from a
without
was
The
of steps
number
is
operations.
in
removing
thus
O (n).
Every
the
element
function
be
can
from
an element
the
eliminating
an element
Because
choice
is mapped is natural.
element
The
5.5
the
and
the conditions
changing
that
requires 0{n)
steps involved
part
once,
In this
Comments
elements
at most
constant
take
queue
initialization
The
next
Problem
Celebrity
example
is a popular
exercise
in
algorithm
design.
It is
a nice example
of
problem
that
has a
solution
that
does
celebrity will
answer.)
The
goal is to
minimize the
number
of questions.
Input: /(an
of
array
S (a
Output:
(f,
Mapping
Algorithm
);
whose values
integers
subset of the
99
from
integers
1 to
n, such
1 to
n (
that
/is
n).
on S).
one-to-one
begin
from
S := A; { A is the set of numbers
:=
:=
1
ndo
to
0;
c[j]
for j
for j := 1to n do increment c [f [j ]];
for j
:=
1 to
n do
ifc[J]=0
while
then
remove/ from
S:=S-{i);
j in
Queue;
empty do
top of
the
the queue;
c [/\"[/]];
decrement
if elf
put
is not
Queue
]] =
0 then
put
/ [/
in
Queue
end
5.3
Figure
is potentially
l)/2 pairs of persons,there
are asked arbitrarily.
worst case, if the questions
the worst case.
a need
n(n-
We can
the
use a graph-theoretical
to
vertices
Mapping.
Algorithm
It is
to ask
We
formulation.
that
we
a directed graph
person A to person
persons
corresponding
knows B. A celebrity
to a sink of the graph
(no pun intended). A
corresponds
vertex with indegree
that
a graph can have at
n -1 and outdegree 0. Notice
sink. The input
to the problem
(whose
corresponds to an n x n adjacency matrix
is 1 if the /th person knows the yth person, and 0 otherwise).
The Problem
there exists
/7th
entry)
an
entry) are
are 0.
Given
the
an
n xn
/ such
1, and
all
adjacency
entries
the entries
in
the
in
the
/th
matrix,
determine
in
with
B if A
sink
most
is a
one
ij entry
whether
(except for
row (except for the //th
/th column
the
in
do better
can
can build
an edge from
and
n(n- 1) questions,
not clear
the
is simple.
case of two persons
Consider as usual
the difference
between the
problem with n -1 persons and that with n persons. We assume that we can find the
Since there is at most one celebrity,
the first n - 1 persons
by induction.
celebrity among
is among the first a?- 1, (2) the celebrity is
there are three possibilities:(1) the celebrity
the
rtth person,
and (3) there is no celebrity. The first case is the easiest to handle. We
The base
need only
to check
that the
nth
person
knows
the celebrity,
and
that
the celebrity
does
not
of Algorithms
100
Design
know
the Mh person.
whether
2(/7
mh
the
in the
questions
1)
The
are
then
consideration,
need
is probably
it
step,
avoid).
We
consider
the
to
easier
we reduce
to eliminate
of
number
to determine
because,
2(n
- 1)questions. If
will be
questions
ask
we
n{n-\\)
approach.
It may be
''backward.\"
problem
to identify
hard
someone
identify
celebrities.
of
size
difficult
to ask
need
another
need
than
the
more
are
the total
then
noncelebrities
more
definitely
not
is to
here
trick
but
celebrity,
mh
we tried to
is what
(which
is the
person
\342\200\224
Induction
by
someone specific;anyone
do.
will
Suppose
Alice
ask
we
that
she cannot
be a celebrity; if she does not, then
If she does, then
Bob.
whether she knows
of
We
can
eliminate
one
them
with
one question.
a
be
Bob cannot
celebrity.
we
with
which
the
three
cases
started.
We do not just take
We now consider again
We use the idea in the last paragraph
to eliminate
an arbitrary person as the mh person.
solve
the problem
for the other
n -1
either Alice or Bob, then
persons. We are
not occur,
since the person eliminated
cannot
be the celebrity.
that case 2 will
guaranteed
- 1
is
there
no
the
n
Furthermore, if case 3 occurs \342\200\224
namely,
celebrity among
persons \342\200\22
1
the n persons.
case
this
case
is easy.
but
there is no celebrity among
then
remains,
Only
it takes two more questions to verify
that
If there is a celebrity among the n - 1 persons,
is
whole
set.
there
is
no
this
a celebrity for the
Otherwise,
celebrity.
The algorithm
as follows.
We ask A whether
she knows \302\243,
and eliminate
proceeds
either A or B accordingto the answer.
Let's assume that we eliminate
A. We then find
the remaining
If there is no celebrity, the
n - 1 persons.
(by induction) a celebrity among
we
check
A
that
knows
the
terminates;
otherwise,
algorithm
celebrity and that the
does
not
know
A.
celebrity
more
Complexity
phase
most
At
to eliminate
-1
candidate is indeed a
n (n
- 1)(the
identify
a priori
number
a celebrity
questions
3(a?-1)
and
persons,
Notice
celebrity.
of
of entries
by looking
at
the
only
be sensitiveto
be
will
at
then
most
that the
matrix).
O(n)
each
n -1 questions in
- 1) questions
to verify
asked:
2{n
size of
This
the
solution
input
shows
is not
that
but
\302\253,
it is
matrix,
first
the
that the
rather
possible
even
though
to
The
5.5
Input:
Problem
101
(Know);
Celebrity
Algorithm
Celebrity
).
celebrity.
Output:
begin
i :=
1;
j:=2;
3 ;
:=
next
{in
while
the
all but
we eliminate
phase
first
ifi
one
candidate
+ 1 do
< n
next
:= j
candidate
else
candidate := i;
that the
check
we
Now
Know [candidate,
{a
if Know
if not
if
if
to pass
the test}
wrong
else celebrity
end
and k<ndo
wrong
[candidate,
k:=k + l
not
celebrity
candidate]:=false ;
variable
dummy
not
while
candidate is indeedthe
wrong .-false
k := J ;
then
:=
celebrity
0 { no
:= candidate
celebrity }
from
spent.
to
Celebrity.
in
this
of Algorithms
102
Design
5.6
A Divide-and-Conquer
Induction
by
The Skyline
Algorithm:
Problem
have
we
far,
So
with
deals
example
Given
Problem
The
these
in a
buildings
rectangular
An
the
exact
locations
draw
the
in
Fig.
and shapes
(in two
skyline
of several
dimensions) of
lines.
hidden
is given
of an input
example
shapes.
city,
eliminating
buildings,
computation. This
and numerical
theory
graph
5.5(a);
the corresponding
in Fig.
is given
output
5.5(b). We are interested in only two-dimensional pictures. We assume that the bottoms
lie on a fixed line (i.e., they
share
a common horizon). Building
of all the buildings
Bt is
represented
by a
and
the
in
5.5(a)
Fig.
(2,6,7), (3,13,9),
(1,11,5),
denote
Rt
the
height.
building's
to
correspond
and
the left
them arranged
connecting
heights
the buildings
example,
//, denotes
and
respectively,
building,
coordinates
/?,). L, and
//,,
(L,,
triple
the
order
in
of the
x coordinates
right
A skyline is
from left to
list
right.
of
For
input:
following
and
(12,7,16),(14,3,25),(19,18,22),
(23,13,29),
(24,4,28).
(The
numbers
in
boldface
type
are the
in Fig.
5.5(b) is
represented
as follows:
(1,11,3,13,9,0,12,7,16,3,19,18,22,3,23,13,29,0).
(Again,
the numbers
The straightforward
time to the skyline.
know how to solve
in
The
the
algorithm
are heights.)
for this problem is
based on adding
induction
hypothesis is
simple
boldface
type
problem
for
-1
the
and
buildings,
one.
then
one
building
We assume
we add
the
nth
that
at a
we
building.
15
10 15 20 25 30
15
(a)
20
15
10
(b)
skyline
problem:
(a) The
input,
(b)
The
skyline.
25
30
5.6
The
for one
is trivial
problem
intersectit
from
to
left
to right
x coordinate
skyline
appropriate
the one
5 is
\"covers\"
a building
add
To
building.
the existing
with
from
this example).
to 9, and its height
we need to
We first scan the
(i.e., we search for the
the horizontal line that
the skyline,
Bn be (5,9,26).
Let
left side
the
where
find
\342\200\224
5 in
to
Bn
103
Problem
The Skyline
Algorithm:
Divide-and-Conquer
of
fits
Bn
this
In
is 13.
case,
We can
scan
now
the
skyline,
skyline
by
(1,11,3,13,9,9,19,18,22,9,23,13,29,0).
This
algorithm
scan for
is clearly
correct,but
requires
0{n)
To
called
the performance of
and conquer. Insteadof
for n
solution
this
improve
divide
(Again,
it is
steps.
In
the
worst
of
steps
case, the
be
will
\342\200\242\342\200\242\342\200\242
= <9(\302\2532).
+<9(1)
+ 0(/i-1)+
0(/i)
the
Bn
the base
- 1 to a solution
case of
one
for
building
we
algorithm,
use a
is
well-known
technique
principle of
extending
n/2 to a solution
trivial.) Divide-and-conquer algorithms
for
n.
divide
if
Therefore,
combine
the
we
divide
solutions
in
the
of
two
into
problem
equal-sized
linear
0(n\\ogn).
examples
then
in
see
time
many
it.
Bn
10
15
Figure5.6 Addition
of a
building
15
20
skyline
25
30
of Figure
104
of
Design
The
key
in this
is the
the divide-and-conquer
algorithm
example
with
the
it takes linear time to merge one
case,
building
In
same
we
about
the
time to merge two different
time,
skylines.
behind
idea
in the
that,
observation
by Induction
Algorithms
worst
technicalabout
this
required, consider
to get
try
Always
algorithm includes a
to a more complicatedpart of the problem.
The
is so useful
is that it uses the combine
approach
step to its
in Section 3.5.2 cover the most
common
dividegiven
should
memorize
these recurrence relations.
this
a binary tree
Let
T be
the
farthest
leaf
step
Balance
Computing
with
the
down
is nothing mysterious or
step that is more general than
There
money.
your
If the
principle.
applying
5.7
more for
root
r.
tree.
The height
The
balance
and
the
difference
labeled with
factor.
between
numbers
representing
hlb, where
10
15
Figure
is the
15
5.7 Merging
node's
25
20
two
skylines.
height
and
30
b is
its balance
5.7
Figure 5.8
balance
in trees
nodes
(by
now know
rather on
their
the
know
the
problem
the
within
original
Stronger
induction
of all
the base
Again,
factor
Furthermore,
the two
The
computing
be an
all
of
remain.
that
on only
of
the root,
we remove
n > 1 nodes,
with
depends
We
chose to
then
remove
node.
The root's
nodes, except
but
factors of the root's
children,
induction does not work in this case. We need to
the height-finding
the root. The idea is to include
for
root.
the
balance
the
on
hypothesis:
We
that
trees
in
trivial.
difference
the
the
to compute
height
balance factors
nodes.
we consider
when
Now,
how
know
have <
the
root,
we can
determine
its
children.
of
height
1.
plus
is that it solves
a slightly extended problem. Instead of
key to the algorithm
turns
only balance factors, we also compute heights. The extended
problem
easier
one
to solve,
simple
easily by calculating
we can also determine
children
balance factors
hypothesis.
problem:
nodes
case is
to compute
of all the
factors
children
the
and heights
balance
out to
of
the
induction
subtrees
factor of a node
Hence,
height.
heights
tree
the two
for
balance
the
how
Given a
induction)
the balance
because
root
nodes, compute
straightforward
the
with
know
the
height and b is
nodes.
< n
have
that
n = 1 is trivial.
the problem
solve
T with n
tree
inductive approach
Induction hypothesis: We
We
is the
105
Trees
Binary
the nodes.
of all
factors
the regular
use
Given a binary
Problem
The
the
in
factor.
balance
We
The numbers
tree.
binary
Balance Factors
Computing
is extended),
then
because the
easier.
of
the
With
a larger
induction
heights
induction,
problem.
step
are easy to
we need
If
may be
the
compute. In many
only to extend a
solution
is broader
easier, since we
have
cases,
solution
(because
more
with
Design of Algorithms
106
which to work.
is a
It
by
5.8
The
one
forget
should be
is from Bentley
problem
there
that
are two
computed separately.
different
in
parameters
will
We
several
present
book.
in the
Consecutive
the Maximum
Finding
following
error to
common
Induction
[1986] (it
also
appeared
Subsequence
in Bates and
Constable
[1985]).
The Problem
necessarily positive)
negative,
then
subsequence is 0).
the sequence
The
in
subsequences
order
only
like to
Induction hypothesis:We
sequences of size <n.
If
n =
1, then
nonnegative,
the
maximum
or
the
is empty (by
have
an
definition,
that
algorithm
the
solves the
sum
sequence
(2,
is 3.5.
There
numbers
are
of the empty
problem and
reads
once.
induction
straightforward
subsequence
would
We
numbers
the
(not
elements.
maximum
the
maximum
the
of
a maximum
a subsequence
xh
subsequence
of consecutive
subsequences
We call such
find
the sum
that
such
elements)
of real numbers
jc,, jc2, -~,x\342\200\236
a sequence
Given
hypothesis
is as
how
to find
know
subsequence
empty
S = U,,*2, \342\200\242\342\200\242\342\200\242,-*,,)
of size
n>\\.
consists
of
follows:
the
the
maximum
single
otherwise.
subsequence
in
subsequence
number
if that number is
Consider
sequence
how
to find a maximum
By induction, we know
If that maximum
subsequencein S' = (x,, x2,..., *\342\200\236_i).
subsequence is empty, then all the
numbers in S' are negative, and we need to consider
only xn. Assume that the maximum
found by induction
in S' is S'M = (xh xi+l,...,
subsequence
jcy), for certain / and j such that
1
1. If j = n- 1 (namely,
the maximum subsequence is & suffix),
then it is easy
^i^j<n-
to extend the
maximum.
solution
to S:
However,
if j
\\i
xn
< n-
is positive,
1,
then
there
then
are
extends
S'M\\ otherwise,
S'yv/ remains
o m remains
two possibilities. Either
is not maximum in S\\ but is maximum
it
5.9
ends
\342\200\224
that
n -1
at
hypothesis
of S'.
a suffix
is,
the
include
to
the Induction
Strengthening
Suppose that
of
knowledge
we
the induction
strengthen
maximum
the
107
Hypothesis
denoted
suffix,
by
<n,a
a
If we
know both
maximum
It
could be
subsequences, the
global
of size
sequences
that is
subsequence
set as
The algorithm
maximum
new
done
need to find
the previous
xn to
to the
maximum
we have
we retain
We also
add
simply
xn
then
Otherwise,
suffix).
yet.
We add
subsequence,
the
new
previous
maximum
new
the
suffix.
maximum
It
maximum
the
that
empty
clear.
becomes
algorithm
than the
is more
sum
the
(as well as
subsequence
We are not
subsequence.
is not true that we always
maximum
suffix.
in
suffix.
suffix. If
the
induction
We know how to find,
hypothesis:
maximum subsequence overall,and the maximum
for
(X, n) ;
Maximum_Consecutive_Subsequence
Algorithm
of the
maximum subsequence).
begin
GlobalJAax := 0 ;
:= 0
JAax
Suffix
fori := J to n do
ifx U ] + Suffix
Suffix
.= Suffix JAax
+ Suffix
Suffix
else
+ x [i ]
:= Suffix JAax
GlobalJAax
else ifx [i ]
then
> GlobalJAax
JAax
JAax
JAax
> 0 then
+ Suffix
JAax
:=x[i]
JAax
:= 0
Suffix JAax
end
Figure 5.9 Algorithm
5.9
induction
the
theorems
mathematical
proving
often
the
Strengthening
Strengthening
the
encounter
can
hypothesis
In many
easier. That
is,
it
can add
by
jOonsecutive_Subsequence.
Induction
Hypothesis
is one of the
hypothesis
with induction. When
following
be denoted
cases, we
Maximum
scenario.
Denote
and
\302\253),
the proof
P (<
another
is easier to
call
assumption,
prove [P
and
most
attempting
theorem
the
important
techniques
an inductive
proof,
by
must conclude
it
Q](<n)
Q,
under
=>
that
The
P.
for
we
induction
P(<n)=>P(n).
which the
than
P(n)
proof becomes
it
is
to
prove
Design of Algorithms
108
by
Induction
/>( <
h)=> P(n).
The
in
the
will
more complicated
6.13.1, 7.5, 8.3, and
present
Sections
6.11.3,
5.10
Dynamic
that
Suppose
may be
the
silicon
chip,
hypothesisprecisely.
examples of strengthening
12.3.1
induction
hypothesis
Problem
The Knapsack
Programming:
in
others).
(among
different
as
and
The
the
we are
many
knapsack
it
induction
but
correct,
seems
assumption
is to include
Problem
that the
/th
item
sizes sum to
Given an
has
integer
an integer
exactly K, or determine
K and n
size kh
that
find
no
with
the
items.
one-dimensional
exercises,
and
in
Chapter
sizes
items of different
of the items
such subset exists.
a subset
Other
11.
such
whose
of items and K
K), such that n denotes the number
problem by P(n,
denotes the size of the knapsack. We will implicitly
that the n items are those
assume
in the
that are given as the input to the problem, and we will not include their
sizes
a
notation of the problem.
with the first / items and
Thus, />(/, k) denotes the problem
we
first concentrate
of size k. For simplicity,
on only the decision
knapsack
problem,
a solution
which is to determine whether
exists.
the straightforward
We start
with
We
denote
the
induction approach.
5.10
Induction hypothesis(first
The
Programming:
Dynamic
We know
attempt):
109
Problem
Knapsack
- 1,K).
case is
base
K,
also
but
easy; there is a
to P(n
is a solution
- 1,K)
for knapsacks
Induction
solution
\342\200\224
that
is, if
of all sizes at
hypothesis (second
most
K.
attempt):
We
know
how
to solve
P{n-\\,k)
forallO<k<K.
The
did not
reduction
previous
of K\\
value
depend on a particular
for
0<k
<K.
all
The
P{n, k)
it will
work for
any
k.
can use
be easily
this
solved:
exponential
of
may
not
hypothesis
a (trivial)
base
solution.
algorithm.
Fortunately,
kinds
to solve
problems.
be too
it is
The main
high.
In
fact,
we
introduced
the
notation
of
P{i,k)
especially
to
and K
demonstrate this observation.
There
are n possibilities for the first parameter
possibilities for the second one. Overall, there are only nK different possible problems!
The exponential running
of problems after every
from doubling the number
time
resulted
but if there are only
nK different
reduction,
problems, then we must have generatedthe
same
all the solutions and never
is to remember
problem
many many times. The solution
solve the same problem twice. This
a
of strengthening
the
is
combination
approach
induction
the assumption
that all
induction
(which is using
hypothesis and using
strong
solutions to smaller cases, and not only that for n - 1, are known). Let's see now how to
implement this approach.
We
all the known results in an n xK matrix. The (/, k)th
store
in the matrix
entry
contains the information
from the secondabout the solution of P(i, k). The
reduction
in the nth
attempt
hypothesis
computes the nth row of the matrix. Each entry
basically
row is computed from
two
of the entries above it. If we are interested in finding
the
actual subset, then
add to each entry
we
a flag
can
that indicates whether
the
item was selected in that step. This flag can then be traced back from the
corresponding
The
is given in Fig. 5.10.
and the subset can be recovered.
(\302\253,
K)th
algorithm
entry,
matrix for a given input.
Figure 5.11 shows the complete
of Algorithms
Design
110
(S, K);
Knapsack
Algorithm
Input: S (an
of size n
array
size of
K (the
and
Induction
by
the
storing
of the
sizes
the
items),
knapsack).
if there
Output:
to that
solution).
( See
Exercise
and a
belongs
this program.
improving
begin
P [090].exist:= true
1 to K do
fork:=
P[0,k].exist :=false
P
need to initialize
P
[0,0]}
computed from
is no
{ there
be
fori := J to
[/, 0]
for / >
1, becauseit
will
n do
fork:=0toKdo
; {thedefault
value
ifP[i-\\,k].existthen
:= true
P[i, k].exist
:=false
k].belong
P[i,
else if k-S[i]>0
then
[i ]].exist then
:= true ;
ifP[i-\\,k-S
k].exist
P[i,
5.10
Figure
\"I\":
*4=6
then
last line,
a solution
been found;
it 3 =5
An example
of four
*2=3
consists
5.11
Figure
*,=2
O
O
of
the
Algorithm Knapsack.
table
for the
constructed
\"-\":
there
no
solution
is no
has not
yet
been
10
\"O\":
12
13
14
15
16
symbols
found.
11
the
table
a solution
(If the
size.)
without
symbol
input
this
appears
item
has
in the
The method
programming.
we
used is
just
of a general
instance
an
programming is to
of dynamic
essence
The
111
Errors
Common
5.11
build
with
tables
large
all
known
example of dynamic
is presented
programming
There are nK
Complexity
from two other entries.
exponential
approach
determining
too
not
Hence, dynamic
is not too large.
running
this
constant
of the
not
but
this
small
can work
the
subproblems.
enough,
computation
when
problem
All
for
using,
(n) time.
can
possible
a large matrix.
maintaining
total number of possible subproblems
by
Errors
we briefly mention
section,
We
have
a recursive
error
problem for n
Changing the hypothesis
of it. A graph
example
of the
into
partitioned
two
of this
The
determine
accordingly.
typical
is effective
approach
programming
We do
programming
Common
algorithms.
instance
in
sizes
the
If
only if the
and
Even then, dynamic programming requires building
matrices,
large
in Fig. 5.10, it is
a large space. (In some cases, as in the program
usually
requires
at any moment.) The
to use less space by storing
only a small part of the matrix
times
are usually at least quadratic.
5.11
In
O (nK).
are computed.
subproblems
it
is computed
one
each
time is
flag in
belong
The dynamic
Comments
be reduced to several smaller,
possible
and
running
then K cannot
large,
actual
the
finding
example, the
thus
table,
total
6.8.
Section
items are
in
in the
entries
Hence,the
time
in
+ 1,
For
induction
example,
is to extend a
instead of
an
for
solution
arbitrary
induction to design
in Section
proofs
the base case
forgetting
essential
to terminate the
to a solution of a special
use of
the
errors in
instance.
is another
G = (V,
fact).
Problem
whether
it is
graph
the
G=(V,
vertices
\302\243),
of Algorithms
Design
112
A wrong
the
to
Induction
by
The main
used. Had
that does
if possible,
graph,
If
to
want
cannot be
think
such
to
make
it
that we
form
can
find
other
simple
it
is included
unless
assumption
avoid
is sometimes very
tempting.
hypothesis
such
\"we know how to find
and such,\"
the
the
the hypothesis
changes to that black
changing
any
the
with
things
in
specifically
box,
same
effort.
the
induction
is to
unintentionally
think of
it
graph
similar
to
related
to redefine
(namely,
explicitly).
5.12
Summary
would
the
If the hypothesis
is
to
then we are tempted
But we cannot use any
One way
hypothesis.
as a black box. Do not
unless you
smaller
the
which
This problem has a better solution,
have
that
solution
to
the
reader
we
leave
connected;
(Exercise 7.32). For
being
of
this
common
see
Section7.5. A result
discussion
error,
example and further
in Exercise 5.24.
is included
this incorrect algorithm
something of
is that
illustrate,
induction
and
instance,
been valid.
Changing
by
to
is connected
disconnect
not
we
the one
the
not
a vertex
removed
we
both
and
solution,
the
blue.
is connected
the
vertex
of
subset
second
the
to vertices from
is connected
(since
bipartite
the rest
partition
subset. If v
the blue
it to
v
and
red, and
subset
first
algorithms.
The rest of
presented
in
can
\342\200\242We
of
this
achieved,
and
main
idea
to one
learn
is devoted
numerous
to precisely
that
purpose.
The
principles
follows:
of
a problem
The
are as
chapter
to
way
book
this
same
methods
known
techniques
The best
chapters.
following
problems.
Additional
all the
the
of
variations
or
induction
more
of
to design
size.
smaller
algorithms
If the
by reducing
can
reduction
an
instance
always
be
base
the
is to
directly.
\342\200\242One
of the
elements.
easiest
That
ways
technique
to
reduce
the size
should
be the
of a problem is to
first
line
of attack.
eliminate
The
some
elimination
of its
can
do not
to simply eliminating elements that
many forms. In addition
clearly
contribute (as in Section
into one, to find
5.3), it is possible to merge two elements
elements
that can be handled
a new element
by special
(easy) cases, or to introduce
that takes on the role of two or more original
elements
(Section
6.6).
\342\200\242We
can reduce the size of the problem
in many ways. Not all reductions,
however,
As a result, all possibilities for reductions should
lead to the same efficiency.
be
In particular, it is worthwhile
considered.
to consider different
for the
orders
take
Bibliographic
induction
element
We
sequence.
first. Sometimes,
of starting from
examples
the
middle
it is
where
to take
is better
it
the
6.2).
(Section
element
smallest
113
Reading
We also
see
will
largest
will
see
of
examples
induction on trees in which the root is removed first (top down), and examples in
which
the leaves are removed first (bottom
up) (Section 6.4.4).
of the most efficient ways
to reduce
the size of a problem is to divide it into
\342\200\242One
(or more) equal parts. Divide
can be divided such that the output
two
for
output
Sections
\342\200\242Since
whole
the
6.4, 6.5,
a reduction
itself, we
and
works
conquer
of the
the
if
problem
generate
algorithms
Divide-and-conquer
problem.
effectively
are
given
the
in
9.5.
the
size
of the
problem,
but
not
the
problem
as possible.
subproblems
of finding some ordering among several items
For example, the problem
can
be
reduced to finding
the item that is first in the order; the relative
(and
removing)
of the first item (see Sections 6.4 and
is independent
order of the rest of the items
for smaller
look
should
are
that
as independent
7.5).
is one
\342\200\242There
be
must
important
very
the
hypothesis
the complete
\342\200\242
Finally,
identical
(see
algorithm
we can
Section
6.10).
hypothesis,
the
approach
divide-and-conquer
so that the
different
or in
subproblems
various
with
become
combinations.
For
strengthening the
to combine
easier
(see Section8.4).
Bibliographic
Reading
method
by no
Burge [1975]and
is
Paull
The celebrity
to save
possible
verification
phase,
[1988]).
problem was
an
additional
questions
Thomas [1982]).Strengthening
Polya [1957]calls this technique
suggested
|_l\302\260g2wJ questions
by Aanderaa
114
Designof
by Induction
Algorithms
or prove,
is
that
something
stronger).
introduced
was
Drill Exercises
5.1
evaluation.
algorithm for polynomial
your algorithm require? Can you think
a divide-and-conquer
Design
and
does
multiplications
How
many additions
this
of an advantage
5.2
5.3
Consider algorithm
end of the algorithm?
5.4
5.5
are
loop
space to
all
T is called
while loop in
that
efficient
algorithm
should
problem. Given a tree T, the algorithm
AVL tree. The answer
should
be only yes or no.
decision
5.6
algorithm
Modify
actual
subsequence
Maximum
Consecutive
and not
the sum.
5.7
Write a program to
5.8
In algorithm
Knapsack,
PV-Uj]).
If
Redraw
5.9
Fig. 5.11
of
solution
we
first
have
the
determine whether or
to a knapsack
problem
using the
that
the
enough
following
not
T is an
it finds
the
belong flag.
whether the
checked
is a solution
to reflect
problem
rest? How
of Exercise5.7?
the
with
i-\\
choice.
this
may
the solution
the policy
do not
to solve
A given knapsack
characteristics
the solution
recover
there
only
the nodes
Assume
at the
(Fig. 5.4).
Celebrity
algorithm
tree (see
AVL
an
are 0, 1, or -1.
factor. Design an
its nodes
balance
the
store
invariant
tree T.
a binary
given
factors of
balance
empty
Mapping
the appropriate
Write
You
steps of
have
obtained
does
many
from
different
solutions.
What
115
Exercises
Creative
Exercises
Creative
5.11 Supposethat
screen
there
efficient
an
is a rectangle
building
a blue
with
Design
extended
following
have roofs.
on a
are two different
(maybe
proposed)
skylines: One is projected
on the first one with
a red color.
color, and the other is superimposed
to compute the shape that will be colored purple.
In other
algorithm
of two skylines.
intersection
be a sequence
..., x\342\200\236
algorithm to
the
find
of
real
xh
subsequence
in it is maximum
numbers
over
all consecutive
an O
Design
elements)
such
(n)
that the
of
as 1.
positive).
(not necessarily
node if
are not AVL
an AVL
T that
its balance
nodes,
but
is equal
given
5.15
in
the adjacency-list
representation.
connecting
maximal
distance
diameter of a given
tree.
these two
over
all
The distance
vertices
pairs
between two
have
(neighbors
of vertices.
vertices
distance
Design a
linear-time
in G
1). The
algorithm
to rind the
tree.
5.16
in algorithm
Knapsack (Section 5.10). Is there
Improve the space utilization
complete n x K matrix? What is the space complexity of the improved algorithm?
5.17
Solve
a need
for a
to
are identical
of the knapsack problem: The assumptions
the following variation
those of Section 5.10,exceptthat there is an unlimited supply of each item. In other words,
the problem is to pack items of given sizes in a given-sized
knapsack, but each item may
appear many times.
variation
5.17 (n items,
of the
unlimited
assumptions
fixed-sized
as in Exercise
116
Design of Algorithms
5.20
Let x i, x2,..., xn
by
be a
set of
Induction
let S =
and
integers,
to partition
an algorithm
\302\243a,.Design
the
/=i
subsets of equal
(nS).
sum,
or determine
that
it
to do
is impossible
run in time O
should
box \342\200\224
you cannot see how it is designed
you are given an algorithm as a black
If
and an
the
has
numbers,
you input any sequence of real
following properties:
is
there
a
subset
will
or
whether
of
answer
the
\"no,\"
indicating
\"yes\"
algorithm
integer k,
to find the subset
how to use this black box
sum is exactly k. Show
the numbers whose
n is the size of
sum is k, if it exists. You should use the black box O (n) times
whose
(where
5.21 Supposethat
\342\200\224
that
the
5.22
sequence).
The towers of
Hanoi
solution.
recursive
puzzle
There
example of a nontrivial
sizes arranged
5.12).
(see
pegs,
Fig.
is a standard
are
that has a
problem
of different
disks
on
a peg
in
simple
decreasing
a. Design
an algorithm
towers of
b.
Hanoi
(by induction) to
for n disks.
a minimal
find
How
moves are used in
many
number of moves,and solve it.
your
algorithm?
Write
5.24
The
a nonrecursive
following
assume
is a
movesthat
Construct a recurrence
5.23
sequence of
the
solves
problem
towers
towers
that
is, prove
of Hanoi
problem (defined
of Hanoi
in Exercise
the
for
relation
exist
5.22).
no
for
that
all the
on one
initially
n disks.
Figure
5.12
The towers
of Hanoi
puzzle.
Creative
5.25
in Section
described
assignment
The problem
very
the
difficult
coloredwith
a. Prove
b. We
find
by
that
within
an
that trees
induction
a partition
connected
the
one subset.
bipartite.
to vertices
the
global
Consideragain
the
and
then
of all the
connections,
we
that maximizes
the
may
need
is bipartite).
are no
want to
We
edges connecting
wrong
5.11: We take
color the vertex in
in Section
with the
vertex
regardless of
are always
means
that the graph
the graph is a tree (which
of the vertices into the two subsets such that there
color
colors.
two
assume
vertices
some
approach
simple
Given
coloring:
graph
wrong
but also
wrong,
to the
5.11. In
is related
exercise
This
117
Exercises
an arbitrary
the
add
that,
best possible
a new
if we
That
way.
color only
color one
whether
color
it,
is, we
vertex is
time
You should
if the
vertex
colors.
up to 1 +log2/i
of colors for every
order
at a
of choosing
on
order
in
The
the
the
depend
following way.
algorithm
vertex
and starts checking the vertex's
edges. At that point, you
picks a vertex as a next
that the graph
are allowed to add edges incident to this vertex as you desire, provided
You
number of colors will be required.
remains a tree, such that, at the end, the maximal
which has
the algorithm,
cannot removean edge after it is put in (that would be cheating
this construction is by induction.
to achieve
seen
the edge). The best way
already
<k colors with few vertices, and build
Assume that you know a construction that requires
new vertices.
without adding too many
one that requires k + 1 colors
design
a construction
vertices. The
construction
can
number
CHAPTER
INVOLVING
ALGORITHMS
SETS
AND
SEQUENCES
is a
Order
on disarray
teaching
lovely thing;
it lays its wing,
simplicity to sing.
Anna
6.1
In
Branch
(1875-1937)
Introduction
this
we deal
chapter,
difference between
elements
not
Since
sequences.
which
sets
are either
is
in sets
finite
it
sequences
is not.
Also,
the order
in
sets. The
or finite
sequences
in
that
sets
in
we assume
the
that an
than
more
so
the
which
for
inputs are
integers, reals),
in
that
and
whereas
this
given order. Throughout
the input is assumed to be an
The elements in the sequences
problems
inputs
important
appear
Nevertheless,
sequences.
with
sequences
are given is
element does
(e.g.,
Hempstead
unless
chapter,
and
array,
or sets
that
elements
they
are
specified otherwise,
we assume
are assumedto
can
all
that
be
the
taken
In
be compared.
of the same type.
size
the
We
of
is known.
array
ordered
totally
chapter,
study
in the
representation
of the
from a
this
as
them
set
we consider
issues
such
as
data
and similarities
of sequences.
compression,
maximality, order, specialsubsequences,
different
This chapter contains
with
a
of
algorithms
many
applications.
variety
Our purposeis to give more examples of the design methodology
introduced in Chapter
some important algorithms. We include
to describe
5, and, at the same time,
algorithms
are
that
and universally
are very
important
very
and Sets
Involving Sequences
Algorithms
120
and
search
(binary
applicable
for
sorting,
have
but
stuttering-subsequence problem).
first
The
in many forms
comes
that
two
compression,
We end this
and
examples of
several
with
and
\342\200\22
sorting
statistics,
data
algorithms.
probabilistic
elegant algorithms
algorithm
discuss
then
\342\200\224
order
problems
text manipulation,
involving
problems
chapter
We
situations.
algorithmic
and elegant
\342\200\224
a basic
search
binary
in many
appears
extensively studied
the most
of
one
in this chapter is
example
interesting
illustrating
design techniques.
6.2
is to
search
Binary
immensely
question.
algorithms
cut
search
the
this
In
a wheel
what
rediscovered
it is
and
important,
is to
search
and Variations
Search
Binary
in half
space
mechanics:
is
It
The
basic
(or approximately
so)
frequently.
several
we describe
section,
is to
of
variations
binary
elegant, and
idea behind binary
simple,
by
one
only
asking
and show
search
its
versatility.
Pure
Search
Binary
The Problem
that
<xn.
<jc2<
jc,
Let A-,,*2
* \342\200\242
in the sequence,
z appears
such
be a sequence
of real numbers
,
z , we want
to find whether
real number
and, if it does, to find an index / such that
\342\200\224
xn
Given
Xj=Z.
For
we look for
simplicity,
interested
idea is to
in
search
the
cut
simplicity
that n is
sequence;
otherwise,
nil,
which
can
comparing
z to
number
to the
the
of
z is
element.
we
version
advantage is
half
the
second
xnl2+\\,
that
that
and
This version
we may be
In general,
one,
middle number.
is clearly in the
so
and
first the
z
then
half. Finding z
We handle
algorithm
.v,=z.
the largest
one,
by checking
by induction.
The
/ such
smallest
comparisons
is given
in
in
either
half
the
base
case
Fig.
6.1.
on.
Assume,
first
half of
The
for
the
is a problem of size
of n = 1 by directly
therefore,
of size n with
half;
comparisons
binary search delays the equality
equality with z in each step. The disadvantage of
there
is no hope for stopping
the
the
search
present is that
early;
is made in every step (instead of one equality
one
comparison
only
one inequality comparison). This searchis thus usually
faster.
Although
comparison
in
is 0(logA?).
search
binary
space
in
index
the
Complexity
the
one
indices,
be solved
the
only
all such
finding
is to check
of
6.2 Binary
Input:
(X,
Binary_Search
Algorithm
X (a
sorted array
Output: Position
that
= z,
X[i]
search key).
z (the
to a?), and
range
/ such
index
(an
121
z);
n,
in the
or 0
index exist).
such
no
if
begin
Position := Find(z,
1, n);
end
begin
if Left
= Right
then
:= Left
Find
else
Middle
f/2(Left+Right)]
:=\\
ifz<X[Middle]then
Find := Find
else
Find := Find
(z, Left,
Middle-1)
(z, Middle,
Right)
end
Figure 6.1
Binary_Search.
Algorithm
it is
it to a
is
in a
Search
Binary
some
sequence
in increasing order.
is a, for
minimal element
Given
in
as
it
linearly.
the
/, and
unknown
The Problem
values
convert
of n
Cyclic Sequence
is said to
x\\,x2,...,xn
sequence
can easily
sorted
a cyclically
list (we assume, for
list,
find
the
simplicity,
position
that
this
number
a*,_j
in
the
is sorted
of the
position
is unique).
To find the
eliminate half
that
k <m.
minimal
the
If xk
whole sequence.
element jc, in
with one
sequence
then
<x\342\200\236n
(Noticethat
/ cannot
we
the
we use
sequence,
cannot
in
the
range
exclude
k <i<m,
xk.) On the
the
of
idea
numbers
two
since
other
xt
hand,
binary search to
xk and xm, such
is minimal
in the
if xk >xm,
then i
122
must
be
Thus,
one
find
i in
Cyclic_Binary_Search
Algorithm
somewhere
is switched
order
we can
comparison,
we can
appropriately,
since the
range k<i<m,
in the
with
and Sets
Sequences
Involving
Algorithms
By
is given
Output:
in
range.
and
6.2.
Fig.
n,z);
(X,
X (a
Input:
that
in
choosing
elements).
begin
:= CyclicJ
Position
n);
ind(\\y
end
Cyclic Jind
function
begin
if Left
then
Right
:= Left
Cyclic Jind
else
Middle := ['A (Left + Right)] ;
ifX [Middle ] < X [Right ] then
Cyclic Jind := Cyclic Jind
else
CyclicJind := Cyclic
(Left, Middle)
(Middle+1,
Jind
Right)
end
Search
Binary
the following
search problem,
index that satisfies a specialproperty.
In
The
a i,
Problem
the
Given
Cyclic
Jearch.
Jinary
Index
key
is not
a sorted
there
sequence
exists
distinct
of
an index
such
looking
for
an
integers
that a{ -i.
binary
search
is not
Binary Search
6.2
Input: X (a sorted
in the
array
index
123
(A, n);
Special_Binary_Search
Algorithm
and Variations
range 1 to
of distinct
integers).
] = Position,
A [Position
satisfying
no
or 0 if
begin
Position:= Special_Find(\\,
n)
end
begin
if Left = Right
if A [Left]
then
= Left then
else Special_Find
SpecialJind
:= 0 { unsuccessful
:= Left
search
else
Middle :=\\'/2{Left+Right)\\
if A [Middle ] < Middle then
Special_Find
:= Special_Find
(Middle +
Special_Find
:= Special_Find
(Left, Middle )
1,Right)
else
end
Figure 6.3 Algorithm
in Sequences
Search
Binary
use a
we
Sometimes
Special_Binary_Search.
of Unknown Size
search
the search
but suppose that the
to double
space rather
size of the
problem,
the search range, since we do not know its
look
for an element jc, that is greater than or equal to z. If we
find
in the range 1 to i. We first
such
search
an element,
then we can perform
binary
<
z to jc i. If z x j, then z can only be equal to jc i. Assume,
compare
by induction, that we
; > 1. If we
know that z >
we
double
the
search
z to
then
for
some
space
j
compare
jc,jc2y,
with one comparison. If
and
we can find z with
then
we know
that
jc, <z<jc27
z<xlr
O
then it
(logy) additional comparisons. Overall, if i is the smallest index such that z<xh
takes 0(log/) comparisons to find an jc, such
that
and
another
0(log/)
z<jc,,
than
to halve
it. Consider
sequence is unknown.
boundaries. Instead, we
comparisons
to
find
the
We
regular
cannot
search
halve
i.
binary
search
like procedures.
Therefore, this
algorithm
is better
only
when
i =
0(yn).
Algorithms
124
and
Sequences
Involving
Problem
Stuttering-Subsequence
The
Sets
of binary
principle
search.
Let
character appearing
times
For
consecutively.
example,
if
B=xyzzx,
then
B3 =xxxyyyzzzzzzxxx.
The
Problem
value of
This
problem
such
Given
that B'
is called
the
two
A and
sequences
B,
find
the
maximal
is a subsequence of A.
problem.
stuttering-subsequence
binary
It may seem
difficult
at
search.
can construct
whether
B'
is a subsequence
of A
for
the
sequence
any
specific
= 0(n\\og(n/m)). Sequencecomparison
also
are
0{{n+m)\\og{nlm))
problems
discussed in Section
6.8.
This solution suggests a general technique.
we are looking for the
Whenever
maximal
/ that satisfies some property,
that
be sufficient to find an algorithm
it may
determines whether
a given
i satisfies that property.
We can do the rest by binary search
if we
have an upper bound
/ satisfies it,
is such that,
whenever
for /, and if the property
then j satisfiesit, for 1 <j </. If we do not know an upper bound for /, we can use the
scheme.
That is, we can start at / = 1 and double the value of / until we find the
doubling
will
This
take longer, but, unless the desired
search
/ is extremely large, it
right range.
will
still be efficient. The resulting
In many
not
be optimal.
algorithm,
however,
may
extra
the
it
is
to
eliminate
cases, such as the stuttering-subsequence
problem,
possible
O (log n) factor.
6.3
Equations
Solving
area doesnot
This subject
here.
mention
(i.e., a<x<b),
f (a)
that
and
is negative).
one
we
which
can
find a
of
the
f (a)
to
solution
that
x is
the
the
in
equation
/is
[a, b ]
range
is positive
and f(b)
short
(a) = 0, where
equation
given
it deserves a
but
chapter,
to
We are
compute.
want
this
a solution
to find
want
'f(b)
We
subject of
to the
conform
that we
Suppose
function
a continuous
other
125
Search
Interpolation
the
and
within
[a, b].
We
given
precision.
Since
of
precision),
required
or
as bisection
at
evaluated
function/is
then we have
solution.
in the range
exist
must
solution
known
search,
binary
follows. The
works as
is continuous,
function
the
a variation
use
Bolzano
the
=(a+b)/2.
a*j
= 0 (within
select one
If/(jtj)
we
Otherwise,
can
can
which
method,
the
of
the
one
half the size of the original,
in which a
or Uj,/?], each being
is guaranteed
to exist. The selectionis done
such
that the values of the function
solution
at the other. We continue
in this way until
the
are positive at one end and
negative
After
k
the
size
of
the
that
a
is
contains
solution
achieved.
desired
steps,
region
precision
[0, Jtj]
subranges
\\$(b-a)l2k.
6.3
In
Search
Interpolation
the search
search,
binary
However,
performance.
number
search
z,
if
space is always
the
during
we
search
find
to
reasonable
more
seems
it
in half,
cut
in
search
the
continue
the
that
if z is very
In particular,
to the next half
\"neighborhood\" instead of blindly
point.
going
of the sequence instead of
start
the search somewhere in the beginning
small, we should
at
the
point.
halfway
Consider
number.
the
Say
200 is
thus
the
around
where to
we
way
about
page
is at about
200,
that
the
now
We
80
the
of
one-fifth
a book
which
be larger than z.
X14]. The algorithm,
in
Fig.
are
percent
way.
we can
We can
flip
search to a range
page 1 and
cut the
mark between
one
this
continue
page
we
when
searching
for a certain
open the
we get page 250 instead.
page
open
at a
the
process
of
250
250.
search
exactly
space by a fixed
half,
we
of
suppose
the
to
close
the
page
Page
indication
pages, and
We now try
until we get
time. This is
book.
that
desired
go back
enough
to
idea
behind
cut
it by an
is determined
to succeed. This amount
by
in Fig. 6.4. The first guess is at X [8], which turns out to
leads to X[5], and then another finally leads to
Another
interpolation
the precise expression used for the interpolation,
is given
including
is illustrated
6.5.
but
and Sets
Sequences
Involving
Algorithms
r&
-r--&-
,OH
X[i]
2 3 4 5 6 7 8 9 10 11 12
13
14 15 16 17
6.4
Figure
(X, n,z);
Interpolation_Search
Algorithm
Input:
(a sorted
Output:
array
Position
in
the
to \302\253),
and
range
/ such
index
(an
search.
Interpolation
that
= z,
X[i]
z (the
or 0
if
search key).
no
such
begin
ifz < X
[ 1]
or z
>
{ unsuccessful
else Position
[n ] then
search
:=
:= 0
Position
}
1, n)
Int_Find(z,
end
begin
= z then
else if Left = Right
IntJFind := 0
ifX [Left ]
:= Left
IntJFind
or X
[Left ] =
[Right
] then
else
Next
Guess
:=
-(z-X[Left])(Right-Left)
Left +
X
ifz<X[Next_Guess]
IntJFind :=
[Right]-X
[Left]
then
IntJFind
(z, Left,
Next
Guess
\\)
else
Int_Find
end
:= Int_Find
(z, NextjGuess,Right)
Interpolation
JSearch.
index exist).
6.4
127
Sorting
a book
It can be shown
that the average
are, of course, uniformly
distributed).
where
the
is taken
number of comparisonsperformed
search,
by interpolation
average
this
seems
to
an
be
order
of
over all possible sequences,is 0(loglog\302\253).
Although
over
the
of
to
search
the
extra
(due
improvement
performance
binary
magnitude
search is not much better than binary search in practice
for two
interpolation
logarithm),
n
is
unless
the
value
of log2\302\253is small enough that the
main reasons. First,
very large,
of it is not much smaller. Second,interpolation
search
more elaborate
requires
logarithm
arithmetic.
pagesof
6.4
Sorting
Sorting
many
of the
is one
basis for
most extensively
algorithms,
many
of sorting
As
creasing
order.
l<ii,/2,
...,/*<> i, such
For simplicity,
unless
the methods described
algorithm
is called
that holds
the
6.4.1
Given
other
In
x ,*2.\"
n numbers
that
words, find a
< ''
<x
in
section
this
in-place
if
no
We
them
\302\273-*fl\302\273
arrange
of
on the
concentrate
problems.
will
into
go
in inindices
distinct
**i.
assume
additional
we
usual,
sequence
jc^
specified otherwise,we
computing
section
other
the
is
It
time for
the
of
subject.
The Problem
of
proportion
variations
dozens
science.
in computer
problems
a large
are numerous
There
applications.
typical
studied
it consumes
and
that
the numbers
nondistinct
work space is
are
used besidesthe
All
distinct.
as well.
numbers
initial
sorting
array
elements.
Bucket
Sort
Perhaps
the
simplest
number
of
''boxes\"
and
sorting
\342\200\224
we
Radix Sort
technique is
call
them
the
\"mailroom\"
buckets
\342\200\224
and
sort:
put
a sufficient
element in
are letters
allocate
each
the
corresponding
need to
they
128
At
the
complexity
of
value.
Sequences and
Involving
Algorithms
end,
this
we scan
the
buckets
algorithm
On the
algorithm.
simple
Sets
and collect all the elements.
0(m+n). If m=0(n), then we
if m is very large relative to n (as
in order
is obviously
other hand,
linear-time sorting
then
with
O(m) is too large. In addition,
be the case
zip codes),
is
an
even more serious problem for large m.
0{m) storage,which
extension of this idea is radix sort. Consider
A natural
the algorithm
the
zip-code
The
get a
may
requires
example
because
the range of zip codesis
again. Using bucket sort for zip codes is not effective
We use induction
to
reduce
the
on the
Can
we
do
handle.
to
too
range?
something
large
we
10
buckets
We
use
several
use
and
sort
in
the
First,
stages.
following
way.
range
of
the
code.
Each
bucket
now
to
the
first
covers
zip
only
digit
according
everything
zip codes
this stage is O (n).
different
Induction
hypothesis:
(correspondingto
We know
how
the
to sort
four
remaining
we have
stage,
elements
with
< k
digits.
extend
6.4
most significant
then, by the induction
digit,
we
have to make sure
Thus,
step.
are in the
they
hypothesis,
the last
that
129
Sorting
before
order
right
order.
This is
the
they stay
right
of
the
is
and
it
a
of
the
use
of
the
inductive
part
algorithm,
good example
only
is correct.
to make sure the algorithm
It is essential that elements
that are put in
approach
in
remain
the
This
can
same
bucket
same
order.
be
achieved
a queue for
the
by using
the d queues at the end of a stage to form
one
each bucket, and by appending
global
to the / least significant
The precise
digits).
queue of all elements (sorted according
algorithm is given in Fig. 6.6.
the
in
subtle
X (an
Input;
(X, n,
Straight_Radix
Algorithm
Output:
integers,
each
in sorted
order).
of
array
(the
array
k);
with
k digits,
in the range
in a
global queue
to n).
begin
{
that
assume
We
use
We
all elements
are
simplicity;
it can
GQfor
fori := 1 toddo
{d is
number
the
initialize
for
Q [i ]
queue
X }
through
numbers }
of decimal
case
do
empty
xfrom GQ ;
pop
d := the
ith
of x
digit
for i :=
to be empty
GQ ;
I do
is not
while GQ
be implemented
digits; d = 10in
of possible
i := k downto
initially
GQ ;
into
Q[t]
1 to
n do
from GQ
pop X[i]
end
Figure6.6 Algorithm
It takes
Complexity
the
initialize
the
they
make
The
compared.
The
main
overall
pushes
it
of the
into
one
time of the
without
the whole
about
the elements
in
the
algorithm
the
types
regard to
the
is executed
It
is O
except
also
k times,
all
concatenates
use direct
keys. Each
more general since
two
can be
elements
techniques
that
of their
are
that
d steps to
(nk).
\"structure\"
GC, and
queue
algorithm, which
of the Q [/]s.
elements
involve
no assumptions
loop
running
of this
remainder
thus
to put all
steps
GC and
from
comparisons betweenthe
comparisonwill
Q[i].
queues
Radix.
Straight
130
Involving
Algorithms
and Sets
Insertion
6.4.2
Sequences
Sort
O (n
\\ogn) comparisons.
also
is
It
Chapter 4).
Scanning
an
by induction
to do is to
possible
AVL
Using
AVL
that
tree
we
to use
balancedtrees for
trees,
to get a list
how to
for example,
of its numbers
know
build
an
AVL
insertion
efficient
each
insertion
in order
tree for
or selection
requires
- 1
numbers,
0(\\ogn)
If we
(see
time.
assume
into
numbers
for insertion
6.4.3
To
and
numbers
sorts are
simple and
are
left as
exercises.
Mergesort
improve
sorted
selection
involves
6.4
place in
set for
first
the
precisely, denote
that both
assume
b{ is
to insert
place to
The
insert
about data
What
It
the
and
b\342\200\236n
right
scan
is inefficient
to move
will be
moved many
elements
same
so
b2,...,
set until
first
the
More
on.
and
b\\,
worst
the
in
Scan
order.
continue
smallest,
set by
the second
\342\200\242..,
an, and
Sincethe
and so on.
movements?
right
go back.
sets.
the
is
insertion
an
since the
Instead,
place
the
merge
producesthe
elements
element
each
and
additional
storage
is available).
and-conquer
First,
the
parts.
is divided
sequence
part
sorted
one
An example
6.7.
that
algorithm,
Second, each
mergedinto
Fig.
it; then
of comparisons,
number
total
is found,
a2,
in increasing
insert
and
the second
number,
set by a\\,
found,
b2
smallest
the
first
the
131
Sorting
into
is sorted
two
can be used as a
The
(in case
or close-to-equal
equal
recursively.
separately
basisfor
the
Third,
be
for
the
of comparisons
number
that n is
simplicity,
recurrence
required
a power of
2.
a divide-
as follows.
works
algorithm
of an
odd
size)
Let T(n)
Complexity
we just described
known
as mergesort.
by
shown).
in the
mergesort
T(n), we
To calculate
relation:
of this
solution
asymptotically
sort.
data movements
Although
drawbacks.
relation
the 0(n2)
is
running
T(n)
time
= 0(n
\\ogn)
required
(see
Chapter
3), which is
or selection
number
The
recurrence
than
better
First,
requires additional
versions
of mergesort that use only
algorithm. (There are more
complicated
amounts of extra storage; see the bibliography
This copying must
section.)
are
which
makes
the
time
two
smaller
sets
every
merged,
procedure slower.
6.4.4
constant
be
done
Quicksort
divide
and
its analysis
(X, n);
in the range
Mergesort
Algorithm
Input:
array
(an
X (the
Output:
and Sets
Sequences
Involving
Algorithms
in sorted
array
to n).
order).
begin
M_Sort(l,n)
end
procedureM_Sort(Left,
Right)
begin
if
Right
{ checking
correctly
Left =
1 then
this
case is not
for
but
it makes
anyway,
then
it
handled
be
will
more efficient}
(X[Left], X[right] )
the program
swap
+ Right)]
l/i (Left
\\
because
necessary,
M_Sort(Left, Middle-l) ;
;
M_Sort(Middle,
Right)
{ we
the two
now
merge
sorted
one
sequence}
i:=Left;
j := Middle
k := 0 ;
(i <
while
k:=k
Middle -\\)and
+ J;
(j<
Right)
do
ifX[i]<X[j]then
TEMP[k]:=X[i] ;
i :=i
else
TEMP[k]:=X[j];
if j
> Right
then
{ we
copy
TEMP
[Left+
back into
do
:=0tok-1
fort
-\\-t];
[Middle
[Right-t]:=X
now
t]:=
TEMP [t]
end
Mergesort.
array
the
}
right
place }
6.4
133
Sorting
10
12
15
13
11
16
14
(2)
(6)
10
12
15
13
11
16
14
(5)
(8)
10
12
15
13
11
16
14
(5)
(6)
<\302\253)
10
12
15
13
11
16
14
(9)
do)
12
15
13
11
16
14
15
(\342\226\2402)
13
11
16
14
(0
15
(\342\226\2402)
13
11
16
14
15
13
11
16
14
13
11
16
14
11
16
14
11
16
14
16
14
(2)
10 (0
(.0)
(9)
10
12
10
12
10
12
10
12
13
15
10
12
13
15
10
12
13
15
10
12
\302\251 \302\251
15
\302\251 \302\251
\302\251 \302\251
11
\302\251 \302\251
\302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251 \302\251
an exchange
either
illustrates
current
An example
6.8
Figure
operation
Supposethat
or
than
equal
partition
can be
partition
divide
the
trivial
accomplished without
We can
step.
the two parts
now
sort
additional
each
space,
subsequence
already occupy
space is required.
Thus far, we have assumed that we
since
elements are
the
correct
as
will
be
in
the
greater
x.
- 1
comparisons. Since the two parts are equal in
half of the array
the other the second half.
and
first
the
of the
one-half
requires
occupy
the
partition
that
are involved
that
row
Each
order.
initial
numbers
of the
one-half
and
elements to x and
a number x such
know
we
to x
in the
part can
one
size,
this
Furthermore,
shown
shortly.
recursively. The
in the array.
positions
This is
combine step is
no
Therefore,
additional
It is
that
used
for the
to
with
partition.
We
partition
following algorithm.
left side of the array
call
the same
algorithm
value
will
work
equal to
We use two
and
the
know
R points
the
pointers
to
the
We
pivot.
to the
right
side
the pivot.
the
than
can achieve
array, L and
of the array.
R.
this
pivot
not.
is
Our purposeis
and the other
with the
partition
L points to
\"move\"
pointers
Initially,
The
do
number
the
in
134
Induction
that
At step
hypothesis:
i <L,
of
the correctness
guarantees
invariant)
The
other.
each
toward
opposite directions
Sets
Sequences and
Involving
Algorithms
the
all j
for
of
induction
following
partition.
the
such
algorithm,
all i such
j >R.
that
true
at the
L to
the
(or loop
hypothesis
the hypothesis.
the
conditions).
invalidating
and
Middle
and
of the
boundary
array), and
boundary).
right
Output:
X [j ]
>X
[Middle
begin
:=X[Left]
pivot
L ;= Left; R :=
whileL <Rdo
Right
X [L]
while
[R ] >
ifL <
then
exchange
Middle
exchange
:=
L < Right
while
pivot and
X [L]
>
X[R]
X [Middle
with
do
Left do R
:=L +
X [Left ]
with
end
Figure
6.9
Algorithm
1;
= R-1 ;
Partition.
Right
6.4 Sorting
the
When
step of the
We
the
from
policies
can be
sequence
exchanged
number
The circled
(6).
we can
exchange
of
are
numbers
those
is given in
that have
Fig.
the
with
the
6.10.
xL at
the last
as required.
partition
then
been
just
it
case,
any
and
first element,
the
with
Partition
of algorithm
example
pivot,
which
other
mention
is chosen as the
first element
partition,
135
any pivot
chosen
algorithm
in Fig.
exchanged.
exchanges, ph points to X [6] = 1, and pt points to X [7] = 12. The last exchange involves
this exchange,
the middle point
(1) and the pivot (6). After
everything to the left of the
is
less
than
or
to
and
the
to
it. The two
it,
equal
right is greater than
everything
pivot
1
to
6
be
and
from
7
to
can
sorted
is thus
16)
Quicksort
subsequences (from
recursively.
an
of it
The
of the
selection
Fig.
of quicksort
time
running
pivot. If
the
10
12
15
13
10
12
15
13
0
4
12
15
15
15
of an
array around
6
6
\302\251
Figure
Algorithm
and an example
12
\302\251
12
\302\251
6.10
Partition
14
13
16
14
10
13
16
14
10
13
16
14
the
pivot
Q_Sort(l,n)
Right);
begin
then
Left, Right);
Q_Sort(Left, Middle
Q_Sort(Middle
\\)
+ 1, Right)
end
Figure
6.11
parts,
16
end
< Right
and on
begin
Partition(X,
equal
14
procedureQ_Sort(Left,
input
two
16
if Left
6.11,
partitions
always
pivot
\302\251
in
6.12.
in Fig.
is presented
Complexity
the
algorithm.
in-place
Algorithm Quicksort.
6.
136
Algorithms
Sequences
and Sets
10
12
15
13
11
16
14
(?)
12
15
10
13
11
16
14
CD
(?)
12
15
10
13
11
16
14
0)
(?)
12
15
10
13
11
16
14
12
15
10
13
11
16
14
11
10
13
15
16
14
11
10
13
15
16
14
10
13
15
16
14
13
15
16
14
15
16
14
16
\302\251
\302\251 \302\251
\302\251 \302\251
\302\251 \302\251
selected
two
each
in
it is
pivots
then
obviously
the
the
then the
eliminate
comparing
(namely,
among
be 0(n2)
first
partition
pivot is
A new
input.
number
appears between
is
requires
sequence is already
- 1 comparisonsand
in
order,
increasing
places
and we
only
the pivot
always
select
in
the
T(n) =
-
analyze
that each
and
of
the
for the
comparisons
n-i.)
running
If each
time is
that
the
smallest
case occur is
very
jcf-
has
the
if the
selectedas the
pivot
is
+ 7(/i-i).
n-\\ +7(i-1)
a chance
still
likelihood
case.
this
The
takes
the running
then
pivot,
worst
assume
We
average
initial
a single
position.
We now
will
the
the
in
in the
element
sizes
is the
\302\251
first element
pivot.
14
We
If the
place.
right
small.
is
time
running
sequence,
right
relation
recurrence
the
the
in
\302\251 \302\251
6.12
Figure
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251
\302\251 \302\251
\302\251
\302\251
!\302\251 \302\251
\302\251
\302\251
1\302\251 \302\251
\302\251
\302\251 \302\251
(It
Involving
partition,
element has the
and
we
same
of
the
137
6.4 Sorting
= \302\273-l +
-\302\243(T(i-l)
\302\273
/-i
r(\302\273)
\302\273-1+
+ T(/i-i))
n;-1)
\302\243
r(n
\302\243
-i)
\" ,=0
This is a recurrencerelation
Section 3.5.3, and its solution
is indeed quick on the average.
with
was
full history.
We
shown there to be
discussed
this
T(n) = 0 (n
in
relation
particular
log\302\253).
Hence,
quicksort
reason
its name. A major
for
In practice, quicksort is very fast, so it well deserves
and conquer, is that
elements
are
divide
its quickness, besides the elegant
many
the same element
(the pivot). The pivot is thus stored in a register and
compared against
In most computers, this saves
from memory.
there is no need for a data movement
time.
considerable
is to use a technique
we call
One way to improve the running time of quicksort
The idea is to start the induction
not always
choosing the base of the induction
wisely.
is called
until
the base case, which
from 1. Quicksort,as describedabove,
recursively
such as insertion
consists of sequencesof size 1. However, simple sorting
techniques,
well
for
small
whereas
of
sort or selectionsort, perform
the efficiency
sequences,
very
for
we
can
define
base
case
for
the
Therefore,
quicksort shows only
large sequences.
1 (it seems that 10 to 20 is a good size, but
that
than
quicksort to be of size larger
sort.
and handle
on the specific implementation),
the base case by insertion
(In
depends
- Threshold\"
other words, we replace the check \"if Left < Right\"
by \"if Left < Right
and add an \"else\"
This change leads to an improvement
sort.)
part which runs insertion
of the running time of quicksort
In Section
we will see how
a
small
constant.
6.11.3,
by
to use the principle
of selecting the base of the induction
to improve asymptotically the
of an algorithm.
running
time
6.4.5
Heapsort
is another
Heapsort
but
\302\253,
is guaranteed.
Unlike
0(n \\ogn).
we emphasizeone
Like mergesort,
heapsort is an
mergesort,
of heapsort
part
\342\200\224
building
the
it is usually
not
other
worst-case
in-place
hand,
running
sorting
the heap.
The
algorithm.
algorithm
quite
as
In this
for
fast
as
section,
building
the
should
be interleaved.
heap
way design and analysis of algorithms
4.
in
were
discussed
We
assume
here
an
Chapter
Heaps
implicit
representation;
in an array A [\\..n], which
to a tree in
specifically, the elements are given
corresponds
the following
tree
is
the
of
root
stored
The
in
A
and
children
the
of
node
[1],
wayi
any
A [i ] (if there are any)
in A [2/ ] and A [2/ + 1 ]. Such an array satisfies the heap
are stored
is greater
than or equal to the values
if the value of each node
of its children.
property
illustrates
the
138
and Sets
Sequences
Involving
Algorithms
as follows.
in
The input is an array A [l../i].
First, the elements
Heapsort works
how
build
a
will
discuss
to
form
We
later.
If
a
to
A
are
heap
the array
heap.
rearranged
We
A
of
the
with
A
A
is
maximal
element
the
then
[1]
[n ]
[1]
array.
exchange
is a heap,
We then consider the array A[\\..n-\\].
so that A [n ] now contains the correct element.
to worry
to form a heap (we have
only about the new
Again, we rearrange the array
with A [l../i -2]. Overall,there
is one
A [n
with
1], and continue
A [1]), exchange A[\\]
and rearranging the
elements
a heap, and n - 1 stepsof exchanging
initial step of building
is given
heapsort
in
6.13.
Fig.
Algorithm
for i :=
below
2 do
downto
swap(A[\\lA[i]);
the
basically
in
same
as RemoveJAaxJrom
procedure
Heap
4.7 }
Fig.
end
Figure 6.13
a Heap
Building
We now
Heapsort.
Algorithm
on the
concentrate
Given
Problem
The
order,
problem of building
rearrange
the
elements
an
array
a heap
of elements
A [\\..n]
so that
the
from an arbitrary
satisfies
array
in
an
array.
arbitrary
the heap
property-
There
ways to
respectively
to
scanning
build
the array
6.14
We
illustrates
then
a heap
\342\200\224
top
representing
both
down
the
methods.
heap
substantial
difference
in performance
6.4 Sorting
and bottom
up heap construction.
left to
(corresponding
methods.
two
Consider
down
139
scanning
the
from
array
right
A[\\..i]
array
to top
is a
down).
heap.
The basecase is
worst
is
element A[i].
to a collection
by
],
A[\\..n
now
array from
the
scanning
to say
like
would
We
+ 1 )J.
|_ log2(/
Consider
the
that
to
right
array
A[i + \\..n]
left
is a
A[i+l..n]
does not
to bottom up).
adding the
(corresponding
heap
to
and
consider
But
array
correspond
more complicated.
Induction
A [i +
A[n]
however.
trees
the
array
itself
The
We need to
trivially.
that
is also
satisfy
trees representedby
condition.
the heap
is obviously
a heap, so the base case is satisfied. We can do better,
whole array A [[n/2\\ + \\..n] represents leaves
in the tree.
Hence, the
+
A
to
are
all
so
\\..n]
corresponding
singletons,
[[n/2\\
they satisfy the heap
by
property
hint
(bottom
hypothesis
\\..n
start
the
induction
process
only
at
|_fl/2_|.
This
is a good
(This
with
care.)
Consider
now
serving as roots to
heap
It has
A[i].
valid
is straightforward.
heaps
A
[i ]
at most
(by the
two
induction
is compared
children
hypothesis).
(A[2/
to
the
maximal
+ l]
and
A [2/]),
both
A [i] into
Incorporating
of its children, and,
if
140
and Sets
Sequences
Involving
Algorithms
reaches
place
illustrated
the
hence,
comparisons;
of
overestimation
the
time,
> XLlog2/J
il\\og2i\\
,=1
height
is
A[i]
[\\og2(n/i)\\,
is
of
number
the
running
running
of
construction
2|_log2(\302\253//)J.
down)
(top
Complexity
>\302\253/2Llog2(\302\253/2)J
Q(/ilog/i).
i=n/2
most twice
the
comparedwith
complexity is at
evaluate
//(/),
look at complete
Let's
in
the
tree of height
i, and
of this
binary
complete
construction
heap
trees
each
in
//(0) = 0.
tree of
height
is
2/+l
-1,
it
is at
step
the
Since
of
the complexity
of
sum
the
a recurrence
i - 1 and a
verify
that
follows
trees of height
of two
can
We
We
/.
height
i consists
by //(/)
can derive
denote
and
first,
binary tree of
complete
noting that a
involved
comparisons
node
the corresponding node (since each
may have to be
the
the tree).
its two children,
Therefore,
exchanged, and so on down
We want to
of all nodes in the tree.
most
twice the sum of the heights
of
number
of
height
sum.
this
The
Up)
(bottom
Complexity
relation
case is
worst
the
in
the
of A[i]
value
old
the
A bottom-up
its children.
both
than
6.15. Since
in Fig.
comparisons
it is larger
where
in a heap
a deletion
to
the
in
bottom-up
2k -1
trees
is O(n) for complete binary
nodes).
(namely,
heaps with
The complexity for a heap with n nodes such that 2k <n < 2k+l - 1 is no more than that
for a heap with 2*+l - 1 nodes, which is still O(n). (A more careful analysis shows that
the bottom-up
the constant
is not increased; see Exercise6.32.)The reason
approach is
10
11
12
13
14
15
16
10
12
15
13
11
16
14
10
12
15
13
11
16
10
14
15
13
11
10
16
14
15
11
12
10
13
16
14
15
11
12
10
13
16
14
11
12
10
13
11
10
13
11
11
2
\302\251
Figure
indices.
\302\251
\302\251
15
6.15
The
16
\302\251
\302\251
15
\302\251
14
10
An example
numbers
circled
\302\251
\302\251
\302\251
\302\251
12
12
a
of building
are those that
\302\251
14
\302\251
heap
have
\302\251
bottom
been
\302\251
\302\251
up. The
exchanged
on
numbers
on
that
step.
\302\251
\302\251
top are
\302\251
1
the
6.4
faster
and
top. Thus,
minimize the work
This is another
The
algorithm.
Comments
that
it is
worthwhile
the
induction
also
have
to summarize
this translates
analogy,
seen
This
We
with an 0(n2)
proof that
prove a lower
approach. We
and
algorithm
that the
to prove
higher
called
a decision
to
the
or
with a
associated
We
the
problem
it
an 0(n
to
for
model
and
general
\342\200\224
lower
hence,
and are
with.
work
such
consist
of computation,
bounds using them
easier to
not just
one
arbitrary (unspecified)
fits the model must be
that
computations
log a?)
a particular
It is much harder to
better.
to an
corresponds
models
respects
and improved
A lower bound
bound.
possible output.
The computation
that
time
running
each
with
a model
lower
but
they are simpler in many
variations of decisiontrees, and
We define decision trees
and leaves. Each internal
node
than
rather
sorting
can solve
Decision
comparisons.
orders
in hand
further?
even
it
we have to
since
or equal
for
algorithm
algorithm
to define
need
not
to improve
no
bound,
than
possibilities,
intuition
started
problem is a
to make
order
different
for Sorting
algorithm.
machines
go
intuition
Bound
it possible
in
hand
A Lower
Is
leads to a better
intuitive, but the
divide
of
trying
(but
have
one
in
subsets
is usually
truth
The
6.4.6
into
must
to develop
analysis is performed.
algorithm.
straightforward
sorting
to special
induction
the
and
superior.
applying
of induction
order
different
more
the
in this
described
particular,
to be
out
is hard
It
were
that
trying a
is
method
bottom
of the tree
bottom nodes rather
the
at
for the
work
the
nodes.
where
top-down
turns
to minimize
the top
for
many nodes
are
there
that
is better
it
example
method
bottom-up
is
approach
top-down
at the
to
than
the
than
few
141
Sorting
mainly
model
of
as are Turing
are weaker\342\200\22
There
are many
lower bound
them.
proofs utilizing
internal
nodes
as binary trees with two types of nodes\342\200\224
is associated
with
a query
whose outcome is one of two
is associated
with
one of the emanating branches. Each leaf
known
many
assume
that the
input
is a
sequence
of numbers x \\,
jc2,
..., xn.
to the
node,
query
is
the
left
or
the
branch
of the query, either
and, according
input
right
taken.
When a leaf is reached, the output
of the
associated
with the leaf is the output
time
with a tree T is the height
associated
of 7\\
computation. The worst-caserunning
which is the maximal
of queries required by an input.
number
A decision tree thus
decision
to an algorithm. Although
trees cannot model every algorithm
corresponds
(for
root
of a number with
we cannot compute a square
a decision
example,
tree), they are
A lower bound
reasonable models of comparison-based
obtained
for
algorithms.
better.
decision trees implies that no algorithm
We now use
of that form can perform
decision trees to prove a lower bound for sorting.
at
starts
to
the
the root
outcome
of
the
tree.
At each
the
is applied
142
and Sets
Sequences
Involving
Algorithms
6.1
Theorem
\342\226\241
for
has height
sorting
The
Proof: The input for sorting is a sequenceX\\,x2,...,*\342\200\236.
sequencein the sorted order. Another way to look at the output is that
is a
correct
permutation
(rearrangement)
of (1,
the decision
tree for
The
Since two
different
permutation.
that
total
The
the tree is
2,..., n)
the
tree,
height
to different
of
a permutation
same
the
possible
as a possible output
in
represented
decision tree is associatedwith the leaves.
in a
output
it is
elements
the
since
all
is the
be
should
be
there must
Therefore,
number of permutations
binary
rearrange
possible output,
if it handles
correspond
permutations
leafs.
different
with
sorting.
how to
indicates
of
output
least
at
tree
the
one
n elements
on
they
outputs,
is at
leaf
is n!.
be associated
must
for every
possible
Since we
least log2(fl!).
By
assume
Stirling's
formula
n\\ =
^2nn(-)\"(\\+0(\\/n)).
=
!)
Hence,log2(fl
kind
This
because
defined
it does
the
kind
\302\243l(n
log
a?), which
completes
the
\342\226\2
proof.
is called
of a lower bound
an information-theoretic
not depend at all on the computation (notice that we
of queries we allow), but only on the amount of information
lower
have
bound,
not even
contained
in
example, to
\"=,\"
\"<,\"
which is
a decision tree
and
\">\.")
as
a tree
In this
three
with
case,
the
height
children
(corresponding,
would
have been at
for
least
still
Q(a?
log
a?).
In
other
words,
program
models
build
the
we can
out that
This is not
of polynomial
trees
decision
build
of the
size
the
\302\253,
trees are
Decision
solutions).
possible
of computation.
The tree dependson
different
trees
for different values of n.
input.
\342\200\224
but
height
nonuniform
can potentially
We
a whimsical
just
143
worry.
exponential
turns
It
size
\342\200\224
for
so decision
trees are too
time,
probably require exponential running
sometimes.
That is, a decision tree lower
bound
may fall far below the actual
optimistic
of the problem. On the other hand, if the lower bound
is equal
to the upper
complexity
\342\200\224
of a particular
as is the case with
then
the lower bound
bound
algorithm \342\200\224
sorting
implies that even if we use a lot more space, we cannot improve the algorithm.
It is interesting to note that
the average
based
running time of any comparison
is
also
We
omit
the
which
is
much
more
\302\243l(n
sorting algorithm
log n).
proof,
for
and
Ullman
[1974]).
complicated(see example Aho, Hopcroft,
that
problems
6.5
Order
Given
a sequence
Statistics
S =x
sequence
by sorting
can be
answeredwithout
the
with
{,jc2,...,xn
the
elements.
problem
6.5.1
Maximum
In this
kih
minimum
such
in
S if Xj is the
a
in
about
ranks that
questions.
We start
elements,
then consider
the
element.
smallest
Elements
Minimum
and
with
of all elements
questions
many
section, we deal
rank
ranks
the
there are
and
Xj has
that
easily determine
the maximum
of finding the
general
we say
elements,
However,
sorting.
of finding
problem
of
We can
in S.
element
\302\243th-smallest
the maximum
or the minimum element of a sequenceis straightforward.
If we
this
the maximum of a sequence of size n-\\,
then we need only
to compare
the
maximum to the nth element to find the maximum
of a sequence of size n (finding
maximum of a sequence of size 1 is trivial).
This process takes one comparison
per
1.
of comparisons
is /i \342\200\224
element,
starting with the second element; hence, the number
now that we want
to find both the maximum and the minimum
elements.
Suppose
Finding
know
The
Problem
Find
the
maximum
and minimum
elements in
a given
sequence.
The
solve
straightforward
comparisons
solution
will be 2n
is to
-3: n-\\
to
both
find
problems
the
independently.
maximum
Involving Sequences
Algorithms
144
one
only
and
solution
by
elements
in
The
Let's try
and Sets
is required for
the elements in a
is irrelevant to the
scanning
the
sequence
We cannot
element.
second
the
comparison
different
because
order,
the
improve
of
position
the
the
problem.
one element
at a time.
the solution by more than
next attempt can be to extend
a
time.
That
assume
that
we know
at
two
elements
the
solution
is,
to extend
by
the problem for n -2 elements, and try to solve it for n. (For this approach
n = 1 and n =2, so that extending by two will
base
cases,
to be complete, we need two
be the maximum
{min)
cover all natural numbers.) Consider xn_x and xn, and let MAX
It is easy to see that finding
elements
first n-2
the
of
induction).
(known
by
(minimum)
the new maximum and minimum
only three comparisons. We first compare
xn_x
requires
two
values to MAX, then compare the smaller of
these
of
the
to xn, then compare
larger
with approximately
3\302\253/2comparisons
to min.
them
So, overall, we have an algorithm
elements
at a
of In comparisons! Can we do better by adding three (or four)
instead
of comparisons. It turns
the same
time? Following
approach leads to the same number
of comparisons
for this problem
out that we cannot reduce the number
by any method. It
to about
also
leads
3\302\253/2
a divide-and-conquer
to note that
is interesting
approach
to solve
how
6.5.2
We
6.14).
(Exercise
comparisons
Finding
consider
now
the general
The Problem
an
integer
k such
problem.
Given
that
a sequence
\\<k<n,
Sorting
There
-OQogn).
element
would
or
times.
is, however,
of k.
another
If k
selection.
This
algorithm
is
by running
algorithm,
that
in S.
very
close
the
algorithm
requires
approach
elements,and
element
\302\243th-smallest
element
be better
...,xn of
S=jtj,jt2,
find the
unless
to 1 or very
close
the
for finding
approximately
k is 0(\\ogn)
finds efficiently
the
kth
kn
or
smallest
except
6.16.
Complexity
Since
this
only one
algorithm
in quicksort,
As
choosing poor pivots leads to a quadratic
algorithm.
in each
time
recursive call, the running
subproblem has to be solved
is lower
than that of quicksort. The average
number
of comparisons
of
is
k) ;
range 1 to
(X, n,
Selection
Algorithm
145
in the
smallest
k (an
and
\302\253),
element;
the
integer).
X is
array
changed).
begin
or (k > n)
if(k<l)
then
\"error\"
else
S :=Select(l,n,k)
end
Select
procedure
(Left,
Right, k);
begin
if Left = Right
then
:= Left
Select
else
Partition
(X,
Left, Right);
be the
Middle
Let
output of Partition
>k then
if Middle -Left+\\
Select
(Left, Middle, k )
else
Select(Middle
+ 1,
Right, k
(Middle
Left +
1))
end
Figure 6.16 Algorithm
but
(a?),
steps
in
we will
the
worst
It is
Selection.
also possible to
practice,
the
find
the
kth smallest
presented
algorithm
in
Fig.
in O(n)
6.16 is
more efficient.
Comments
the
A?/2-smallest
Most
There is no simpler
algorithm
problem
median-finding
6.6
statistics require
Algorithm Selection is an excellent
for finding
only medians. In
of order
applications
element.
to
finding
example
values
any
\302\243th-smallest
of strengthening
of k.
the
finding
the
median,
median-finding
other
element
induction
words,
makes
hypothesis
that is,
algorithm.
extending the
the algorithm
since
the
Compression
compression
consider
as a string
is an
important
technique
of characters,we
want
for saving
storage.
to find a compressed
greater
of processing
cost
the
than
and Sets
it.
Data
particular
the
compression
a very
is
of sending
information is
cost
has many more applications, and
for one
one
we describe only
algorithm
problems where
in communication
important
also
it
Involving Sequences
Algorithms
146
a sequenceof
letters.
Each of the
English
of the
of
the
called
is represented by
bits,
encoding
unique
string
26 characters
standard
character. If the length of all encodings is the same (as is the case for most
of
on the number
file
the
of
bits
number
the
only
depends
representing
encodings),
to choose smaller bit
it is possible
characters in that file. On the other hand,
more often and larger
characters
for
(such as A) that
appear
representations
a
as
Z)
example, in
ASCII
are represented by
Standard
(American
B
of size 7. A is represented
1000001,
bit strings
by 1000010, and so on.
by the bit string
lower-case
and special characters.) The
including
(There is room for 128 characters,
three letters) requires 21 bits. If we change the
with
\"AND\"
word
(and any other word
time A appears. However, not
of A to, say 1001, we save 3 bits every
representation
For example, we cannot
valid.
There
be
is
of
set
ambiguities.
may
encodings
every
of M as 1001101, because
for A and leave the encoding
choose 1001as an encoding
of M. We could use
it is A or is part
1001 we cannot determine whether
we read
when
In
to separate
characters, but that would
only add to the representation.
specialdelimiters
not be equal to a complete
of one character must
general, the prefixes of an encoding
Whenever
the prefix constraint.
We call this constraint
character.
of another
encoding
to lengthen
the encodings of
have
of one character, we may
we shorten
the encoding
of
the frequency
others.
The
is to find the best balance, assuming we know
problem
Code for
appearances of the
encoding
we have to
First,
this
value
tables
length
the
total
of
of times
number
the
find
character.
(In
an
and that
text;
we call
frequency
L(E,F)=
for
of characters),
(a sequence
satisfies the prefix
constraint
of bits needed to encodethe text.
text
that
number
compute the
the frequency
computed
particular
/i>/2>
Given a
characters
the
for
minimizes
characters.
different
Problem
The
Information
typical
texts,
instead
isrfi.
1=1
Our
goal
L(\302\243,F).
is to
find
an
encoding
E that satisfies
the
prefix
constraint
and minimizes
6.6
The prefix
a decoding
to an
is equal
which
in this
root
constraint
file is
encoded
character.
scanned
a graphic
is
not
n -1
define
the
characters
induction
Consider
characters.
of
and
is based
to solve
problem
(and
of
reduction
and
in
the
that minimizes
It is
problem.
a problem
As
order
which
to
tree
binary
in
of 0s
and
Is on
The
L(\302\243,F).
tree
useful, however, to
its constraints).
case is trivial).
(the base
hypothesis
in order
on a
correspond
0, or no emanating
a tree
such
construct
the
must
is reached, we
a leaf
and
necessary
illustration
The algorithm
with
says
Our problem is to
representation
have
the
so let's look at
unambiguous,
of bits one by one until we get a
labeled
two
147
Compression
emanating
edges
by
tree correspond to the characters.
The sequence
to a leaf corresponds to the character's
encoding
The prefix
decoding
We need to
procedure.
sequencethat
to make the
is needed
constraint
Data
with
usual,
n characters
the main
to eliminate
difficulty
to a
problem
is how
to
is different
character
exercise.
010
Figure 6.17
The tree
representation
Oil
of encoding.
Algorithms
148
Sets
and
Sequences
Involving
for Huffman's
The operationsrequired
encoding are (1) insertions
Impl6ni6ntati0n
with
of
minimal
from the
deletions
the
two
characters
data
structure,
a
(2)
into
frequency
a
the
tree.
is
data
structure
for
the
first two
A
and
(3) building
data structure,
heap
good
in
the
worst
case.
of
which
The
is
each
requires
steps
0(\\ogn)
algorithm
operations,
known
as
This
is
Huffman's
6.18.
encoding after
compression
technique
given in Fig.
D. Huffman
[1952], who proposed this algorithm.
(S, f);
Huffman_Encoding
Algorithm
Input; S (a string
Output;
of characters),
begin
insert all characters
while H is
not
make X
one
only
root
the
H according to their
frequencies
do
empty
ifH contains
frequencies).
S).
a heap
into
array of
and/(an
tree for
Huffman
(the
X then
character
ofT
else
pick two
replace
lowest
with
frequencies
H ;
from
new
Z whose
character
of the frequencies
ZtoH
make X
them
Y with
and
sum
the
insert
X and
characters
and delete
ofX and Y
frequency is
;
Y children
and
{Z has no
ofZ inT
yet}
parent
end
Figure 6.18
Example
Suppose
in
1, respectively
The Huffman
6.19.
The internal
O (log
6.7
Let
n)
String
A -a
assume
\\
time
characters
although
sequence of
and
an
the
it
is
characters
a i a2
time
time
to the
per node.
of the
frequencies
characters
created.
were
they
5, 2,
is given
\342\226\2
algorithm
(n log
n).
Matching
\342\200\242
\342\200\242
\342\200\242
a2
that
characters,
substring
according
Building
Complexity
take
characters 4, \302\243,
C, D, E, and F, with
tree correspondingto these
10,and
Fig.
Encoding.
Huffman
6.1
\342\226\241
3, 4,
Algorithm
' '
' a{ (h\\
\342\200\242
\342\200\224
B=bxb2
come from a
not
b2
'
m<n,
set.
be two strings
(It is convenient
A substring of
\342\200\242\342\200\242\342\200\242
from
A We denote
a}
' '
bt).
necessary.)
atai+\\
bm,
finite
a
by
string
A(i)
of
characters.
to
think
A is a
(B(i))
of
We
English
consecutive
the special
6.7 StringMatching
6.19
Figure
The
Huffman
for example
tree
149
6.1.
Given
A and \302\243,
find
the first occurrence
two
The Problem
strings
(if any) of B in A. In other words, find the smallest k such that, for all /,
=
1 < / < w, we have ak+i
bt.
or pattern in a
word
obvious exampleof this problem is a search for a certain
to find patterns.
The problem also has
editor must contain commands
molecular
applications to other areas \342\200\224
biology, where it is useful to find
including
The most
text
certain
inside large
RNA
or
DNA
molecules.
an example
will
use
In
this
example,
at a 4
mismatch
leads
to
a
which
a2,
right
we start at a3, which is a match, but a4*b2.
The next attempt is more
Next,
away.
We
have a match from a4 to a-,, only
to have
a mismatch at a%. Now, we
promising:
need to backtrack severalstepsand to compare
bx to a5 (mismatch), then
b{ to a6, and
so on. Eventually,
find a match starting
we
at a ,3. We may have to backtrack and
of times,
of comparisons
compare
again a substantial number
leading to O(mn) number
by
we
that
AyxxyxyxyyAyxyxyyxyxyxx,
since b4*aA. We now must
throughout
and B=xyxyyxyxyxx.
start comparing b{ to
this
The
editing a text
section.
first
file.
mismatch
occurs
150
Involving Sequences
Algorithms
and Sets
B = xyxyyxyxyxx.
xyxxyxyxyyxyxyxyyxyxyxx.
10 1112
123456789
13 1415 1617
18
19 20
2122 23
xyxxyxyxyyxyxyxyyxyxyxx
1:
xyxy-
2:
3:
4:
\342\200\242
\342\200\242
'
\342\200\242
y
x
5:
\342\200\242
\342\200\242
6:
7:
y
y
\342\200\242
\342\200\242
xyxyyxyxyxx
x - x y
8:
9:
10:
11:
12:
13:
y
-
\342\200\242
\342\200\242
\342\200\242
xyxyyxyxyxx
Figure 6.20
in the
\342\200\242
\342\200\242
\342\200\242
worst case.
An
Noticethat
example
a lot
of a
of the
straightforward
work
is redundant.
string
matching.
For example, we
find
twice
of
the subpattern xyxy fits inside A starting at au (lines 6 and 11). In the example
a
in
a
will
since
word
text
of
be
the
number
file,
finding
very small,
backtracking
steps
most of the time the mismatch will occur early on. This simple algorithm
is fairly
good
that
for such applications. In other cases, where the alphabet is small and the patterns have
above
of backtracking
many repetitions, the number
steps may be large. The algorithm
to the same place in the text many times. We would
may compare the same subpattern
like
to find an algorithm that
avoids
such worst cases. The problem is to arrange the
on
information
we learn throughout the algorithm
later
such that it can be used efficiently
when the same matches occur in other places.
To improve the straightforward
the reasons for
we must first understand
algorithm
its
A
The bad case we discussed was caused by the need to backtrack.
inefficiency.
We
bad case will occur
and the text is yyyyyyyyyyyyx.
if the pattern is yyyyyx
particular
will compare the five ys in the pattern
to the text, find the mismatch with the jc, move one
step to the right, and make four redundant
again and again. (This simple
comparisons
case is easy to handle, but it illustrates the general problem.)
consider
On the other hand,
the pattern xyyyyy.
this pattern in the text, we look for occurrences of x
To
match
followed
there is no need to backtrack.
sufficient,
by five ys. If the number of ys is not
We
will need to find
the next x, and all the matched
will
The straightforward
not help.
ys
is
to
the
runs
in
since
no backtracking
linear
time
xyyyyy,
pattern
algorithm,
adapted
needed.
6.7
String
151
Matching
steps
A that caused
the
right. We continue
to the
mismatch
(a
in the
the
match
by comparing
the
already
same
character
know
that b
in
{
and
line
6 of Fig. 6.20, except
(In fact, that is exactly
in
three more redundant
\342\200\224jc
line
and
of
5,
comparisons
xy at the beginning
Notice that this whole discussion is completely independent
of A!
6 \342\200\224
to get there.)
line
We know the last few characters in A since they have matched B so far.
In the following
discussion, we will not assume that there are only two characters
will contain only two
even though, for simplicity, the examples
in the text (and pattern),
is the subject of Exercise 6.45) to make
It is possible (and that
the algorithm
characters.
even more efficient in this case.
The mismatch at line 6 of
Let's look at another
example
by continuing the match.
b
We
can
now
a
lot
more sliding. Consider
is
at
the
last
character
of
do
\302\243,{{.
Fig. 6.20
'''
=
the
B(\\0)
b{ b2
bw. We know that \302\243(10) is exactly the same as the
subpattern
b2 matched.
it took us
that
in A; that
We want
10 characters
is, B(\\0)=A [6..15], becausethey matched.
preceding
to the right until
there
is some
to determine exactly how many steps B can be shifted
for
a
maximum
of
of
We
determine
this
number
another
match.
by
looking
suffix
hope
In
that
suffix
is
of
as
is
that
is
to
a
of
B.
this
3
case,
#(10)
length
(xyx),
prefix
equal
is
at
a
is
in Fig.
one
and
illustrated
6.21. In the figure,
time,
shifted,
\302\243(10)
step
last character,
(The
bn, is ignored
compared to itself, until a prefix matches a suffix.
since it is the cause of the mismatch.)
Since we know that
\302\243[1..3]=\302\243[8..10], we can
We save
occurs.
continue by comparing
a{6 to b4, and so on, until the complete match
all
the comparisons
difference
on lines 7 to 12 and half those on line 13. The only
on B.
between
in Fig. 6.21 depends only
Fig. 6.21 and Fig. 6.20 is that the information
This is important
because
we can preprocess B once, and find all the relevant information
about
it regardless
of the text A. We now can take advantage
of all the matches done in
line
6 of
will
be repeated.
xyxyyxyxyxx
x
x y
Figure
6.21
Matching
the
pattern
against
itself.
Algorithms
152
Sets
and
Sequences
Involving
= the
next(i)
B (J),
0 if
and
maximum j (0 < 7
no such j exists.
< i \342\200\224
1) such
that
fe/__/-fe/_y+1
\342\200\242\342\200\242\342\200\242
=
bj_{
we define
convenience
next.
The matching
until
there is
character
in
match). If this
so on.
The
At
is compared
is a
mismatch
exception
only
1=
B
next=
The characters
as follows.
proceeds
a mismatch.
that
point, say
at
bh
the next
A are compared
to those in B
table is consulted and the same
in
10
11
-1001201234
Figure
6.22
The values
of next.
we want
is -1.
which
Algorithm
Input:
{
We
Output:
at A
value of next(\\),
the
by
Fig.
153
special
6.23.
String_Match (A, n, B, m) ;
of size n), and B (a string of size m).
assume
that next has been computed;seeFig.6.25)
Start (the first index such that B is a substring of A starting
(a string
[Start]).
begin
Start :=
0;
and i<ndo
= 0
Start
while
ifB[j]=A[i]then
i := i +
else
j :=next[j]
ifj = 0
/:=/
ifj = m
+ 1;
then
+ 1
/;
then Start
:= i - m
end
6.23
Figure
Algorithm StringMatch.
use
of the next table. We
algorithm to compute the values
=
We
assume
which
takes
of
the
base
case.
next
care
0,
mentioned,
(2)
that
the values
of next for 1, 2,...,/- 1 have been computed, and we consider next (i). At
In other
will happen if bj_\\ = bnext{i_])+\\.
best, next(i) can be next(i- \\) + \\, which
suffix that is equal to a prefix
is extended
words, the largest
by bj_{. This is the easy
case.
The difficult case is when
We need to find a new suffix that is
b(_{ *bnext{i_i)+i.
suffix of B(i-2): It
know how to fit the largest
equal to a prefix. However,we already
\342\200\242\342\200\242\342\200\242
fits in b{b2
But
(see Fig. 6.24).
bi_x *bnexni-\\)+\\ is exactly the
having
JVvf(/-i)
same
as having a regular
what
to do
mismatch
at bnext{i.\\)+\\\\
And we already know
about that.
If there
is a mismatch at index j\\ we go to next (J). So, we have a mismatch
at index next(i- 1)+ 1,and we go to next (next (/- 1)+ 1). That
is, we try to match bj_\\
to
= next (next (i - 1)+ 1) + 1. Otherwise,
bnext(nextu-\\)+\\)+\\ - ^ tnev match, we set next (/)
we continue
we either
in the same fashion until
to the beginning.
get a match or we return
It
induction.
to find an
remains
As we
6.2
\342\226\241
Example
Let B = xyxyyxyxyxx
next(\\Q),
which
(the
is 4,
and
same
compare
as
in
Fig.
bw
6.21),
to b5. If
and consider
they
had
been
next(\\
1).
the same,
We
then
first look
the
largest
at
next(i-l)+l
next(j)+l
at b5y
mismatch
happen to be the
The
to a
is equal
that
and we
look at
next(5)
same.
Hence,
next(\\ 1) = 3, which
for computing
The program
algorithm
to implement.
been
have
would
suffix
Computing next(i).
6.24
Figure
prefix
and Sets
Involving Sequences
Algorithms
154
the
which
table
next
is given
in Fig.
5,
but
can
is difficult
are
not.
compare
they
We now
is 2.
10 to b3y
be verified
easily
to
So, we have
but
understand,
by
hand.
it is
not
and they
\342\226\
difficult
6.25.
begin
next(\\)
:=-l
next(2):=0;
for i := 3 to m
do
j:=next{i-\\)+\\
whilebi_x
*bj
j:=next(j)+\\
and
j>0do
next(i):=j
end
Figure 6.25 Algorithm
A character
Complexity
there is a
pointed
mismatch,
then
the same
of A
may
Compute
be compared
character of
If there is another
.4 until there is
Next.
against
many
is compared
mismatch,
then
characters
of B.
If
of
either a match or we reach the beginning
How
we claim that the running time of this algorithm
is still O(n).
times
can we backtrack for one characterfrom
Let's assume that the
many
A> say aft
first mismatch involved bk. Since each backtrack
leads
us to a smaller index in By we can
to reach bk we must have gone forward k times
backtrack only k times.
However,
without
moves,
to the forward
any backtracking! If we assign the costs of backtracking
then we at most double the cost of the forward
moves.
But there are exactly n forward
\302\27
so the number of comparisonsis O (n).
moves,
against
B. Nevertheless,
6.8
155
Comparisons
Sequence
This algorithm
was developed
by Knuth, Morris, and Pratt [1977J, and it is known
for this problem was developed by
KMP algorithm. Another
fast
algorithm
We
The
is
and
Moore
sketch
it
difference between the algorithms
[1977].
briefly.
Boyer
B
from
the
end
rather
than
scans
from
the
the
that
Boyer-Moore algorithm
beginning.
be of bm against
then
the next
That
is, the first comparison will
am. If there is a match,
we use the
am_x, and so on. If there is a mismatch,
comparison will be of bm_\\ against
information, much as we did in the previous
algorithm, to shift the whole pattern to the
- \"Z,\" and Z does not appear
if am
For example,
at all in B, then the whole
right.
as the
can
pattern
comparisons,
6.8
The
by
m.
Comparisons
Sequence
that
attention
concentrate here on
required
section
assume
would
of sequence
subject
reason for
to
that
one string
change
is dynamic
Let A=a{a2
one
only
programming (discussed
\"'
'\" an and B-b\\b2
come
change
character
steps
throughout
this
of characters.
We
5.10).
two
be
bm
number of edit
strings
set (English
a finite
from
Section
We
biology.
technique used
main
The
another.
into
The main
lately.
molecular
minimum
the
finding
in
the characters
like to
\342\200\224
problem
attention
in
such
by character
that
string
abbe
insert an a
into the
between
two
bs (babe),
However, we can
changes.
and then delete the last c, for a total
three
of single-character
also insert
of
first
the
replace
b at
a new
a, forming
the
last c
the
beginning
the
with
bbc\\
string
a b
for a
of
babbc),
(forming
two changes.
then
total
the
number
changes.
The string-edit
maintenance. We
modification of the
delete
and then
we can
babb,
string
the
problem
has
also applications
to
file
comparisons
and
revisions
is a
It is convenient to extract the differences
between
the two
nles. There may be several versions of the same program, and, if the versions are similar
and
need to be archived, it is more efficient to store only
the differences
instead of
they
In such cases, we may allow only insertions and deletions.
the whole
storing
programs.
to each of the edit steps.
costs
In other
cases, we may assign different
may
first
have
one.
a text
file
(or
program)
and
another
file
that
156
Involving Sequences
Algorithms
and Sets
There
quite
a few
try induction.
we
corresponding to the
delete: if an is deleted
scenario above holds.
B(m)
insert:if
nothing:
change
from
be the
would
deletion.
more
one
minimum
the
the
induction)
from
change
bm, then we
match
to
then
\302\243,
one from
(n
other
In
the
- 1) to
words,
have C(n, m)
minimum
change
to B
insertion of
involves
1)+ 1. That
from A(n) to B(m-\\)
= C(n,
m-
a character
is, we find
(by
insert
and
equal to bm.
character
replace: if
change from
match:
in the minimum
to doing
= C(fl-l,m)+l.
C(/i, m)
to
and
then
and
edit steps
different
three
if
an
an
is
replacing
- 1) to B (m
A (n
to
is equal
bm,
bm,
-
then
we
then
1) and
C(n,
then
first
to add
m) = C(n
to
need
1
if a\342\200\236
* bm.
- 1,m- 1).
Denote
f
'<''\342\200\242\302\273
=|l
We can now
combine
if ai
bj
ifai*bj.
these
four cases
into
the
(deleting
C(A?-l,m)+l
C(rt,
m)
mini
C(a?,
m-
C(n - 1,m
withC(/,
0) = /
for all/,
recurrence
following
l) + c(rt,
0</<a?,
and
a\342\200\236)
1)+ 1
-
relation.
m)
C(0J)=j
( replacing
for
all
V matching
a\342\200\236),
y, 0<j<m.
It is not difficult
to prove that these possibilities are the only ones.
Consider an. It
be
handled
must
somehow. It is either deleted, which is handled by the first case, or it is
into some character in B. In the latter case, either an is mapped
is
into bnn which
mapped
6.8
handled
the third
by
or
fourth
in which
or it is mapped
case,
be inserted after an.
a character
into
157
Sequence Comparisons
appearing
before
bm,
with
this
is that we used induction
too
The problem
We
approach
many times!
of size (a?, m) to three problems of only
smaller
If
sizes.
we
reduced a problem
slightly
for
each
smaller
we
the
work
recursion
time
we
use
problem,
separately
every
triple
in
reduce the size by a constant. That leads to an exponential algorithm. Fortunately,
The key to this
this case there is no need to solve each
subproblem
separately.
too
is that there are not
different
observation
subproblems altogether. Each
many
< / <n,
involves
C
for
some/
and
(/,
j)
j in the ranges 0
computing
possiblesubproblem
and
There are
0<j<m.
for more
nm
than
of such
combinations
nm
is
This
subproblems.
/s
and
same
the
ys,
should
so there
not
be
we observed
phenomenon
a need
in
the
than
bound and
corresponding
will
be able
to use
We
to us.
available
To
compute
the figure.
already
order
version
We
create
of C(/,
the value
We
visited
traversal
of the
want
the
(i.e.,
to
j),
the
with
need
we
other entries
an
example
its bound.\"
the
subproblems
are
results
so that, whenever
necessary for its
approach is
greater than
solutions of all
the matrix
scan
three
if the
induction
strong
a table
is no
one
other
the
is sufficient.
to
left
we arrive
computation.
right)
of dynamic
programming.
at an entry, we
In this case, a
This
have
row-
two-dimensional
of the
Each
C[\\..n, \\..m\\.
entry C[i,j]
move
that
leads
to
matrix
holds the value of C (/, j). Let M [/, j ] denote the last
(change)
the
we can
we need only the last change is that
minimum
value of C[/, j]. The reason
of delete(i),
This move is any
one
backtrack and find all the changes from the matrix.
Implementation
We
maintain
a matrix
'
\\','
?/fPYZ:
fK%-T<
Figure
C(iJ)
C (/,
j).
Algorithms
158
insertij), or
[/
the
1 ],
j ], we need to know
last change can be determined
to the minimum value for C[/, j]. The
possibilities
Sets
and
Sequences
Involving
of C [i -
the values
C [/,
is given
algorithm
1,7],
of
to which
according
in
Fig.
6.27.
Algorithm Minimum_Edit_Distance
of size n), and B
A (a string
Input:
C (the
Output:
cost
(A, n,
B, m);
(a string
of size
m).
matrix).
Begin
:= /;
fori:=OtondoC[i,0]
forj:=
:=j;
tomdoC[0J]
for i := 1 to
forj :=
n do
1
do
torn
x:=C[i-\\J]+\\;
y:=C[U-l]+l;
ifdj=bj
then
z:=C[i-\\J-\\]
else
z:=C[i-l,y-l]+l;
end
Figure
Complexity
is clear
It
Comments
problem
is the
Dynamic
on many
depends
6.27
Algorithm Minimum_Edit_Distance.
from the
program in
need for
an
solutions of
conquer
6.9
The
programming
is
programming
store
the
O (nm)
in
approach
slightly
useful
usually
at
less
where
cases
in
smaller
which leads to
is O
time
running
(nm).
well.
The table is
least
quadratic
efficient
than,
of a table
usually
running
say,
solution of
the
The use
problems.
programming.
dynamic
is
that the
6.27
Fig.
space as
the
to
scanned
times. Thus
divide-and-
approach.
Probabilistic
algorithms
If we use
we discussed
a deterministic
Algorithms
so
far
were
deterministic
\342\200\224
every
step
was predetermined.
for the
6.9 Probabilistic
a simple
example
Suppose
with
continue
and
that
we
have
a set
that
start
the
finding
of
half\"
x {,
numbers
one of
For
(i.e.,
a \"good\" student
to
her
or
his
One
according
grades.
in the upper half). We have
is, of course, always
(which
the maximum
Another
requires n 1 comparisons.
possibility is
it
maximum-finding
algorithm and
one-half
number
that is greater than
reached. A
half. This algorithm
to prove that it
159
Algorithms
is greater
to
stop
of the
just after
numbers is
the
halfway
definitely
point
is
in the
upper
is not difficult
is impossible
about
Involving
Sequences and
transform
every
Algorithms
160
cannot
however,
efficient
Sets
Las Vegas
efficient
an
into
algorithm
deterministic algorithm.
The idea of
algorithm.
Numbers
Random
6.9.1
Probabilistic
for doing
methods
numbers
according
we select numbers
that
require
algorithms
efficient
We
random.
at
deterministic
must
find
procedure
generate
of the procedure.
If the
scheme is completely deterministic,
then
the numbers generated cannot be random
in the
true sense of the word.
in a specific way. Fortunately, this
They will relate to one another
is not a major practical problem. In practice,
it is sufficient to use pseudorandom
numbers.These numbers are generated by a deterministic procedure\342\200\224
and thus are not
\342\200\224
random
but
the
makes
the numbers
truly
procedure
any
among
relationship
that.
However,
any
on
will
the steps
by most applications.
It is beyond the scope
of this book to discuss this
in depth.
issue
We restrict the
discussion
to one very effective method, called the linear
for
method,
congruential
numbers.
The first step is to choose an integer
seed r(l),
generating
pseudorandom
unnoticable
which is a
r(/) =
where
can
t,
and
record
- 1)-ft+1)mod
t are
and
same
numbers.
range
0 to
t- 1.
avoid
The numbers
We
appropriate factor (t
can
should
end
should
hitting
by
the
t must
and
be quite
with
some bad
generated
linear
be done
large,
in
be
about
should
a*21,
with
cases that
a different range
chosen
to be a multiple
achieve
be
are
numbers
f,
constants.
in
time
current
the
rule:
following
The selectionof ft
the following guidelines: t should
suggests
be a power of 2 (or 10)if that is convenient;
ft
[1981]
and
(r(i
at random
selected
number
cause
range).
at least,
many
multiplying
of that
millions
the
even.
congruential
by
carefully. Knuth
method are
the
numbers
of
the
in
the
by the
6.9 Probabilistic
Let S be
each
a set
coloring
the
that
this
elementof
S, contains
subset
each
of
a collection
be
Sk
subsets
distinct
of S,
k<2'~2.
that
one of
S with
one
least
at
two
red
colors,
red and
at
least
one
is called
a valid coloring. It turns out that, under
there
is
a valid coloring.
We present a simple
conditions
subsets,
always
a probabilistic
that is adapted from
a
algorithm
proof of existence of such
The algorithm is almost as simple
as possible:
that
satisfies
condition
on the
given
probabilistic
coloring.
element of S and
Take every
1/2) independently
probability
This
Color each
Problem
or blue, such
blue element.
S\\,S2,...,
such
r elements,
exactly
containing
The
Problem
A Coloring
6.9.2
161
Algorithms
of
probability
of the
coloring
The probability
failure.
that at
probability
< 1 /4 (because
kl~r
it either
not always
does
obviously
algorithm
color
least one of
of the bound
the
on
k).
lead to a
valid
subsets
is colored
Let's
coloring.
of S,
elements
all
that
(with
calculate
only
that
is
no more
a random
the
The
than
coloring is
most
most 1/2 (sincethere is
1/4 of a subset entirely
This is a proof that
a valid
always exists (otherwise the
coloring
that the random algorithm
is
of
failure
must
be
It
also
probability
exactly 1).
implies
We
check
the
We
can
test
the
of
a
validity
coloring.
simply
very good.
easily
particular
colors.
We have a 50-50 chanceof
elements
of each subset until
we find two of different
we need to run
success.
If we fail, we simply try again.
The expected number of times
the algorithm
is clearly a Las Vegas algorithm,
to get a valid coloring is 2. The
algorithm
because
we check each coloring and terminate
only when we find a valid one. This is a
is at
valid
not
colored
blue).
often
a valid
6.9.3
so simple.
methods.
Next, we show
that
probabilistic
algorithms are
can be modified such that it finds
Unfortunately,
this
algorithm
coloring deterministically.
A
for
Technique
Transforming
Probabilistic
Algorithms
now
show
a deterministic
algorithm.
algorithm
We
into
to transform
use induction
the probabilistic coloring algorithm
into
algorithm. The technique we presentdoesnot work for every Las Vegas
it is possible
to transform
do not believe that
efficiently
every Las Vegas
a deterministic
is interesting,
however, because
algorithm. This technique
how to
at
first
reading.
it
Involving Sequences
Algorithms
162
idea of
the
employs
algorithm
resulting
problem,
general
Let S be
a set
of n elements,
The probabilistic
of S.
subsets
be efficient and
some of the restrictions
not only
removing
again
induction
the
strengthening
will
and Sets
in
a powerful
but will also
on the
imposed
on the fact
based
way. The
solve a more
problem.
original
of
a collection
...,\302\243*be
S 1,^2,
and
was
algorithm
hypothesis
deterministic,
the
that
distinct
that
probability
we get a
can
we
coloring
to an
lead
will
algorithm
by
probability of successremains
we are
Since
induction
hypothesis
The most
important
denote the
size
2~s,
The
l.
F(/i)
such
that
condition
of subset
probability
is not
elements
to
trying
+
\302\2432-5'
We are on
require
that
of
probability
S, is
that
probability
(i.e., the
of
we must
time,
all
with
Let s(
color
one
only
size.
same
the
nonzero.
remain
colored
the
strengthen
be of
subsets
success
that
probability
of
the sizes
solid grounds
Induction hypothesis: We
of
the
is that
at
coloring of
a random
is
all
as
the
long
sets,
but we
as F(n)
< 1.
write
it
function
as a
Let'stry
the
of
following
hypothesis.
provided
If one
one element
no longer
that
I.
F(n) is a function
induction
such
can color one element
color all elementsby induction.
If we
n.
we can
convenience.
for
then
handle
we
Sh The
of failure
This probability
n
on
induction
nonzero,
the
that
subsets
F(n)
has
know
how
to color
a set S with
<
n elements,
< 1.
only one
element,
then
this
element
contributes
to
F(n),
so
to be distinct,
F(n) cannot be less than 1. If a? =2, then, since the subsets are assumed
there can be only one subset with the two elements,
blue
and we can color one element
and one element red. Hence, the base
case is established.
We now try to reduce the
to one for n - 1 elements.
coloring problem for n elements
\342\200\224
Let a be an arbitrary
blue
element
of S. There are two possible
ways to color a*
or red. Suppose that x is colored blue. What
is the probability
that a random coloring of
the
other
n - 1 elements
is valid?
A subset S, that does not include x has the same
\342\200\224
of failure
a has one fewer
2~s,+ . A subset Sj that
includes
probability
namely
element, and it only needs to have at least one red-colored element (it already has a
blue-coloredelement).
subset Sj is 2 s'
to color
the probability
of failing
Thus,
Noticethat this probability is the same as it was before we colored a! Therefore,F(n)
remains
less than
Does that mean that
1, and we now have to color only n - 1 elements.
we
now have an algorithm?
No. It means only that the first choice
can be made
After
the first choice is made, the problem
is different.
arbitrarily.
We can no longer use the same induction
because, after we color the
hypothesis,
need to be colored with
first element, some of the subsets
two colors,
and some of them
one color. We have to strengthen
need
to be colored with
the induction
only
hypothesis
further to reflect this change.
that some elements are already
Suppose
A subset
colored.
6.9 Probabilistic
of four
163
Algorithms
may
do not
(1) the
states:
it
to consider
have
F(n)
(6.1)
2fi<l.
/=i
induction
The
to include
problem
collection of subsets,each
is
(6.1) satisfied.
now a
The Problem
at
The induction
extension
is the
hypothesis
of the
each
red
the
The base
We
each leading
We extend the
words, the input is
all subsets.
In other
assume
We
neutral.
that condition
element
subset
hypothesis for
this
(nonstraightforward)
problem.
of the
conditions
case is similar
that (6.1) is
blue, or
straightforward
Induction hypothesis:We
satisfy
red,
of
subsets.
labeled
status
to
know
how to color a set S with
problem, providedthat (6.1) is
< n
elements
to
satisfied.
n elements
the previous base case. Given a set 5 with
one element of 5 and to leave (6.1) satisfied.
color
such
satisfied, we need to
pick
again
to
an arbitrary
different
element x e
S. There
are
two
possible
x red, then
red
subsets
red
all blue subsets containing
containing x remain
x become successfullycolored(and can be removed), and all neutral
x
subsets
containing
become red subsets. Subsetsthat do not contain x are not changed.
leads
x
blue
Coloring
to similar
We can now compute the value
of F(nwe denote
1), which
changes.
by
x
red.
We
color
we
that
also
to
indicate
denote
the
1),
FR(n
by FB(n 1)
corresponding
value of F(n1) in the case when x is colored blue. The key to the algorithm is the
following
lemma.
Involving Sequences
Algorithms
164
and Sets
6.2
Lemma
\342\226\241
be
Let F(n)
the
of failure
probability
initially,
x red,
coloring
after
failure
of
probability
of failure
probability
blue.
coloring
after
and
let
FR(n-\\)
be
the
let
FB(n-\\)
be
the
Then,
FR(n-\\)
FB(n-\\)<2F(n).
Proof:A subset that
Its contribution
x remains
to
does not contain
unchanged.
with
is
the
claim.
which
consistent
is
the
We
and
same,
F(n)
FB(n-\\),
/^(fl-i),
consider now the subsets that contain x. There are three possibilities,
according to the
it
is now
because
to
subset
contributes
A
red
status.
subset
(1)
FB(n-\\),
nothing
to
is twice as much
as that
to FR(n-\\)
colored; its contribution
F(n),
successfully
with
the claim.
this is consistent
because it has one fewer element. Again,
(2) The case
of a blue subset is the same as that of a red subset. (3) A neutral subset with w, elements
+'
one less element.
red or blue with
This subset becomeseither
to F(n).
2~\"'
contributes
In
either case, its
2~(M_1)
and
to
both
contributes
Thus, it
FR(n-\\)
FB(n-\\).
is
the
claim.
to F(n), FR(n
\342\226\
contribution
1) the same, establishing
1),and FB(n
6.2 leads
Lemma
directly to
the
can
there
one
be only
then we
subset,
of one element
base case
The
algorithm.
is simple,
subset
blue
element
containing
the other
with
the
reader.
a Majority
Finding
\302\243
be
a sequence
in E.
x appears
times
Given a sequenceof
or determine that none exists.
Problem
The
sequence
For example,
whether
an
someone
integer
can
won the
can be
We
assume
here
a vote
represent
election. If the
that there is no
limit
problem
multiplicity
E if
find the
numbers,
in
number
an
of x
election,
of candidates
in O(n)
in
its multiplicity
E is the number of
is greater than nil.
majority
and the
in
the
problem is to
is small, then
bucket
find
sort
on
decisions.
6.10
This
a Majority
problem
straightforward
thinking
Finding
but
is
whose
problem
that
first
165
requires
some
several
discuss
to the
approaches
straightforward
Xi * Xj
majority
in
and we
the
eliminate
original
both
of
list remains
these
a majority
elements
in the
from
the
list,
then
the
new list.
but if we remove 1
the opposite
is not true: the list 1,2,5,5,3 has no majority,
5 becomes a new majority.)
in the
smaller
find the majority
So, if we find two unequal
votes, we eliminate both,
it
is
in
not
check
whether
the
list.
if
we
do
find
and
a
What
list,
majority
original
unequal
votes? If we scan the votes and they are all equal, then we have to keep track of only one
possible
candidate; once we find a vote that is not equal to this one candidate, we can use
the
If all the remaining
observation
above.
votes
are equal, then we have to check only
one candidate.
This is the seed of the idea; we now show how to implement it.
The
votes
are scanned in the order
C
variables,
they appear. We use two
When
and M (multiplicity).
we consider
is
for
the
candidate
C
(candidate)
xh
only
so far
C appeared
majority among x,, x2,...,*/_i, and M is the number of times
In other
eliminated.
the
times
was
C
the
votes
can
be
words,
excluding
x{,x2, ...,*/-i
into two groups of sizes 2k and M, such that 2\302\243+M = /-l,
divided
the first group
can be eliminated by the observation),
votes
k pairs of unequal
contains
(which
and the
If there is a majority
M times.
x
second group contains C appearing
\342\200\242\342\200
among
i, x2,
*,_! ,
elimination
this
then it must \"survive\"
scheme, and so it must be equal to C. (Notice
(Notice that
and
2, then
166
the
that
again
majority.)
is
opposite
not
survive
it to C,
C may
true;
consider
we
When
and Sets
Sequences
Involving
Algorithms
jc,- we
compare
the
elimination
and
either
being the
or decrement
without
increment
on whether
depending
multiplicity
of having
in Fig.
is given
algorithm
6.28.
(X, n) ;
of size n
Majority
Algorithm
Input:
array
(an
Majority (the
Output:
of positive
numbers).
if it exists, or -1
in X
majority
otherwise).
begin
C:=X[1];
M:=l
{
;
all
eliminate
scan;
first
but
candidate
one
C}
for i := 2 to n do
ifM = 0 then
C:=X[i];
M :=
else
ifC = X[i]thenM:=M+1
M
:-
else
{ second
ifM-0
then
C is
a majority
:= -1
Majority
else
Count :=
for i :=I
0;
= C then
ifX[i]
if Count
n do
to
Count
:= Count
then Majority
> nil
elseMajority
:=
+ 1;
:= C
-1
end
Figure 6.28
We use
Complexity
the worst
at
most
2az-2
3n/2+ 1, and
constant
- 1
comparisons
case, to determine
number
whether
this
Algorithm
Majority.
to
a candidate
find
candidate
and n
is a majority. Thus,
comparisons,
in
that
6.11 ThreeProblems
167
Techniques
Three Problems
6.11
Proof
Interesting
Exhibiting
Techniques3
this
In
we present three
section,
Each algorithm
the
is an
of
principle
strengthening
times during
hypothesis
four
algorithm.
The second
6.11.1
not
a sequence
have Xi<xir
The
we
xit,
obvious
an
strengthen
efficient
\342\200\224
technique
improving
that this
shows
to
improve
an
wisely.
xx,x2,...,xn.
subsequence (IS)
An increasing
such
(LIS)
subsequence
for all
that,
of 5 is
longest increasingsubsequenceof
< k,
<j
an
we
increasing
a given
integers.
this section is
excellent
an
We
hypothesis.
of problems
a result
will
encounteredin
the
principle of
of the
example
the
strengthen
hypothesis
attempt.
previous
several
Consider
induction.
straightforward
Induction
induction
Subsequence
increasing
longest
in
develop
the induction
as
induction
integers
Find a
of
know
of
an example
with ix<i2<
jc,2 ,..., xik,
of distinct
algorithm
the
distinct
Problem
of
length.
strengthening
the
Increasing
of maximum
sequence
The
of
the base
choosing
subsequence
algorithm
always
a subsequence
5 is
of this
development
is
the
an
to
solution, leading
We
hypothesis.
Longest
5 be
Let
of
the
technique.
proof
induction
involving
problems
different
by eliminating all
the \"theorem\"
principle is
algorithm by
unrelated
of a
the
example
how to find
a longest
Given
sequence
of size
<
m,
we
subsequence of it.
increasing
Given
a
which the problem is trivial.
- 1 elements, and consider
xm. If xm is
then
greater than the last element in the LIS, given by the induction,
xm can be appended
to the LIS, creating a new longer LIS, and we are done. Otherwise, however, it is not
clear how to proceed. For example, there
be several different LISs and
xm may
may
extend one of them,
the one found
but not necessarily
The next step
by the induction.
seems to be a strengthening
of the induction
hypothesis as follows:
The
base
sequence
case consists
of size
Induction
we
*
know
m,
we
of sequencesof
find an LIS of
(second
hypothesis
how
to find
at
all
first
the
first
attempt):
longest
reading.
1 for
size
its
Given a
increasing
sequenceof
subsequences
size
of
it.
< m,
The
check
is still trivial.
case
base
all of
against
xm
and Sets
Involving Sequences
Algorithms
168
We use induction
the LISs
in the
same way,
a longer
whether
find
and
have to
\342\200\224
we now
another
we can
solves
LISs.
problem
be
IS
1
less
still
an
of
than
the
LIS,
may
length
any
new
seems
we
will
a
LIS.
It
that
have
extend
create
can
which
it,
gotten
longest, and xm
have
to find all ISs of largest and second
into a hole, because we now
ourselves
largest
ISs,
largest
length. But to find all the second largest ISs, we will need to find all the third
where
the
then all fourth largest, and so on. This is a good example
strengthening
the previous
If xm cannot
induction
problem,
it introduces
but
extend
We
there
then
is overdone.
hypothesis
look back at
only to know
Let's
need
all
find
the
induction
stronger
extend
xm can
whether
hypothesis.
one of them.
Can
find
the
is positive.
The best LIS is
of extension? The answer
one
\"best\"
the smallest
If we can extend any LIS, we can surely
number!
that ends with
one
the
LISs
the same
that end with
extend this one. (There may be several different
number,
we talk about
and they are all equivalent in terms of extension potential. For simplicity,
best one.\")
Let's try another
induction
\"the best one\" instead of \"an
arbitrary
a
this
one
little
weaker
than
the
last
one:
hypothesis,
terms
in
of potential
know
increasing
longest increasingsubsequenceof
The basecaseis
the
then
found
LIS
have
we
so
still
Given
trivial.
by the induction.
a new LIS, which
it is
Given
attempt):
a longest
we
jcm,
is
can
longer
no other
whether
determine
it
can
be appended
s. If
of length
than the previous one; thus,
and we are done. Otherwise,
we are still not done. It may
that
Assume
we
that
subsequence of
a smaller last number.
such
it,
it has
of size < m,
a sequence
the LIS is
xm can be
LIS is
new
this
know
to
added,
that no
one,
definitely
be the case that
longer increasing subsequence exists. But
be added to the best LIS (since it is smaller than the last number in that LIS),
xm cannot
but it can be added to an IS of length s - 1, making
the latter
an LIS with a smaller
last
number. To accountfor this possibility,
we need to know
the best IS of length
s - 1. But
then
if the induction
the best IS of length
states that we know
5-1,
again,
hypothesis
then xm may extend an IS of length
We
s-2
new
IS
of
s1.
it
the
best
length
making
will
have
to be able to determine whether
in
with
IS
order
to
extends
such
an
xm
proceed
the induction.
s - 2, s - 3, and so on down
So, we will need to know the best IS of length
to the best IS of length 1, which
so far.
is simply
the smallest number in the sequence
(Even without
ISs cannot
be discarded
shorter
one can see that
induction,
using
\342\200\224
there
is always the possibility
that
one of these ISs is the start of the final
arbitrarily
unique,
the
\"best\"
we
LIS.)
Yet again
we
try
to
increasing subsequence of
number (if there is more than
denote
by
B\\S(k).last
Induction
know
how
the induction
strengthen
best
the last
one
hypothesis.
k \342\200\224
namely,
length
such
number
(fourth
hypothesis
to find BIS(k)for all
in
the
the
subsequence
sequence
denote
We
we take
an
BIS(\302\243) the
the
arbitrary
smallest
one).
BIS(\302\243).
attempt):
Given a sequenceof
if
<m-\\,
by
with
they
exist.
size
< m,
we
We
> B\\S(k).last,
xm
with
< B\\S(J-
B\\S(J).last
better than
values of
\\).last
1).
for
B\\S(i).last,
the
and
induction.
(2) xm <
BIS(\302\243+
1).
that BIS(l)./atf
of
the LIS.
This claim is
y, then the
first
j-
so
\\).last,
BIS(\302\243)
< B\\S(2).last<
true
would be
BIS(s)
and
it
unchanged.)
is quite
xm,
creating
we have
BIS(/>m.
simple
binary
by
performed
with
Otherwise,
We then
).last.
extend
we
search,
searchinga sorted
the
6.11.2
common
which is
technique,
for
thoroughly
proof
\"One should
assumptions;
should
boundaries
simple,but
and second
are
assume
or steps
that
almost
not
are
any
is to
theorem,
essential.
search the
such
Removing
from
that
a?
the
is a
better
the
theorem.
proof
set S of
n numbers
\\,
jc2,
..., xn,
the
number
find
the
first
largest of them.
for
looking
elements
that
Given a
Problem
The
We
in
an indication
sometimes
in proving
important
assumptions
is
also
Having inessential assumptions
be
and
[1927]:
Polya
wrong.
Quoting
Szego
may
in fact
scrutinize
each
made use of all the
proof to see if one has
fewer
. .. and
one should try to get the same consequence from
assumptions
at
the
not
be satisfied
until
show
that
one
has
arrived
counterexamples
is true for algorithms. This principle sounds
of the possibilities.\" The same
times it is not, as seen in the next
example.
many
assumptions results
one
total
of this
running
if
because,
of BIS(/)
numbers
can
This
following
it
occur: (1)
conditions
claim
two
the BISs
of
which
find
169
Techniques
We
size
the
have to
BIS(\302\243),
BIS(\302\243+
some
for
\342\200\224The
BIS(y
to
be added
can
xm
xnn we
and only if
BIS(\302\243) if
Proof
Interesting
Exhibiting
Given
trivial.
a certain
xm extends
change.
Problems
Three
6.11
an
set.
algorithm
that
minimizes
We ignore other
power of
only
operations.
Furthermore,
of comparisons
for
simplicity,
of
we
2.
try
the usual
170
Involving
Algorithms
one is betweenthe
approachleads to
\"loser\"
the
and Sets
Sequences
and
recurrence
This
Fig. 6.29).
(see
7(2)=1,
is
solution
whose
2n -3 comparisons,and it is very
than
the straightforward
T(n) = 3n/2-2. Thisis better
minimal
the
maximal
and
elements
of
the
to
similar
finding
presented in Section
problem
to
do
even
better.
want
We
6.5.1.
are necessary for the inductive
can
we
If the two comparisons
step, then how
at the comparisons
of comparisons?
in Fig.
Looking carefully
improve the total number
in the algorithm.
Therefore, the computation
6.29, we see that q2 will not be used further
If we can avoid this computation,
was
then
we will
its
to
unnecessary.
discovery
leading
of comparisons.
to
we
do not
However, until we compare
save significant number
p {
q {,
If
or
can
be
we
had
known
which
subset
was
whether
know
p2
q2
ignored.
going to
have
the
we
could
used
for
this
then
subset,
\"lose,\"
regular
maximum-finding
algorithm
So, we suspectthat
saving many comparisons.
which
ones they are.
we do not know
quite
a few
comparisons can
be avoided,
but
size
< n,
candidates
largest
we
we
do not
know
element
assume
how
until
the
that
we
to find
the
end.
know
element.
pi
p2
\302\2532
Figure
correspond
6.29
the largest
Finding
comparisons).
to the
and
second
largest
elements
(the dashed
lines
6.11
of Cq
elements
q x to
only
At the
CP.
Proof
Interesting
most
is at
q { which
largest
second
the
largest,
set of candidates
and a
element
171
Techniques
which
from
for finding
which
additional
-1
\\og2n
n-
the set we
of
size
set, because
candidate
maximum
for
eliminated.
by
of size
most
be a multiset
of (not
(jc!, jc2, ...,*\342\200\236)
ordered set. A mode of a multiset is defined
is called
its
will
minimize the
be required is that
mode is
try
the
mode of a multiset
This is not easy
element
may
not
finding
then
high,
Let's
is
candidates
for
find
the
second
the
there
the
necessary.
the majority (Section
may be
can scan
a fast
way
suspect
it
multiplicity.
that,
without
to
way
sorted
mode
sequence and
find the
sorted sequence). We
that
thinking
of finding
induction
straightforward
with
n - 1 elements, and try
the
in the
consecutive
This leads us to
occurs
element
an
highest
One possible
always
in
frequently
S.
multiset
are sorted, we
be
of times
with the
element
of comparisons.
will
occurs most
that
a totally
from
elements
distinct)
The number
mode of a given
elements
0(n
requires
sorting
how to
know
we
n,
necessarily
as an element
mode).
thus
(equal elements
the
multiplicities
see that sorting
one
mode is
number
the
Once
sorting.
than
Find a
Problem
Our goal is to
count
The
multiplicity.
The
is to use
<
\\og2n
The
Modeofa Multiset
5 =
the multiset
of
number
more efficient.
which are
double the
element requires
is thus
comparisons
is constructed,
is a good
Once an algorithm
it
idea to examine it
to the final
result.
these parts can be
Often,
parts that do not contribute
if the redundant operations cannot be eliminated,
Even
they
may be replaced
operations,
simpler
set when we
largest
element.
maximum
Comments
second
hypothesis,
carefully
candidate
which,
+log2Ai-1,
induction
the
the
if the
may not
whereas
time,
sorting
in linear
of
multiplicity
the
sorting.
know
the
multiset.
the
tie,
Then, we can
multiplicity.
highest
but it may also increase the multiplicity
determine
of another
whether
number,
the
nih
which
Sets
Sequences and
Involving
Algorithms
172
track
already seen (Section6.13.1)that keeping
will
be
the
cost
too
On
but
is
high.
probably
possible,
of all
\342\200\224
be
we
can
element
choose
a
Arth
is
not
that
it
the
arbitrary
necessary
the other hand,
maximum
element.
We
still
is
the
have
the
Arth
element
that
special one. Suppose
We
can
but
now we are closer to a solution.
basically the same problemsas before,
but
all
occurrences
of the maximal
reduce the size of the problem
by removing not one
the multiplicity
of the mode
element. We then solve the reduced problem, and compare
element.
of the maximal
with
the multiplicity
of the reduced multiset
it is still
but unfortunately,
too slow. Finding
the
We now have an algorithm,
- 1
n
If
the
multiset
contains
of n elements
of a multiset
maximum
comparisons.
requires
to be
has
now
added to
distinct
a few
quite
In
performed.
this
then
so
disjoint,
two
is basically
the
same
of
of using
the multiset into
lead
they
We
have
multiset
the
seen
already
6.5). It is
although we have not
(Section
in
our
splitting,
multiset
can be
smaller
multisets
linear
we get
time,
the
T(n)<2T(n/2)
which
is basically
algorithm
We
at the
that
now
come
base of the
we can
parts
start
to
the
find
same
divide-and-conquer
The two
size.
of
subproblems of size no
the
disjoint.
Since
familiar
recurrence
+ 0(n)9
7(2)= 1,
the same
to the
heart
induction
more
be
into
multiset
and
the
the median.
of
in
comparisons
the
worst
algorithm
median-finding
find
should
a multiset
parts
than
and
than
induction, we
in the
elements
greater
equal to,
in O(n) expected number
in O(n) time
median
We use the
size n, we first
result.
multiset
induction.
the
the
than,
a median
find
this
\342\200\224
less
element
splitter
parts
of about
that
implies
small set of
or a
algorithm,
possible
proved
Given
algorithm.
then solve two
as the
the
a set
fact
to independent
how to
also
in
element
two
equal disjoint
into three
approximately
then split
one
maximum
many
multiset is
the performance
Instead
that
too
then
if the
particular,
improve
technique.
to divide
try
solutions
elements,
algorithm
To
We have
list.
the
\"best\"
different
case,
as a step
the
nil.
of the two
the median and
modes
finding
relation
In fact, if the
this is no better than
sorting.
instead of being the exact
then this
median,
But
as quicksort.
of this
Suppose
from
To
algorithm.
that the multiplicity
improve the
of
submultisets of size M.
In
we look
performance,
the mode
other
is M.
words,
We
claim
we do
not
M. Since
all parts are
into parts smaller than
splitting the multiset
one of the parts of size M must
contain
disjoint,
only the mode. At this point, the mode
will
be discovered
because the multiplicity
of all other elements cannot exceed M.
there
is no need to divide the multiset
Therefore,
any further.
The implementation of this algorithm
We cannot use
is not straightforward.
we do not know
because
The
beforehand
how far to carry out the recursion.
recursion,
when the size of the multiset
recursion should
be terminated
becomes
at most M, but the
the execution
value of M is found
of the algorithm
the smaller
all
during
by checking
In each step, all the submultisets
multisets.
are checked,
and, if none of them contains
have
to continue
one distinct
only
element, then
contains
details
Complexity
The
which
of
this
number
the
this is
c, then
constant
6.12
and
linear-time
often
try to
therefore
we employed
in
of
Many
be obtained
the size
of
are
this
a major
plays
the
subjects
basic
abstract
are quite
chapter
input
also better
Unfortunately,
discussed
[1973b].
combinatorics,
binary-search
subsequence
paradigm
problem
are
problems
practice,
5. Induction
in Chapter
about
presented
can
and
Further
in
searching,
involving
and
Stanton
be found
of Section
in
6.2 is
Manna
from
too
are used
If
the size
Whenever
algorithms
and
algorithms
not
often
most
that
examples.
of the
input
inefficient
sorting
solutions
have straightforward
chapter
search
and selection sort are two
this
in
solutions.
sophisticated
Additional
are
problems
and others.
algorithms,
In
order
sorting,
searching,
for
often
but
is not
The use
of
common.
inputs.4
large
Reading
including
sequences
White
and
their
and
[1986].
history,
Waldinger
Mirzaian
be found
can
sets, as well
[1987],
formal
of a
derivation
[1987]. The
where
in
as topics in
stuttering-
a linear-time
example highlighted this issue unexpectantly when the virus (or worm) that attacked over
the United States on November
2, 1988, slowed down those computers considerably,
all the search algorithms in it used linear search (see Spafford
[1988]).
recent
computers across
because
high
fairly
overhead for
\342\200\224
subject.
of a given
is small,
than
Notes
of material
wealth
parts
\342\200\224
linear
effort
little
Bibliographic
4A
is
if M
is an
this
role.
the problems
with
as are the
main
the
Knuth
= cn for
if M
a significant
probabilistic
in each
problems
small (e.g.,
only
a linear
takes
then
constant,
of
a multiset
particular,
sorting
is
chapter
manipulation,
string
two
this
the
in
as clean
not
should
one or
only
presented
they
In
intuitive
An
Summary
can
base:
subproblems).
remembering
again
implementation
is thus superior
to
0(n log a?) algorithm. This algorithm
if the cost of comparisons is also high
and
(there
We
in its
submultisets
the
is that
expression
is
size M is encountered,
which
number of comparisonsto divide
some
is modified only
relation
of
We leave the
T(M)= 0(M),
+ 0(n\\
that
implies
explanation
If any
divided.
further
we can terminate.
then
recurrence
resulting
T(n)<2T(nl2)
are
of them
all
element,
173
Reading
6(XX)
partly
Algorithms
174
and Sets
is presented.
algorithm
Perl,
Involving Sequences
Itai,
and
Avni
search
was
studied
by
[1979].
in 1945, and it was one
version
of mergesort was
[1988], and Dvorak and
first developed
by von Neumann
to
be
of the
implemented. An in-place
programs
and Langston
first
by Kronrod [1969]; see also Huang
developed
Durian
Quicksort is due to Hoare [1962]. A detailed
study of quicksort appears
[1988].
A wonderful film
[1964].
in Sedgewick
[1978].
Heapsort was developed by Williams
nine
all
shown
with
beautiful
of
major
sorting
techniques
descriptions
containing
at
the
of
Toronto
was produced by the computer
animation
graphics group
University
has
been
studied
for
there
are still
Even
extensively
many years,
sorting
though
[1981].
exact
number
of
The
for
n
numbers
comparisons
required
sorting
open problems.
many
outlined
in Exercise 6.30 is by Ford and Johnson [1959].
is still unknown. The algorithm
for some time in terms of number of comparisons,but it was
the \"champion\"
It was
used
to
be
not
[1979]. Another widely
by Manacher
sorting
algorithm is
optimal
proved
is
Shellsort
and
to
Shell
shellsort invented
[1959].
very
by
simple
easy
implement.
is still unknown; see Incerpi and
for recent
However, its complexity
[1987]
Sedgewick
Decision trees have been used successfully
observations.
to prove
results and empirical
several
Moret
a
bounds
for
basic
of
their
uses.
lower
[1982] presents
problems;
survey
selection
An analysis of the probabilistic
and Rivest
algorithm was given by Floyd
deterministic
[1975]. A linear-time
algorithm for order statistics was first developed
by
stored
first
Blum,
Floyd,
Pratt,
and
Rivest,
the constant
CD(n)
a
median
presents
finding
bound
(on the number
since
is
very
algorithm
of
Tarjan
high.
with
comparisons)
time
However, the running
and
Schbnhage, Paterson,
Pippenger
at most 3n comparisons. The best-known
for finding the median is 2n (Bent
[1972].
is
in
fact
[1976]
lower
and
John
order
statistic
paper contains results for the general
problem; the
for the general lower bounds are more
expressions
complicated.
Data
has been studied widely
due
to its great importance. The
compression
in
Section
6.6
is
due
to
Huffman
also
Knuth [1973a]). Variations
[1952]
algorithm
(see
of Huffman's
that
use
one
are
described
Knuth
and Vitter
[1985]
algorithm
only
pass
by
Another
More
effective
and
is
due
to
Ziv
and
[1985].
[1978].
popular algorithm
Lempel
on data compression in general
can be found in Lynch
[1985].
The string-matching algorithms
in Section 6.7 are due to Knuth,
Morris,
presented
and Pratt [1977], and to Boyer and Moore [1977]. Galil [1979]improved
the worst-case
time of the
of the
More on the
running
Boyer-Moore
algorithm.
complexity
be found in Guibas
can
and Odlyzko
Boyer-Moore algorithm
[1980] and in Schaback
[1988]. Empirical comparisons betweenvarious
algorithms can be found
matching
string
in Smit
A
was
[1982].
probabilistic string
developed by Karp and
algorithm
matching
Rabin
short
to
make
This
of
idea
uses
the
[1987].
fingerprinting
algorithm
be
also
of
It
can
so that they can be comparedefficiently.
representations
large strings
to
used with two-dimensional
be
extended
can
The string matching
patterns.
problem
look for patterns
are
than
\"wild cards\"
more
For
complicated
just strings.
example,
we may want to search for all occurrences
of strings of the form
useful;
B*C, where B
* denotes
and
C are given strings and
any string. A more general problem is to look for
more
For
on
sets
of
these problems
see Aho and Corasick [1975].
any
strings.
regular
[1985]).
This
175
Drill Exercises
Another
Suffix trees
preprocessed.
and Myers
(Manber
Sankoff
and
in
strings
text
and
fixed
[1976])
McCreight
[1973],
(Weiner
comparisons
Sequence
search for
is to
problem
important
been
has
that
Suffix
Arrays
search.
their
many
are covered
applications
Various
a book
in
[1983].
subsets is NP-complete(Lovasz
for arbitrary-sized
present many
edited by
Salzberg [1982]
steps) can be reduced
excellent
which
that
by
tournaments,
mode
in
by Lewis
is given in Dobkin
The solution to
set
a very
of
length
was
first
Carroll (see
and
one presented
Section
in
6.10,
Using
Fischer
number
the
elements
the
than
structure
decreasingsubsequenceof
largest elements
[1982].
and
of comparisons (but not the number of other
1 in the worst case, and
that this bound is optimal.
to 3/7/2+
of a solution to the longest
subsequence
increasing
description
Erdos and
we
is given by Gries
borrowed
[1981].
heavily)
data
showed
An
[1974]
properties.
and Gries
Misra
in
and Spencer
combinatorial
proving
sophisticated
problem (from
Erdos
[1973]).
techniques for
of probabilistic
examples
The majority
a more
are
included
a book
problems involving
strings
in
is
The algorithm given
6.8
due to
Section
edited by Apostolico and Galil
[1985].
in many
This algorithm can be improved
[1974].
ways, including
Wagner and Fischer
times
when the alphabet is
running
savings of storage (Hirschberg[1975]),improved
and
when
the
are close (Ukkonen
and
(Hunt
[1977]),
sequences
Szymanski
large
very
in
A
relevant
results
of
and
[1983].
survey
appears Hirschberg
[ 1985]
Myers [1986]).
finds an element in the upper half is due to Yao
that
The probabilistic algorithm
in Knuth
is covered in detail
number
[1981]. The
generation
[1977]. Random
in Section 6.9.2 is based on a probabilistic proofof
given
probabilistic coloringalgorithm
for
in Bollobas [1986]. The technique
existence
probabilistic
converting
given
in
is due to
which
was
illustrated
Section
deterministic
to
6.9.3,
algorithms,
algorithms
use
this
to
solve
the
of
Section
The
of
[1986].
technique
coloring problem
Raghavan
6.9.2was pointed out to us by K. Pruhs. The general problem of finding a valid coloring
Kruskal
and
Munro
suggested,
[Knuth
1973b]).
in
principle,
pigeonhole
have
either
of finding
the
context
Another
the
that every
an increasing or a
and second
largest
of arranging
algorithm
for
finding
tennis
the
't [ 1984]).
Exercise6.34is
from
Drill Exercises
6.1
well-known
strategy for the following
game: One player thinks of a number
other
The
player attempts to find the number by asking questions of the
range
than) a?\" The object is to ask as few questions as
form \"is the number less than (greater
that nobody cheats.)
possible. (Assume
Design
in the
a good
to //.
176
6.2
Algorithms
choiceis unknown
a strategy
Find
\342\200\224
that is,
6.3
Sets
and
Sequences
Involving
handles
large texts, for example, a word
using a program that
a
as
The
takes
text,
represented as a sequence of
input
program
processing
an error
some output. Once in a while, the program encounters
and produces
characters,
what error it is, or
recover.
Not only that, but it cannot even indicate
from which it cannot
is to halt and to output
action the program takes
it is. In other words, the only
where
Assume that the error is local; in other words, it results only from a particular
4'Error.\"
string in the text which the program, for some unknown reason, does not like. The error is
a strategy
of the context in which
the offending string appears. Suggest
to
independent
locate the source of the error.
that
Suppose
are
you
program.
6.4
6.5
Construct
an
example
searching
in a
table of
6.7
6.11
this
6.12
for
if
to the skyline
to
use one solution
possible
invariant
loop
of the
Construct
an
chosen by
taking
cases,
and
for
the main
statement
\"if
Left
* Right.\"
problem
almost
loop in
Mergesort
(Fig. 6.7)
will not change if we
(Fig. 6.9),
Partition
Algorithm
and
quicksort
of the
first,
use
will
last, and
Q(n2)
comparisons
the
when
pivot
is
sequence.
almost
is already
sorted, which means that
elements is small. Describehow the different sorting algorithms
6.4 perform for almost sorted inputs. Which algorithm would you use?
input
in Section
suggested
The
the median
the
search)
binary
of out-of-order
the number
Design
and
algorithm.
for which
example
the if
with
the solution
the correctness
6.14
number
it be
with
Will
the appropriate
Construct
search
linear
sort
to
statement,
prove
6.13
(with
sort.
Write
(You are
sort
insertion
used
Compute the
selection
Comparemergesort
In some
for
programs
range
a loop invariant,
that the first if statement in algorithm
In other words, prove
that the result of the algorithm
the similarities.
other problem?
6.11
is in the
digit
(by quicksort).
Prove, by using
is not necessary.
remove
6.10
Each
of comparisons
number
the
Count
6.9
comparisons
Q(n)
for radix-exchange
program
digits.
Fig.
6.8
use
will
Write
each with k
6.6
for
size n.
encouraged to
a table
design
similar to that
a divide-and-conquer
algorithm
should
your own.)
in Fig.
algorithm to
use at most
3n/2
less than
find
the
a heap
minimal
comparisons
the straightforward
(for
top down.
and
In
elements
maximal
n=2k).
Can
- 3 comparisons
you
in a set.
pinpoint
algorithm?
the
6.15
the Huffman
tree for the set of characters in this question.
many bits are saved in the storage of this question using
storage based on a fixed-length encoding?
Include
Build
How
6.16
Construct
6.17
Construct
the matrices
baacbabaccaba
6.18
using algorithm
the appropriate
loop
invariant
the correctness
of the
first
Unless
are
that
Given
an
Prove
6.22
not
is a
problem
algorithm
that
your
the
of size
n,
if they
time
consist
to
and
run
time
in
of
O (n).
6.19 is optimal
you developed for Exercise
until you can prove that it is optimal).
algorithm
algorithm
n real
on
a number
required to
that
to determine
algorithm
in time O
should
the set
whether
there
run in time O (n
S is given
in a
are two
elements of S
sorted order.
Design
an algorithm
to determine whether
two sets are disjoint. State
of the sizes m and n of the given sets. Make
is substantially smaller than n.
algorithm
case where
in
sum
is
to solve
this
(n).
two sets S, and S2, and a real number jc, find whether
there exists
and an element
from S2 whose sum
is exactly
x. The algorithm
O (n log n\\ where n is the total number elements in both sets.
an
whose
log n).
Given
your algorithm
6.25
and
set S containing
an
Design
Design
(Fig. 6.28),
Majority
Algorithm
b. Supposehow
6.24
in
in the
exactly x. The
6.23
loop
this problem.
The input
a.
1. Let
decision trees
case (or improve
is a
input
that is
solve
aabccbbaabca to
case.
using
by
the worst
The
aabbaabababbaabbaabb.
of
array
|A [i ] -A [i + 1 ] | <
algorithm to find j
number of comparisons
6.21
versus a
Edit
Minimum
otherwise,
sequences and
real numbers. Algorithms
specified
in
characters.
all
trees
Huffman
Exercises
the running
6.20
the string
M obtained
and
prove
elements
6.19
Write
Creative
All
the next
177
Exercises
Creative
terms
element
an
should
the
sure
run
from S,
in time
complexity
of
to consider
the
178
6.26
Involving Sequences
Algorithms
The
to
compute
the
partition
of its
sum
the
find the
should
of
is a sequence
input
algorithm
real
and Sets
numbers
partition
the
minimizes
the following
in
pairs
numbers. Denote by
that
is even.
such
that n
...\302\273\342\226\240*\342\200\236\302\273
Jt,,*2,
into nil
input
s2,
5,,
n^
tnese
\342\200\224^n/2
an
Design
each
For
way.
sums. The
we
pair,
algorithm
sum.
maximum
*6.27
Modify lexicographic sort to work for variable-length strings. In other words, you can no
k digits.
Some numbers may be long and some
that all numbers have exactly
assume
longer
of
all
numbers
is
course
to
It
short.
by adding
\"dummy\" (0) digits to make
\"pad\"
possible
Find
an algorithm that avoids
all of the same length.
them
doing that and achieves a
in
of
linear
the
total
number
time
digits.
running
6.28
The input
is a sequencex,, jc2,
of distinct
tf\342\200\236
a,,02,...,
1, 2,..., n).
6.29
of n
total
sorted
6.30
is d sequencesof elements
elements. Design an
The input
Johnson
sequences
is
into one
1. Arbitrarily form
This
nil
description of a sorting
incomplete
distinct
known
algorithm
as the
Ford
of elements
pairs
in each pair
elements
sort
in some
Insert
and
sorting.
2. Comparethe
3. Recursively
4.
logd)
and there
sorted,
sequence.
sequence is already
algorithm to merge all the
that each
such
0(n
into the
sorted list
of larger
elements
almost
that the
any other algorithm, provided
order. Consider the cases of n = 5, 6, and 8. Find
a good order in which
to insert in step 4. You should end up with an optimal sorting
values of n (in fact, you will
(in terms of the number of comparisons) for these
algorithm
get an optimal algorithm for any n < 12 with this algorithm).
6-31
The
uses
algorithm
insertions
is a
input
distinct
integers
a. Design
comparisons
b.
6.32
is the
Why
n -1.
n) is at
than
a \"good\"
sorting
algorithm
in the worst case.
to
such
sort
(A
representation using
most
comparisons
done in
the sum
that
Prove
fewer
step 4 are
in
of
balanced
the
heights
binary
an array
of all
n.)
satisfied
nodes
tree with
of size
sequences
Show
in
in this
a balanced
nodes
a tree
is one
using
such
that
at
most
the number
O (n log
of
log n)
case?
binary tree
that
corresponds
with
n nodes
is
at
to an implicit
is exactly n - 1.
Creative Exercises
6.34
6.35
179
input is a heap of size n (in which the largest element is on top), given as an array, and a
real number x. Design
an algorithm
to determine whether
the fan largest element in the
is
less
than
or
to
x.
The
worst-case
time
of your algorithm should be
heap
equal
running
of the size of the heap.
You can use 0(k) space.(Notice
that you do
0(k\\ independent
not have to find the fan largest element; you need only determine
its relationship to x.)
The
The
weighted
numbers
jc,,
it. Let W be
number Xj such
that
<
2>(jcf-)
..., xn
jc2,
X,
Xj > Xt
and
>
5>(jc,)
w(xj)+
X.
Xj > Xt
6.36
Let
are 1, this
A be an
Prove that
fan
algorithm
A collects
that
finds
enough
the
fan
information
largest
that
when
of n elements
of comparisons.
by a sequence
which elements are greater
than the
can partition
it. (In other words, you
the set
more comparisons.)
to determine
the fan
Considerthe
becomes
problem
and which
largest
around
6.37
to solve the
an efficient algorithm
Design
all weights
making
fan
element, and suppose that we are interested
largest
is a sequence of
fills
Each
element
one
memory cell. The input
only
space.
elements, given one at a time, inserted into a fixed cell C. That is, in the ith input step x-, is
put into C (and C's previous content is erased). You can perform any computation between
of C to a temporary
the content
two input steps (including, of course, moving
location).
Give an
The purpose is to minimize
the extra number of cells required
by the algorithm.
find
of
cells
needed
to
the fan
and
a
lower
on
the
number
bound
bound
memory
upper
element.
largest
of finding the
problem
in minimizing
problem is to find the fan smallest element, as in Exercise 6.37, but this time
not
as well as to use very little space (although
the running
time
minimal space). The input is again a sequence of elements xx, x2,..., x,n given
necessarily
element
an O (n) expected time
to compute the fan smallest
one at a time. Design
algorithm
ahead
of
time
that
sufficient
cells.
The
value
of
k
is
known
0(k)
(so
memory
using only
until the last element
can be allocated), but the value
of n is not known
amount of memory
we
want
of this
to
minimize
is seen.
6.39
Let
A and
B be two
P and Q can
6.40
Given
a set of
sets,
both with
elements,
such that A
communicate
integers S =
such
that
180
Jt,
\302\243
x,&R
6.41
Involving Sequences
Algorithms
= 0
and Sets
(modulo n).
or determining
jt,,Jt2, ...,-*\342\200\236,
6.2 and 6.4.6.)
to prove a
bound
information-theoretic
problem
of
such
lower
of Q(logrt)
bound
that x, = i
in
the
sequence
Sections
in
6.42 Supposethat
language
6.43
you
that
want to
lets you
tree has
that a Huffman's encoding has been applied to a certain text. The Huffman
been constructed and it is available to you. The frequencies of all characters in the text are
now that the text has been changed
also known.
Assume
such that the frequency of
slightly
X has been increased by 1. You want to update
one (existing)
character
the tree so that
it
remains optimal for the modified
text. A friend
makes
the following suggestion for an
the tree.
algorithm to modify
Assume
First, he notes
associatedwith
that
the
an
important
nodes
6.44
The
6.45
The KMP
\342\200\242
\342\200\242
\342\200\242
\342\200\242
\342\200\242
\342\200\242
string
matching
algorithm
can be
improved
for
binary
strings
in
the
following
Creative
181
Exercises
way:
so far,
we
a. Give a precisedefinition
b.
6.46
the string
Modify
on-line
KMP
the
Modify
Let T and
Design an
all
y,
Design
of this
advantage
change.
string
words,
you
match (but
the largest
to find the
algorithm
matching
it
do
has
to
not need
to start
with
largest
all
match
of B
inside A;
for Exercise
algorithm
6.48
such
that,
if there
are
subsequences
many
have
and
to
f\\
-pr
\302\243c(/y)
y=i
that
.A
1 \302\243./\302\243*,
we
is maximized.
\302\243/,
Find the
sequence of
in T
is maximum.
then
Consider Exercise6.48;assume
and
you
it.
instead,
b x).
O(n)
an
tj
that matches a
prefix
of B
with
indices
</,
that
subsequence
matching
< i2 <
the ith
character of T has
that
a positive
sum of costs.
f\302\260r
all
is maximized.
The largest
L such that
a. Design
b.
Let
efficient
d(T,
allowed
d(T,
algorithms
to
find
the LCS
and SCS of
two
given
P)be
P)=
sequences.
that
no replacements
Prove
LCS(7\\ P) \\)
delete).
(in
size of the
6.52
the
P be
<j
are equal
6.51
In other
of A.
subsequence
6.50
for
string
want to find
6.49
values
its new
show
and
a relatively
at
substring
6.48
table,
An
time
6.47
of
in Fig. 6.21.
example
\\
(|
are
that
is
the
182
edit distance
the
only
insert at the
of B to
the
of A
subsequence
without cost,
of B
beginning
and Sets
Sequences
Involving
Algorithms
you
to
it fits.
which
(Notice
the insertions at
count
must
that,
the end of
if
you
A, and
vice versa.)
6.53
The
For example, suppose that the sequences are aabb, bbb, and ebb. One possible
a b in bbb and
is inserting a in front of bbb and ebb (which costs 1),replacing
an 0(n3)
an a, and then
the rest matches; the total cost is 2. Design
a e in ebb with
three given sequences.
algorithm to find the minimal edit distance between
necessary).
edit
6.54
sequence
Let A
= a,
'''
a2
string 0,-0/+i
between B and
all i,
6.55
A [i].
The
input
that
selects
is a
Design
sequence of
6.58
an
be two
bm
of
suffix
Denote
strings of characters.
by A [i ] the
Let d( be the minimal
edit distance
to find the minimum value of d, (among
A).
algorithm
0(n2)
numbers
\\j/2 n\\
is in
terms
deterministic algorithm
to
greater than or equal
that any
Prove
...,*\342\200\236.
jc,,jc2,
set which
(i.e.,
number
of steps required
of both k and r.
by
the
coloring algorithm of
probabilistic
Assume
What
is the
percent?
Discuss
not precise. What
percent
margin
of error.
absolute
bounds
is
Each
examples
of public-opinion
with an \"error\"
polls are usually
given
indicate that candidate X has x percent of the vote, and add
may
they
probability
to k, for
the
comparisons.
Section 6.9.2,in
6.57
the zth
<i<n).
'''
= b, b2
an and B
\342\200\242
\342\200\242
\342\200\242
a\342\200\236
(namely,
why
stating
would
exercise is to compare
Monte Carlo algorithms
the bounds
be the
on
precise way
the
that
percentage
to define
poll has a
of error
\302\2613
as
the error?
6.61
a nutshell,
Design
of this
an algorithm
that, given a
list
Monte
guarantee
of n
elements,
finds
all the
elements
that
appear
more
Exercises
Creative
than
nlA
times
should
algorithm
use O(n)
comparisons.
(Hint:
183
Modify
the
majority algorithm.)
6.62
6.63
players.
to design a schedule
Each player must play every
n - 1
Denote the
Design
an algorithm
You
are
number
asked
of players.
to arrange a
If
the
number
for
a round-robin
tennis
round-robin
of players
is odd,
tournament
then
in each
participate.
and
cx,c2,...,
cn be
whose
sum is
equal; namely,
either
sequences are
realizable,
and
second element of
the
second
row can
be nonzero,
two given
whether
Design an algorithm to determine
a matrix with
row and column
construct
the corresponding
1.
problem
if they
can be solved.)
CHAPTER
ALGORITHMS
GRAPH
is the
shortcut
longest distance
two points.
between
Anon
7.1
Introduction
of objects.
The
were limited to ordering, multiplicities, overlappings, and so
relationships
on.
In this chapter, we discuss more
involved
relationships
among objects. We use
a large variety of situations,
and
to model
these relationships. Graphs can model
graphs
social
We
to
have
in
fields
from
been
used
diverse
they
archaeology
psychology.
ranging
to
and
certain
to
several
basic
present
manipulate
compute
graphs
important
algorithms
In
the
graph
previous
let's see
Finding
streets
the
examples of modeling
a good
intersections
sets
involving
and sequences
by
graphs.
a restaurant
in
to the
to
an
expected
time delay,
undesirable.The
3.
route
correspond
associatedwith
2.
discussed algorithms
properties.
First
1.
chapter, we
we studied
problem
into
the
states.
is to
problem
which states
the
states
can lead
to
the
find
proceed. Some of
of finding
is a graph-theoretical problem
edge indicates a possiblemove
and
the
\"quickest\"
program
may
may be considered
an undesirable
state
and an
correspond to the vertices
from
one state to another.
The problem
of scheduling
classes in a university
can be viewed as a graphThe vertices correspond to the classes,
theoretical
and two classes are
problem.
connectedif there is a student who wishes to take them both or they are both taught
is to schedule
the classes such that the
by the same professor. The problem
in
which
the states
Graph
186
Algorithms
This
is a difficult
problem
and
good
solutions
to
it
are
to find.
4.
arrow
from
one vertex (the tail) to another
The edges
head).
(the
are unordered pairs; we draw
them
as
line
simply
segments. A
is
a
with
several
between
the
same
of
vertices (i.e.,
multigraph
graph
possibly
edges
pair
\302\243
is a multiset).
that
are
not
are
sometimes
called
Graphs
multigraphs
simplegraphs.
Unless
The
that the graphs we deal with
are simple.
otherwise, we will assume
specified
we
v is the number of edges incident
to v. In a directed graph,
degree d(v) of a vertex
also distinguish
between the indegree,
is the number of edges for which
v is the
which
which is the number
v is the tail.
of edges for which
head, and the outdegree,
A path
from V! to v* is a sequence of vertices v1( v2,..., v* that are connected
by
the edges (v1( v2), (v2, v3),..., (vk_{, vk) (these edges are also usually
to be
considered
A path is called simple if each
vertex
part of the path).
appears in it at most once.
Vertex u is said to be reachable from vertex v if there is a path (directed
or undirected,
on
the
from v to u. A circuit
is a path whose first and last vertices are
depending
graph)
the
same.
A circuit is called simple if, except
for the first and last vertices, no vertex
more than once. A simple
are sometimes
is also called a cycle. (Circuits
circuit
appears
called cycles even if they are not simple; we will assume
that cycles are always simple.)
The undirected form of a directed graph G=(V, E) is the the same graph without
directions on the edges.
A graph is called connected if (in its undirected
form) there is a
from
vertex
vertex.
to
other
its
undirected
form)
A
that
is
a
forest
path
(in
any
any
graph
does not contain a cycle. A tree is a connected
as an
A rooted tree (also known
forest.
is a directedtree with one distinguished vertex called the root, such that
arborescence)
from the root.
all the edges are pointing
away
A
of a graph G = (V, E) is a graph // = (\302\243/,
such that UQV and
subgraph
F)
F \302\243E. A spanning
tree of an undirected
is
G
a
of
G that is a tree and
graph
subgraph
an
in
edge as an
undirected
an
pointing
graph
7.2
Eulerian
187
Graphs
vertices of G. A spanning
forest
of an undirected graph
G is a
forest and that contains
all the vertices of G. A vertex-induced
H = (U, F) such that
of a graph
G = (V, E) is a subgraph
(/cV
and F consists
subgraph
of
whose
the
in
E
both
vertices
to
U.
A
all
is
vertex-induced
of
edges
belong
subgraph
=
If
an
induced
a
called
G
is
not connected, then
(V,
E)
usually
simply
subgraph.
graph
be partitioned in a unique
called the
it can
way into a set of connected
subgraphs
that
all the
contains
subgraph of G that
is a
connected components of
G.
G such
subgraph of
connected subgraph.
other
no
that
a maximal
is
component
vertices
can be
vertices
in
with
associated
divided
other
the
into
set. A
two
sets
component of
G contains
A
it.
connect verticesfrom
with
of
subgraph
words, a connected
is a graph
whose
graph
In other
bipartite
G is a connected
weights
(or
one
costs, or
set to
lengths)
the edges.
for
definitions
Many
A connected
connected
directed
example,
and undirected
directed
paths
graphs
are
similar,
and undirected
paths
except
are
for some
defined
in
Eulerian
7.2
Graphs
of Eulerian
notion
problem
of
graph
theory.
188
Graph Algorithms
Figure
bridges problem.
Figure
7.2 The
graph
The Problem
that all the
each
It is
When
edge
Since each
be even.
Given
connected
all
a closed
vertices
by
must have
path, we enter
To prove
undirected
an
of E
to the
corresponding
induction
once,
and
even degreefor
leave
the number
that the
condition
each
of edges
is
such
vertex the
a closed
same number
adjacent
sufficient,
to
each
path to exist:
of times.
vertex
we first have to
must
decide
189
to
parameter
induction.
the
apply
without
it.
If we
Induction hypothesis:A
vertices
resulting
be
must
though
Let
cannot simply
G\\,G\\,...,G\\ be the
edges
that
edges, all
includes
that
of
whose
each edge
path.
of
the
of edges
number
rather
than
edges adjacent to
removed
induction
the
apply
with < m
in terms
hypothesis
G =
graph
graph
to find
how
know
number
the
a
connected
have even
hypothesis
any
may
vertex
not
is even.
But we
Let
be connected.
the degrees of
component,
components
of edges in each
Furthermore, the number
(indeed, in
component
we
can
now
the
induction
all of them together) is < m.
Hence,
hypothesis to each
apply
a closedpath that
the
each
has
That
induction
is, by
hypothesis,
component
component.
these k closed
includes
every edge exactly once, and we know how to find it. Denote
We now need to merge all these paths to one closed path covering
paths
by P! ,/>2,...,/>athe
we meet the first
with
whole
any vertex in P and traverse P until
graph G. We start
the path P}
vertex vy belonging
this
we
traverse
At
to one of the components
point,
G'y.
the
the
the
of
to
We
can
this
continue
components
paths
returning
way, traversing
vy.
first
vertex.
At this point, all edges will
time
we meet them, until
we return
to the starting
have
been traversed
exactly once. This closedpath is called an Eulerian circuit. The
the
is not yet complete.
We still need to find an efficient method to identify
algorithm
Both of those
connected
method
to traverse the graph.
and an efficient
components,
issues are discussednext. The implementation
is left as
of the Eulerian
circuit
algorithm
all
vertices
an
exercise.
7.3
Traversals
was
This
of
dimensionality
Scanning a
traversal
we encounter
problem
input.
connected
even.
Graph
The first
the
are
graph,
algorithms
the
or
trivial
input
trying
problem
\342\200\224
traversing
\342\200\224
when
depth-first
sequences
it, as we
search
in
call
it,
is not
(DFS), and
We present two
straightforward.
breadth-first search (BFS).Most
of
Algorithms
Graph
190
the
this chapter
in
algorithms
depend,
one
in
of
on one
or another,
way
these
techniques.
7.3.1
Search
Depth-First
The
for directed
algorithms
we also want to
search
depth-first
identical. However,
different
since
directed
in
and
graphs
undirected
in
undirected
and
graphs
graphs
graphs,
the
are almost
properties
that
are
discussion
into
two
parts.
Undirected
Graphs
to an art gallery
of
undirected
consisting
graph G = (V, E) corresponds
The
are
of
G
corridors
where
the
to
of
an arrangement
correspond
paintings
hung.
edges
the
intersections
of
the
corridors.
We
the
vertices
to
and
want
the corridors,
correspond
We
assume
that we can see both
and
see all the paintings.
to walk
through the gallery
Suppose
the
that
when we walk
possible to walk throughout
of a corridor
sides
it is
then
it in
through
If the
direction.
any
graph is
corridorexactly
visiting each
the gallery
Eulerian,
We
once.
we allow
each edge to be traversed
graph is Eulerian, and
more than once (as it turns out, each edge will be traversed exactly twice). The idea
We walk through the gallery
to enter
behind
depth-first search is the following.
trying
new
corridors
whenever
we can. The first time we visit an intersection, we leave a
do not
pebble there,
from
we
which
return
through
intersection
essential
the
part
new edges
The
(going deeper
adjacency
list
the
see
now
representation.
of the DFS.
the root
vertex
rx, connected to r, is
adjacent to r are
unmarked
already
marked,
vertex r2
an
root
The
then
stops
the
the
from
this
it is not an
only to clean the gallery;
to explore new corridors;
we return from
intersection,
only if we tried all other
that
search (DFS)to indicate
The
main
reasons
for
the
gallery).
its adaptability
to recursive algorithms.
of DFS
marked.
then
first
in
was
terms
DFS is implemented
The traversal is started
called
connectedto
try
and
graph
how
always
entered
the
into
we
same corridor
corridors leading
depth-first
we gave
description
Let's
pebbles.
first
we
the
is done
pebbles
We
approach
it divides
way
we
the
a deadend). When
through
remove
we
which
from
corridor
the
this
call
We
the
another
try
it is
(unless
a pebble, we return
corridor.
If all the
has
which
from
corridors.
DFS is
and
corridor
another
already
again. (Removing
of the algorithm.)
corridor
visit
came,
that
have already
intersection
from
we continue
and
at an intersection
arrive
and
the
is marked
picked
and
for undirected
from
starting
An
r{
to
of
down
putting
vertex r,
arbitrary
from
try
usefulness
in the
given
graphs
an arbitrary
as visited.
a DFS
and
of walking
we first
which
is
(unmarked)
is performed
all the vertices
when
from
r2 is
performed,
and
so
on.
There
applications
postWORK,
7.3 Graph
Traversals
191
that
is marked,
the edge
(G, v);
Depth_First_Search
Algorithm
and
v (a
vertex of
G).
begin
mark
perform
for all
preWORK
edges
(v,
ifw is unmarked
onv ;
{preWORK
on the
depends
application of DFS}
w) do
then
Depth
First
Search(Gf w) ;
perform postWORK
for (v, w)
{ postWORK dependson the application
performed only on edges leadingto newly
of DFS;
marked
is sometimes
it
vertices.}
end
Figure 7.4
\342\226\241
Lemma
Algorithm
First_Search.
Depth
7.1
then
all its vertices
// G is connected,
Depth First Search, and all its edges will
the execution
of the algorithm.
Figure 7.4
DFS
for an
will
be
be
looked
marked
at at
by
algorithm
undirected graph.
Graph Algorithms
192
Proof:
remaining
must
of
the
is visited,
whenever a vertex
all
vertices
Since
marked) too.
edges areconsidered,all edges
If
one marked
all the
vertices
to
adjacent
vertices
unmarked
vertex from U
one
it
visited
are
a vertex
(hence
all its
is visited
are considered.
not
\342\226\
we
connected,
have to modify
is connected
graph
Thus, we
of
least
start
we
Otherwise,
at
are marked
vertices
connected,
But this
vertex.
unmarked
are visited,
G=(V, E) is
a graph
G is
Since
the set
U denote
let
and
algorithm.
to at least
connected
be
the contrary,
Suppose
at the end
DFS
and we are
another
is connected
all
If
slightly.
done.
on.
and so
DFS,
and to
its
find
is given
in Fig 7.5. We will
The corresponding algorithm
components.
we
because
otherwise
can
deal with
connected
consider
usually
only
graphs,
generally
in Fig.
we
will
use
DFS
it
is
as
described
connected
each
component
separately. Thus,
it
run
several
times
as
in
that
have
to
be
7.3, without
may
mentioning
Fig.
specifically
connected
7.5.
Algorithm
Input: G
Output:
for
( G );
ConnectedjComponents
E) (an undirected graph).
is set to the number
v.Component
=(V,
every
of the
component
containing
v,
vertex v.
begin
Component
:=
Number
while there is an
1 ;
Depth First_Search{G,
( using
the following
v do
vertex
unmarked
v);
preWORK:
v.Component .= Component
Number
Component
Number
:- Component
;)
+ 7
Number
end
Figure 7.5
It is
Complexity
since the
graph
them
must
time.
Therefore,
be
Constructing
Next,
we
present
and building a
DFS tree exhibit
may
running
contain
examined),
Connected
Algorithm
edge
is looked
proportional
to the
each
time is
Components.
twice
exactly
number
(once
of edges.
from
However,
all of
(and
many vertices that are not connected to anything
for the running
we must include 0(\\V |) in the expression
the overall
time
running
is O
(|
+ |E
special spanning
special
at
properties
tree,
called
that are
the DFS
useful
for
|).
the vertices
with
DFS
numbers,
algorithms.
Even
if
and
the
the
tree
Graph Traversals
7.3
193
it is easier to understand
many
explicitly,
algorithms
by considering it. To
these algorithms, we need only to describe
either preWORK or postWORK.
The
the
vertices
is
in
for
with
DFS
numbers
and
the
7.6,
numbering
given
Fig.
algorithm
DFS
tree
is
in
for
the
7.7.
These
two
need
not
given
building
Fig.
algorithms
algorithm
is not
built
describe
be performed separately.
(G, v);
DFS_Numbering
Algorithm
G = (V,
Input:
Output:
for
vertex
every
UseDFSwith
graph), and
v.DFS is set to the
v,
:= 1
Number
DFS
Initially
(an undirected
E)
the following
vertex of
v (a
DFS
number
G).
of v.
;
preWORK:
preWORK:
v.DFS := DFSNumber;
+ 1
DFSNumber
:=
DFSNumber
DFS
Output: T
Use DFS
(a DFS
tree of
G; T is initially
the following
with
postWORK:
ifw was unmarked
{the
Depth
algorithm
A vertex
path
from
w to
in
T.
(line 4)
of
Search }
First
DFS
Build
Algorithm
If v is an
\342\226\241
Lemma
Let G = (V,
vertex of G).
can
an ancestor
is called
v (a
empty).
add
then
Figure 7.7
unique
and
Numbering.
postWORK:
above
statement
of a vertex
w in
ancestor of
u\\
property
of
a tree
then
undirected
Tree.
T with
w is
root
r, if
is on
called a descendant of
the
v.
DFS trees)
of
E)be
a connected
constructed
by
algorithm
Proof: Let
After
is marked,
(v,
Algorithms
Graph
194
In other words,
tree.
cross
Avoiding
edges is
are
edges
for
important
vertices
connecting
recursive
procedures
Directed
The
Graphs
DFS for
for
procedure
However,directed
directed
graphs
is
identical
have different
graphs.
DFS
to children
back edges
connect
parents
forward
\"related\"
shown
in
the
next
in
the
It
tree,
is no
and
connect descendantsto ancestors,
cross edges connect verticesnot
Only
must
cross
from
\"right
\342\226\241
Lemma
property
of directed
as
left,\"
DFS trees)
to
lemma.
descendant
ofv
in
the
tree
T.
ofG.
then
If
w
is
Graph Traversals
7.3
G =(V, E) (an
Input:
We
we
the
do that
not
and
graph),
of G).
(a vertex
application.
Pascal
use the
will
connected
undirected
Output: dependson
(
(G, v) ;
Nonrecursive_Depth_First_Search
Algorithm
this chapter.
rest of
the
here;
explicitly
symbolA
pointer
in
begin
while
mark
do
v ;
perform preWORK on
Edge := v.First ;
v and Edge to the
push
Parent := v
of the
top
up to here;
stack is not empty
remove Edge from the
{ initialization
Edge
comes
now
while the
while
stack ;
of the
top
recursion
stack ;
is unmarked
mark
loop of the
do
* nil
Child := Edge*.Vertex
if Child
the main
do
then
Child;
preWORK on Child ;
to the top of the
push
Edge*.Next
{ so that we can return to the next
perform
with
Parent := Child
to
Parent
push
else {Edge
Edge
if
the
{the
is a
:=
stack
are done
the
top
of the
stack ;
back edge
Child)
postWORK
only on
Edge*.Next;
Child from
stack
when we
}
perform postWORKfor(Parent,
{this step is skippedif we perform
tree edges}
remove
edge
Child}
:= Child.First
Edge
stack
is not
the
top
empty
empty
becomes
of the
stack ;
when
Child
then
the top
them }
be at
perform postWORKfor
is the
of the
root}
stack
(Parent, Child)
end
Figure
7.8 Algorithm
Nonrecursive_Depth_First_Search.
Figure
Since the
Proof:
(v, w) is
Since
time
was
v.
DFS number
in
edge
Hence,
\302\243,
(v,
(v, w)
unmarked,
is satisfied.
condition
from
an
A DFS
7.9
would
be
w was
be a descendant
w must
graph.
w is
of
w) must
Otherwise,
of v, w
the
was visited
DFS
hence, (v,
after
v.
of v. If at that
w)e
the recursive
F,
and the
call of DFS
\342\226\2
for connected
This is
not
so
for
Figure
7.10
#4
+*
<L
<L
*<
#<
An example of a directed
DFS
that does
whole
graph.
Graph Traversals
7.3
As
an
we show
example,
DFS to
use
to
how
determine
graph is
not a
or
whether
197
acyclic.
The Problem
whether
it
a (directed)
contains
graph G
a directed
Given
=(V,
determine
\302\243),
cycle.
7.4
\342\226\241
Lemma
G
to
completes
The
a cycle.
algorithm
for determining
graph is
a directed
whether
acyclic is given
in Fig.
7.11.
Output: FindaCycle
Use DFS,
starting
from
(true
if G
contains
a cycle and
vertex,
an arbitrary
false
otherwise).
the following
with
preWORK
andpostWORK:
preWORK:
v.on_the_path
:=
{ x.onjhejpath is true
{ initially
x.onjhejpath
true
ifx
is on
= false
the
from
path
the root
to
the
current
postWORK:
then
FindjijCycle
ifw.onjhejpath
on v's list
vertex
last
is
the
ifw
Figure 7.11
:= true
then
halt
v.onjhe_path
Algorithm
Findji_Cycle.
:=false
vertex}
is
false
Search
Breadth-First
7.3.2
search
Breadth-first
by
If we
level.
level includes
second
The
is
all the
to
visit
is replaced
to DFS numbers.
\"grandchildren,\"
to
similarly
implemented
stack
the
that
traverses
(BFS)
so level
\342\200\224
it does
first.
Algorithms
Graph
198
We can
a queue.
by
nonrecursive
the
and
on (see
so
implementation
Fig.
of
with
similarly
That
7.5
Lemma
\342\226\241
Proof:
The
Left to
level
traverses the
first-in-first-out
property
of the
T is a
shortestpath
queue.
\342\226\2
7.6
Lemma
\342\226\241
Proof:
the
then
of a
graph
w,
the path
from
the
root
to w in
from
G.
the
reader.
vertex w
level
\342\226\2
is the
length of
the
path
in the
tree from
by level.
Figure 7.12
BFS
tree
for a
directed graph.
the
root
to w.
BFS
7.4
Breadth_First_Search
(G, v);
E) (an undirected connected graph),
Algorithm
Input: G = (V,
Output: depends
and
199
Sorting
Topological
vertex of G).
v (a
the application.
on
begin
mark v
put
the
while
a queue
v in
is not
queue
do
empty
remove
the
queue
perform preWORK on w ;
{ preWORK depends on the application
for all edges(u\\ x) such that x is unmarked
mark
do
x;
add
to the
x)
(u\\
x in
put
ofBFS
tree T ;
the queue
end
7.7
Lemma
\342\226\241
// (v,w)
is an
vertices
Proof: Left
7.4
edge
to the
we know
that
Now
graphs.
Breadth_First_Search.
how
use the
We again
Topological
in E
that does
not
differ
to T,
belong
by at most
then
it
two
connects
1.
reader.
to
\342\226\2
a graph, we
traverse
design-by-induction
involving
very heavily.
technique
Sorting
need
Suppose
to be
be started
performed one
at
Some tasks
a time.
tasks are
completed. All
for
the tasks
a
schedule
we want to arrange
and
dependencies
performing
which is consistent with the dependencies
(i.e., every task is scheduled to be performed
after all the tasks on which
it is dependent
are completed). We want to design a fast
only
to generate such a schedule. This
is called topological sorting. We
algorithm
problem
other
the
are known,
the
can
associate
way.
Each
until
if y
task
cannot
sometaskscan
a directed graph
is associated
start
never
until
x is
with
with
finished.
be started.
the
a vertex
Obviously,
the
dependencies
in the
directededgefrom
graph
must
task
be acyclic;
following
x to task
otherwise,
Graph
200
Algorithms
The
Problem
vertices,
label
vertices
Given a directedacyclic
n
G = (V, E) with
graph
from 1 to n such that, if v is labeled k, then all
be reached from v by a directed path are labeled with
vertices
the
can
that
>k.
labels
The straightforward
induction
is the
hypothesis
The base
remove
one vertex,
are free
to choose
will
It
apply
a vertex
is clearly
zero. This
(task)
be labeled
can
vertex
A directed
go
and
We
it
a cycle,
a vertex
will see
shortly
it
its
algorithm
with
we
once
followed
variable
and
that
easiest to label?
whose indegree is
of
a vertex
find
always
able
be
somewhere.
to start
with
edges, and
2 to n.
adjacent
decided
little
labels
1
to
how
traverse
could
we
then
0.
indegree
the
to select
in an acyclic
graph.
\342\226\2
with
vertex
0.
indegree
label the
(To be
instead
graph
however, we
(By the same
many vertices,
finitely
0.)
a vertex
to find
are
impossible
outdegree
how
indegrees,
stop. Sincethere
is
with
vertex
of 2
of
rest
Once we
of the
to
n,
but
indegree
graph
this
causes
0 for
it, we
find
the
label
is still
\342\200\224
which
precise,
completely
induction
no problems.)
the
reduction,
the
effort.
Implementation
indegree 0
The
Can we
a vertex
choose
is the
a vertex
namely,
any problems.
positive
which
with
with
acyclic, of course \342\200\224
hypothesis assumed labels of
Notice that
without
had
vertices
the
there is
1, remove
to
this fact.
establishes
never have to
through
argument,
Proof: If all
must
try
\342\200\224
dependencies
is intuitively
answer
and
hypothesis,
a graph with
n vertices,
extend
the labeling. We
consider
we
usual,
7.8
\342\226\241
Lemma
\"backward\"
no
with
graphs
above.
any vertex
our work. We
simplify
As
induction
the
to label
how
conditions
is trivial.
vertex
following.
edges
in
any
order
(using
7.5
of
vertex
in
indegree
becomes empty,
in
the
Initializing
Complexity
a vertex with
indegree
Finding
(v, w)
In
this
time of the
Indegree
algorithm
we deal
section,
with
in the
can
think
of
we
path;
it as
with
will
a directed
the
is called
that
such
directed
the
with
Input:
Output:
well.
(G)
Topological^Sorting
Algorithm
We
edges.
problem
graph
Let G
graphs.
weighted
(Length of a path
be careful to avoid
problem).
path
lightest
Paths
Shortest
nonnegative
weights associated
because traditionally the
section,
edges
is given
input.
Single-Source
the
algorithm
the queue
0 takes
The running
7.5
when
algorithm
variables
+ |\302\243|) time.
0(|V|
requires
constant time (accessinga queue). Each
edge
v is taken from the
Thus, the number of times
queue).
is exactly
of edges in the graph.
equal to the number
is therefore O (| V \\ + \\ E |), which is linear in the size
the
variables
of the
terminates
graph.
7.14.
Fig.
the
The
remaining
in which
201
Shortest Paths
Single-Source
G =(V, E) (a directed
acyclic
a
The Label field indicates
=(V, E) be a directed
the weights
call
will
the shortest
path
the
each undirected
same
length.
The length
graph
Thus,
of a
the
path
graph).
sorting of G.
begin
Initialize
GJabel
for all
v.Indegree
:= 0
vertices ; {e.g.,by
DFS
fori := Hondo
if
= 0
Vj.Indegree
then put
v,
in Queue
repeat
if w.Indegree
w)
do
:= w.Indegree - I ;
= 0 then
put w in
Queue
Figure
7.14
this
in
(rather
than
number
of
the
is undirected, we
edge correspondsto
topological
lengths
problem
denotes
sometimes
also
confusion.) If
with
graph
Algorithm Topological_Sorting.
discussion
is
the
two
in this
sum of
the
shortest
For
Given a
Problem
The
discuss
we
simplicity,
v to all
from
paths
only
find
to
how
and
of
the length
v, find
a vertex
shortest
the
a segment
through
it,
may
or to
actual
to its
correspond
of constructing
the cost
the
to
length,
on the
it, depending
time
expected
it
The
paths.
of shortest
of
the length
to travel
takes
problem.
Let's
that
assume
induction.
the graph
us to
will
solution
Consider
last
the
solution
find
the vertex z
vertex,
namely,
know the shortest
with
label
n.
Suppose
that
Induction
the
Given
lengths
an acyclic
graph
with
n vertices
Considerthe
by
that
such
such
the
z)e
E.
we want to
shortest
how to find
vertices.
take
that the
words,
the algorithm
first
then
(w\\
we know
ordering,
to the
a topological
in
induction,
v to all
from
be
two
found
passes,
hand
in
hand
one for
the
is found).
way
Assume, for
simplicity,
will
be
that
executed
the
7.5
Output:
from
w e
vertex
every
vertex),
length of the
is the
w.SP
V,
v (a
graph),
shortest
algorithm,
path
to w.
assume
We
For
v
(G, v, n)
Acyclic_Shortest_Paths
Algorithm
203
Shortest Paths
Single-Source
An
in
improved
7.16.
Fig.
begin
be
let z
ifz * v
n { in
labeled
vertex
the
the
order };
topological
then
Acyclic
_Shorte st_Paths
{ G-z
results
(u\\ z)
that
incident
its
with
edges from
G}
e E do
z.SP
z) <
lengthfw,
:= w.SP
z.SP
(G-z,v,n-\\);
removing
from
then
z) ;
length(w,
:= 0
else vSP
end
Figure7.15Algorithm
case, there may be
execute the
same
at the
algorithms
labeled
to m
need to
labels
assume
We
we
and
to
them
are already
path to
shortest
(w\\ z).
w.SP
u\\
+ length (w,
z).
sense
of labels
in advance, and
that the lengths of
edges
coming
vertices
with
the
each
such edge
the shortest
hence
length
of
as
before,
Each
0(|V|
when
vertex
its
tail
is
+ |\302\243|).
is removed
considered
checkedonce in
from the
only
once.
+ 1,
(u\\
the
Therefore,
the
the
induction
the
the length
of the
+
z
is
w.SP
to
edge
we know
over
minimum,
worry
no way
initialization
To find
of the shortest
lengths
this
path to z is the
we need not
is
paths
call it z.
labels. By
the
z),
both
run
to vertices
The topologicalorder
into z.
through
path
the shortest
Furthermore,
Each edge is
Complexity
try to
This order
be able to
will
shortest
the
smaller
shortest paths
to vertices
with lower labels, since there
from z. The improved
is given in Fig. 7.16.
algorithm
once
be
will
now
v.
from
starting
we
labeled m
the vertex
consider
For
known.
paths
we
\"backward,\"
w.SP,
Therefore,
the corresponding
and
w,
order
increasing
the
know
same time.
are known,
in
from
in some
order of labels.
in increasing
is applied
these vertices
hypothesis,
of
v and/or
recursion
shortest
guarantees
length
x from
applying
steps
the
eliminate
paths
of
induction
The
will
edges to
Instead
compared.
Paths.
Shortest
Acyclic
to reach
any
all
w>
adjusting
of
them
is accessed
worst-case
about
Algorithms
Graph
204
topological sorting
to w.
begin
for all
vertices w
Initialize
; { e.g.,by
all vertices
for
v.indegree
:= 1 to n
fori
do
:= oo ;
w.SP
DFS
do
= 0
ifvj.indegree
then
put
v, in
Queue ;
v.SP:= 0 ;
repeat
vertex
remove
Queue ;
wfrom
z) do
+ length(w,
z) < z.SP then
z.SP := w.SP+ length(w,
z);
z.indegree := z.indegree 1 ;
if w.SP
^ 0
ifz.indegree
until
then
z in
put
Queue ;
is empty
Queue
end
Figure 7.16
the
When
algorithms
use the
Improved_Acyclic_Shortest_Paths.
Case
General
The
Algorithm
is not
graph
we presented
If z
is a
is a
vertex
with
following
k, then
label
< k,
labels
with
of the
result
thing
directly.
case.
a topological
It may
feature of topologicalorder:
(I) there are no paths
are no
from
paths
vertices
from
z to
vertices
with
labels
> k
to
z.
This feature
enables
vertices of a general
The ideais to
of their shortest
find
them
paths
during the
(v, x) be
of v. Let
positive, the
us to
after z
that are
vertices
shortest
in
graph
consider
from
the
find
the
shortest
topological
path
consider the
order
on the
that
execution of the
the
edge
from
of
v
algorithm.
minimum
to x is the
First,
we check
length among
edge (v,
x).
them.
All other
all
the
Since
coming
edges
all
paths from
lengths
v
are
out
are
at least
Shortest Paths
7.5 Single-Source
as long.
Let's
vertex? We
the shortest
we know
So,
induction.
choose
consider any
is the
other
this
to jc, and
path
can serve
edges
from
or
paths
the
(x is
as
the
shortest
the
first
consisting
The
closest).
of
two
only
\342\200\224
the
edges
other
second
paths,
induction
hypothesis.
general
and
a vertex v, we
Induction hypothesis: Given a graph
vertices that are closestto v and the lengths of the shortest
paths
Notice that
205
is on
induction
the
not
to
above), so the
is
problem solved.
vertex (x
on
and
base
the
know
to them.
number
the
find
a vertex
min (u.SP
+ length
minimal
over all
is the
closest vertex to v;
The
algorithm
thus,
to
and
the k
the vertices
<n i \\
(w, w))
not
adding
in Vk. By
it extends
is complete
now,
but
the
the
arguments
above,
induction
hypothesis.
its efficiency
w is
indeed
can be improved.
the
(k+
The main
l)th
step
the
the next closest vertex. This is done by computing
algorithm involves finding
minimal path length
all
the
it
is
not
to
check
to
However,
(7.1).
necessary
according
values
u.SP + length (w, w) in every step. Most of these
a
values
are not changed when
new vertex is added; only
those
that correspond
that go through
the new vertex
to paths
We can maintain
the lengths
of the known
shortest
may change.
paths to all vertices in
and
is
when
extended.
The only way to find better shortest
Vky
Vk
update them only
when
w is added to Vk is to go through w. Therefore,
we need to check all edges
paths
in Vk. For each such edge (u\\ z), we check the length
out of w to vertices not
of
coming
w.SP + length (u\\ z), and update z.SP if necessary. Thus, each iteration
involves
finding
a vertex with minimum
the SP values of some of the
SP value, and updating
remaining
as
vertices.
This algorithm is known
Dijkstra's
algorithm.
of the
Graph
206
Algorithms
Implementation
and to update
path
minimum elements
say, z is updated,
To do
accordingly.
heap
a heap is
the
in
the
structure;
heap can be
heap
need
we
that,
a search
not
Locating z
z's place in
it
does
done
with
data
another
connected to
structure
the
heap.
in an array
ahead of time, we can put them
the identities of all verticesare known
to their location in the heap.
with
Finding a vertex in the heap thus requires only
pointers
of the heap are the vertices
of the graph, the
Since the elements
the array.
accessing
Since
of
O (|
is only
requirement
space
an element
the
heap
which
|),
smaller
becomes
is reasonable.
than its
This
its appropriate position is found.
The
shortest
maintenance
procedures (e.g., insert).
up until
parent,
Path
the same
is exactly
paths
as the
in
If
moved
and
regular
is given
algorithm
(G, v);
Single_Source_Shortest_Paths
and
E) (a weighted directed graph),
Output: for each vertex u>, w.SP is the length of the
are assumed to be nonnegative.
( all lengths
)
decrease.
only
lengths
be exchanged
it can
Fig.
heap
7.17.
Algorithm
Input: G
source vertex).
v (the
=(V,
shortest
path
from
to w.
begin
for
all vertices
w.mark
do
:= false ;
W.SP:= oo
:= 0;
v.SP
w.SP
is minimal;
end
Figure 7.17
Updating
Complexity
the
size
of the
the
most
|E |
of
length
Single_Source_Shortest_Paths.
Algorithm
updates
a path
iterations,
takes 0(log
leading
(since each
to | V
edge can
where
comparisons,
m)
|
cause
deletions
the heap.
one update),
from
at most
m is
7.5 Single-Source
Shortest
leading to O (| E | log
O ((| E | + | V |) log | V
V |)
comparisons
since the
heap.
algorithm
in the
vertex
next
Hence, the
is slower than
the
the
in
this
latter
was
algorithm
207
Paths
is
time
running
same
taken
algorithm
from
the
ordered)
7.1
\342\226\241
Example
An example
line includes
of
is given
Single_Source_Shortest_Paths
algorithm
in
7.18.
Fig.
The first
only
edge from v. The shortest path is chosen, in this case,
paths
to
vertex
The
a.
second
line shows the update
of the paths including
now
all
leading
v or ay and the shortest path now
either
leads
to c. A new
vertex
paths of one edge from
is chosen in each
known
shortest
line, and the current
paths from v are listed to every
vertex. Thecircleddistances
are those that are known
to be the shortest.
\342\226\
of one
This type
Comments
of
is assigned a priority
(e.g.,
vertices are traversed according to
vertex
the
current
that
priority.
called
the
considered,
-\342\226\272#
Figure
oo
oo
oo
oo
oo
oo
oo
oo
oo
oo
12
oo
oo
12
oo
oo
12
11
oo
12
11
12
11
7.18
\302\251
\302\251
\302\251
An example
of the
single-source
12
\302\251
\302\251 \302\251
shortest-paths
and
all
\342\226\272#
\342\200\224
each
source),
++
search
priority
sometimes
is
algorithm
algorithm.
its
Graph
208
Algorithms
a change in some
are checked. That check may
The
priorities.
trigger
one
search
from
that
is
another.
for
what
priority
making
change
distinguishes
procedure
It is useful
for problems
search.
involving
Priority search is more expensivethan regular
weighted graphs.
one path at a
the shortest paths from v to all other vertices by finding
found
We
led from
was identified
a previously
time. Each additional
path
by one edge, which
v as its
All those edges together form a tree with
vertex.
shortest
known
path to a new
in dealing
with
root (Exercise 7.6). This tree, called the shortest path tree, is important
adjacent
edges
of path
variety
problems.
Minimum-Cost
7.6
Trees
Spanning
on
costs
positive
the
The
edges.
is to
problem
in the
used
broadcast),
find
fixed
all
containing
connected
the
subgraph
subgraph is minimum. It is
we could
have
subgraph
any
cycle had been present, then
broken it by deleting one of its edges;
the graph would still
be connected,
but the cost
would
be smaller
since all costs are positive. This subgraph
is called the minimum-cost
and
it has many uses besides broadcasts. Our
is to find an
spanning tree (MCST),
goal
efficient
the
ties
are
a tree.
For simplicity,
MCST.1
an
that
implies
to discuss.
If
The
the MCST is
algorithm
unique
the
remains
we assume
(Exercise
the
that
7.11),
same without
costs
are
which makes
this
assumption,
that, when
broken
Problem
The
G =
(Notice
find
assumption
easier
problem
except
to
algorithm
distinct. This
in the
edges
must be
this
that
(V,
find
\302\243),
that we
now
an
Given
a spanning
call
tree T of G of
the weights
connected
undirected
minimum
costs.) The
straightforward
weighted
graph
cost.
induction
hypothesis
is
the following.
1
is known to us. The complete
We assume here that the whole graph
network
topology of a communication
and all current costsare usually unknown only at the local sites; therefore, a distributed algorithm
is needed.
Induction
graphs
< m
< m
with
If it
is
the MCST
Given
We claim
edges?
not
how
to find
this
problem
the
that
then
included,
edge from
other
any
removing
Minimum-Cost
Spanning Trees
the MCST
209
for connected
edges.
know
1: We
hypothesis
with
7.6
adding
it
creates
cycle
m edges,
with
minimum-cost
to
the
reduce it
how do we
to
a tree again,
but
with
smaller
cost,
already selected in
of edges
number
Induction hypothesis 2:
how
to find
tree that
is a
of
subgraph
Given
T ofG
subgraph
the
fixed graph.
a given
connected
with k
MCST
graph
G=(V, E), we
edges (k<\\V\\-\\),
such
that
know
T is
ofG.
is choosing
the first
discussed the base case for this hypothesis, which
We assume that we have already found the tree T satisfying the induction
and we need to extend T by one more edge. How can we find another
edge
hypothesis,
that is guaranteed to be in the MCST?
that was used to find
We apply the same argument
the first edge.
there must be at least
T is already known
MCST.
to be part of the
Hence,
one edge in the MCST
will
T
in
T.
We
to
vertices
not
try to find one such
connecting
in
T.
We claim that the
edge. Let Ek be the set of all edges connectingT to vertices not
this edge by (w, w) (see
Denote
cost in Ek belongs to the MCST.
edge with minimum
a unique path from u to w
is a spanning tree, it contains
Fig. 7.19). Since the MCST
If (w, w) does not
between
(there exists a unique
path
every two vertices in a tree).
We have already
edge.
belong
to the
belong to T and
MCST,
it is
then
w does
not included
not belong to
in
that
7, there must
u to w. But,
one edge (jc,
path from
at least
since
u does
y) in this
path
a vertex not in T. The cost of this edge is higher than the cost of
cost among all such edges. But now we can use the
since (w, w) has the minimum
(w, w),
the
same
to
first selected
we
that
argument
applied
edge. If we add (w, w) to the MCST
and
remove
tree with smaller cost, which
is a
the edge (jc, y), we get another
spanning
that
connects
be
T to
contradiction.
Implementation
algorithm
presented
This
in
the
algorithm
previous
is
very
section.
similar
The
to
first
the
single-source
chosen
edge
shortest-path
is the
edge
with
Algorithms
Graph
210
\342\200\224#
(1
>u
fg f
\342\200\224i
>
7.19
Figure
the
Finding
T is
cost.
minimum
\342\200\242
\342\200\242
next
with
edge
MCST.
of the
this edge.
only
each
In
we need
iteration,
of T. In the shortest-path
edge connecting T to verticesoutside
the minimum-length
outside of T. Hence, the only
leading
path
and
the shortest-path
difference
between the MCST algorithm
algorithm is that the
minimum
is taken not on the length of a path but on the cost of an edge.
Tue rest of the
in 7, the
is
same.
We
each
w
included
the
for
vertex
not
maintain,
algorithm
virtually
\302\253>
w
in
T
if
minimum-cost
a
such
to
from
vertex
no
(or
edge leading
edge exists). In each
we
w to
choose the minimum-cost
vertex
iteration,
edge and connect the corresponding
T. We then check all the edges incident
to w. If the cost of any
such
edge (u\\ z) (for z
in T) is smaller than
not
z's cost.
the cost of the current
best edge leading to z, we update
The algorithm is presented in Fig. 7.20.
to
find
minimum-cost
the
we
algorithm
found
Complexity
shortest-path algorithm
+ |\302\243|)log|V|).
0((|V|
Example
7.2
An example
of algorithm
of
is the
\342\226\241
the
table
connectedto
v are
is chosen
each
vertices
in
are updated
Comments
one, of a method
elements, each
elements
with
presented
MCST
one
that
is
step (only
maximum
an
to
the single-source
time is
running
of
that
is illustrated
added
is identical
algorithm
in the
in
Fig.
7.21.
The vertex
tails
of the
their
vertex
with
costs)
the
in
the
first column
is v, and
the edges
minimum-cost
leading
to
edge
unmarked
\342\226\2
an MCST
not a pure
is an example, although
method. Suppose that we are dealing with a set of
associated
the set of
cost, and that we are interested in finding
cost satisfying
In the MCST
some
constraints.
(or minimum)
finding
7.6
Algorithm MCST
Input: G
Minimum-Cost
Spanning
Trees
211
(G);
undirected
(a weighted
Output: T (a minimum-cost
graph).
of G).
tree
spanning
begin
w do
w.Mark
let (jc,
; { w.Mark
:=false
cost edge
be a minimum
y)
:=
ifw is
in
T}
z.Edge
loop }
T to
; {
:=
z) do
{ a minimum
z);
(xy
let w be
ifw. Cost = oo
the
z}
cost
is minimal
then
\"G is
marked
be
will
z.Cost := cost(x,z);
there
in
in the main
true
is true
.= oo ;
W.CoSt
x.Mark
set;
the empty
T is
Initially
not connected\" ;
halt
else
:= true
w.Mark
add w.Edgeto
{ we
now
for all
if
T ;
the costs
update
(w,
edges
not
z.Mark
z) do
then
z.Cost := cost(w,
z)
end
Figure
were the
7.20
Algorithm
MCST.
the elements
to
tree.
edges
/?
C t
,
i
d]
\342\200\242
Figure 7.21 An
consider
We
now
in
a graph.
The
/?
/i
oo
oo
a{2)
v(9)
oo
oo
oo
oo
v(6)
c(10)
oo
oo
v(6)
c(4)
-
oo
the problem
with
of the
d{l)
c(10)
rf(12)
oo
6(3)
c(10)
</(12)
oo
c(10)
</(12)
*(5)
c(10)
-
MID
MID
Paths
of
shortest
computing
Given a
Problem
undirected)
/
oo
example
All Shortest
7.7
e
oo
v(6)
v(9)
v(6)
v(D
11
oo
\342\200\242
\342\200\242
13
72
70
graph
weighted
weights,
nonnegative
find
paths
between
all pairs of
vertices
(directedor
G =
(V, E)
the
minimum-length
paths
This
the
as lengths.
weights
how
For
we
discuss
shortest-paths
all-pairs
problem.
simplicity,
to find only the lengths
We
of the shortest
than
the
themselves.
rather
paths,
paths
assume that the graph is directed; the same arguments hold for undirected graphs. We
assume
this section that all weights are nonnegative; Exercise 7.73deals with
throughout
Again,
since
we are
negative
lengths.
talking
about
shortest
paths,
we refer to
As
or on the
edge, say
(w,
with
is involved
there
We can
induction.
straightforward
What
vertices.
to a
w),
Furthermore,
start
let's
usual,
the edges
edge
in
of shortest
terms
paths
worst
on
either
in
a new
adding
u and
between
path
In the
that
use induction
paths
a shorter
may form
213
Paths
All Shortest
7.7
w.
we need
case,
the path).
on
Induction hypothesis:
all pairs of vertices
such
The base of
the
the
is obvious.
it
solution
to
We
m + 1.
that
paths
between
all
Denote by
1, in
vertex
to consider
consider are
induction
all
u and
&-paths
only direct
hypothesis
such that
have
whether
they
shortest
w-path
is the
shortest
the
some k
We
m-paths.
and to check
labeled m. Any
between
w-path
case
which
We assumethe
have
of
lengths
only k-paths,for
of vertices,
pairs
vm the
The shortest
and
now
we need to
that
is m =
induction
the
know
We
shortest
between
paths
are considered.
< m,
k <m
to find
improve
and
+ 1.
the
on the
must include
\302\243-path (for
some
So, the
shortest
&-paths
vm
k <
k <m\\
and
hence,
between u
only
extend
new
m-paths
for k < m.
exactly
once.
m) betweenu
be equal to k)
vm appended
betweenvm
try to
we
to w.
w.
Not only is
this
algorithm
faster
(by a constant
factor) than
the
one
using
Graph
214
Algorithms
Weight (an n
[a\\ y ] is the
Weight
Weight
also simple to
(Weight);
All_Pairs_Shortest_Paths
Algorithm
Input:
it is
7.22.
in Fig.
on vertices, but
induction
straightforward
the
is given
jc ]
[jc,
Output:
adjacency
of the
weight
for all x )
end, the matrix
is 0,
the
At
shortest
x n
of
the lengths
contains
Weight
the
paths.
begin
for m:=
{the induction
to ndo
forx := I to
:= I to n
fory
sequence }
n do
do
y] < Weight[x,
m] + Weight[m,
ifWeight[x,
y] :=
Weight[x,
m]
Weight[x,
then
y]
+ Weight[m,
y]
end
that
of
independent
two
loops
can be
check
of the
algorithm are
applied to
the
the
others.
Such
For
each m,
the
Paths.
All_Pairs_Shortest
pairs
used to
check
of vertices
is
flexibility
in
any
important,
all pairs
Notice
of vertices.
check is
order,
for example, for parallel
since each
algorithms.
Complexity
algorithm
involves
and
one
comparison
On
implement.
O (| E | | V
log
the other
| V
hand, if
the
graph
using
is relatively
the
sparse,
single-source
then
the
algorithm
running
|V
\\
time of
times,
is
better.
7.8
Given
graph
w
in
problem
Transitive
Closure
the transitive
closure C=(V, F) of G is a directed
graph G=(V, \302\243),
that there is an edge (v, w) in C if and only if there is a directedpath from v to
G. The transitive closure is related, for example,
to the user-accounts security
of this
mentioned
at the beginning
chapter. The vertices correspondto the users,
a directed
such
and the
edges correspondto
permissions.
The
transitive
closure
identifies
for
each
user
7.8 Transitive
all
the
other
There are
other
many
(either directly
of
applications
or
transitive
the
indirectly)
and so
closure,
215
Closure
account.
is
it efficiently
finding
important.
The Problem
a directed
Given
= (V,
graph G
find
\302\243),
its transitive
closure.
of the
instance
We solve this problem
is, we transform
by using a reduction. That
any
of another problem that
we already
know how
transitive closure problem to an instance
of the
to a solution
the solution of the other
to solve. We then transform
problem
is from the all-pairs shortest-paths problem.
transitive closure problem. The reduction
Let G' = (V, E') be a complete directed graph (i.e., all vertices are connected in
0 if e e \302\243,
1 otherwise.
Each edge e in E' is assigned the length
and
We
both
directions).
for
If
there
a
from
v
w
in
G'.
is
to
now solve the all-pairs
G,
shortest-paths
problem
path
in G' is 0, since all edges
of G have length
0 in G'. Therefore, there is a
its length
then
path
is
easy
algorithm,
and
v and w if
between
is 0. Thus, an answer
into an answer for the
The idea of using
10. We used reduction
to the
closure
mainly
problem.
two problems is
to illustrate
the
between
and
can be transformed
explored in
with
technique
detail
a simple
directly to a
algorithm
shortest-paths
all-pairs
in Fig
path
between
reductions
to modify the
as is shown
shortest
the
transitive
here
length of
if the
only
w in
G'
directly
in Chapter
example.
transitive
It
closure
7.23.
:= 1 to
fory
if
the
induction
sequence
ndo
ndo
[jc, m ] and A
{this step
[my
is improved
A [jc, v ] ;= true
the next algorithm }
v ] then
in
end
The
first
problem
fact that we
is general
TransitiveJClosure.
solution
to another
of the
of the
general
216
solutions
expensive.
is easier
problem
to
Algorithms
Graph
When
it.
for
pay
cases,
many
used
is
reduction
where a more
cases
seen
have
We
general
solution
by using the special characteristicsof the problem.
It consists of two checks,
the {/statement.
Consider the main step of the algorithm:
if
both of these checks are
An action is taken
for A[my y].
for A[x, m] and
only
for each pair of vertices. Any
This
satisfied.
if statement is performed n times
would
to
a
substantial
statement
lead
this
of
improvement of the algorithm.
improvement
all the time? The first check
on
Do we really need to perform the two checks
depends
on
m
and
we
check
the
second
whereas
can
and
x
Therefore,
m,
y.
depends only
only
x and a certain m. If the
first check
the first check only once for a certain
fails,
perform
for
of
check
value
If
the first check
the second
then there is no need to perform
y.
any
This change is incorporated in the
it again.
then there is no need to perform
succeeds,
but this
unchanged,
The
7.24.
in
Fig.
run
about
Input:
{ A [jc, y ]
[jc, x ]
n xn
(an
the
end,
the
complexity
remains
(A) ;
matrix
(jc, y)
edge
Output:
adjacency
is true if
asymptotic
twice as fast.
Improved_Transitive_Closure
Algorithm
presented
algorithm will
algorithm
(improved)
a directed
representing
belongs to
the
graph,
the
transitive
graph).
otherwise;
and false
matrix
A represents
of G.
closure
begin
{
for m:= 1 to ndo
forx := Hondo
if A
[jc, m ]
the
induction
sequence
then
fory :- I to
ndo
if A
y ] then
[my
A [jc, y
true
:=
end
Figure 7.24
The
Implementation
however,
Algorithm
implementation
the
same
mth
time, a row
in
practice,
Since
or
(the number of
of a
bits
parallel algorithm.
that
of
algorithm is straightforward.
as an or operation on the
the
effect
is set
the
Closure.
jcth
to
the
row
value
of itself
the or
operation on
to be
Notice,
or
of the
can perform
an or
many
performed faster than several bit-by-bit
of steps for this algorithm
is 0(az3/u>), where w
can be or'd together in one step).
This is a very
issue is also discussed in Section
This
9.5.3.
many computers
can be
operation
the number
Transitive
Improved
jcth
row
that
of
jcth
row
of the
(m, y)and the
same
So,
operations.
size
is the word
bits
simple
at the
example
7.9
have
components.
In
when we need to
design an
that
algorithm
and
that
require
its
that the
the
be
to
these
graph
into
Then,
be possible to
For example, we
property.
connected.
apply
the
property.
it may
graph,
desirable
graph
partition
By partitioning
the
algorithms to each
to avoid
thus
\342\200\224
biconnected
decompositions
to use
into connected
partition
desirable
certain
manipulates
component separately,and
other
decomposition
graph
separately
its connected
into
graph
is to
subgraphs satisfies a
of the
each
that
\342\200\224
the
decomposition
graph
idea of
the
general,
such
subgraphs
of
already
217
of Graphs
7.9 Decompositions
We
of Graphs
Decompositions
and
the second one to directed graphs. Both
applies to undirected graphs
in
In
both decompositions
on
are
designing algorithms.
particular,
depend heavily
in
the
and directed cycles respectively).
the cycles
(undirected
Therefore,
graph
there is a problem that involves
or another
whenever
(and many graph
cycles in one way
one
first
The
useful
problems involve
not
always
that the
section
cycles),
are
graphs
Biconnected
7.9.1
The notion
a good
it is
useful,
at
idea to
these
consider
They are
decompositions.
We assume
be considered.
least
throughout
this
connected.
Components
in a natural
extends the regular
connectivity
concept
way.
graph is connected if there is a path from every vertex to every other
are at least two vertex
vertex. An undirected
disjoint
paths
graph is biconnected if there
from every vertex to every other vertex. Biconnected
thus
exhibit
a
level
graphs
higher
of connectivity:
If for some reason one of the paths connecting two
vertices
can no
if a graph is not
It turns out that,
longer be used, then the two vertices are still connected.
is
We
of
which
biconnected.
into subgraphs,
each
then it can be partitioned
biconnected,
will
is called kbe mainly
In general, an undirected
interested in that partition.
graph
between
connected if there are at least k vertex disjoint
every two vertices. We
paths
of
An
undirected
first
study
biconnectivity
of ^-connectedgraphs.
property of ^-connected graphs
several properties
first
The
[1927]
important
relates
that
the number
of vertex
vertices
a theorem
vertices
graph.
E)
be an
nonadjacent verticesin
from
disjoint
are
A simple
G disconnects
paths
removed
corollary
due to Menger
of
to the number
Theorem
\342\226\241
Menger's
Let G = (V,
paths
disjoint
is
between
from
from
v is
u to v. (When
equal
vertex
to
the
maximal
is removed,
v be
number of vertex
all its incident
edges
as well.)
of Menger's
two
removal
\342\226\
theorem
is the
following,
due to
Whitney
[1932].
218
Graph
Algorithms
Theorem
\342\226\241
Whitney's
must
vertices
be
\342\226\
the
condition
Definition:
biconnected
biconnected
components.
it and
contains
component
Indeed,
each
is a
component
subgraph
is
maximal subset of
biconnected
(namely,
the
edges
there is no subset
induces a biconnectedgraph).
is defined as
articulation
Figure 7.25
The structure
a set of
edges.
of a
can belong to
A vertex
nonbiconnected
than
one
graph.
component.
several
(In
7.9 Decompositionsof
219
Graphs
characterization
of articulation
description providesanother
points.) The set of
be
into
can
in a unique
biconnected
partitioned
graph
edges
components
way.
one component.
The following
two
claims
Each edge belongs to exactly
prove the
and its uniqueness.
existence of the partition
fact, this
of every
7.9
Lemma
\342\226\241
First, we show
Proof:
extended subgraph
vertex. This contradicts
theorem, if the
is
is always
a cycle
that
these
of
components
biconnected
since
still
the
biconnected
one
than
by adding
a cycle cannot
be
component. For
of the
maximality
of
rest
the
biconnected
in one
contained
entirely
contains
if
component,
the cycle. The
by one
of the
disconnected
side
other
the
edges
the
7.10
Lemma
\342\226\241
Proof: Each
component,
A
edge definitely
combination
want
We
with the
It
to
find
belong
Induction
hypothesis:
connected
graphs
and
edges
one
than
from both
edges of
containing
two
biconnected
components.
components.
\342\226\2
components. Let's
start
as
usual
hypothesis.
We know
with < m
with one
it
into biconnected
partition
induction
straightforward
to more
is impossible.
this
the
component.
at least
to
cannot
be cycles
since there would
containing
of the two cycles is one larger
cycle
We have
belongs
itself).
only
containing
(possibly
one biconnected
exactly
how
to find
edges.
edge is biconnected.Considera
m edges and
the
induction,
graph
arbitrary
by
to
have
now
We
determine
what
effect
x
would
have
biconnected
adding
components.
x connects
on the partition.
two vertices from the same
The easiest case is when
In this case, adding
x has no effect
(for example the edge (a, n) in Fig. 7.25).
component
A
pick
connected
an
graph
edge x.
We
remove
x from
the
and
graph
find,
with
220
Graph Algorithms
the
partition
them efficiently.
7.25
and
(b)
(a)
Figure 7.26 An
components
edge
corresponding
The biconnected
that connects
to the
component
two
different
biconnected
the
components,
articulation
points
(a) The
indicated, (b)
7.9
tree
the
is called
graph
x in
edge
an
edge
tree.
the biconnected
in Fig.
7.25,
Fig. 7.26
and
Fig.
illustrates
a and k
connecting
in
shows
7.26(a)
Figure
shows the
7.26(b)
the
addition
the
original
Decompositions
of Graphs
the biconnected
components of
corresponding biconnectedtree.
of an edge;
it
can
correspond,
221
The
for example, to
graph.
from
think
of
the
different
two
biconnected
components
we can
determine
this
fact.
DFS tree we
traverse
want to
know
how
high
using DFS. At
and so on. Let Tx
the graph
in the
each
vertex
\\\\
we
can
first visit
Graph Algorithms
222
DFS tree
are
High(Wj)
edges from
is simply
will describe
(We
is an
than
higher
v.
Indeed,
subtree rooted
(The
beauty
point
u>, to
at
of
the
7.27,
Fig.
v
if
if
how
DFS,
perform
hence
and
High
vertices
higher
the tree;
v in
than
the
We claim
values.
High
= w$;
edges from
hence,
in exactly
graph
is an
that a
the
vertex v
that High(Wj) is
v such
of
u>,
w, exists,
vertex
the
(v) = r.
such
whether one
have
is that it traverses
DFS
= rJ
High(wl)
to q,
goes
we
(v) if we know
all the back
among
High
compute
highest
shortly
So, if we
than another.)
is higher
For
values.
example, in
highest
We can easily
w {, u>2, \342\200\242\342\200\242\342\200\242,
wk (see Fig. 7.27).
the
articulation
order
right
not
in the
vertices
point.
for
our
purposes.)
Computing
the
induction
following
High
goes hand
values
this
of the induction
follows
a
perform (recursively) DFSfor all
whether
compute
High(y)
a vertex
the
DFS,
by
we know
been visited
how
and are
vertex.
The order
and
according to
the DFS,
with
hand
in
hypothesis.
is an articulation
Figure
children
to
according
of DFS.
the order
the
of v,
find
(by
At
definition.
we
When
induction)
the same
point.
7.27
Computing
the
High
values.
a vertex v, we
their High values,
reach
time, we can
decide
7.9
223
of Graphs
Decompositions
root.
the
by brute
them
find
of the
root
The
exceed
Noticethat,
at the
force,
to be
the edges
constant
|). The
be remembered
| + |E
that b is
traversed.
the
During
an
7.25.
back at Fig.
look
Let's
way.
articulation
we put
traversal,
the
point,
new
the
as
are
they
encountered.
per
space
as they
amount
vertex.
of
the
Hence,
also
are
requirements
in addition to
work,
running
O (|
V | + |E
work
the
this
of
time
|) since
the
involved
in
algorithm
is
components
traversed.
are
7.3
\342\226\241
Example
An example
of
in
(decreasing)
when
is discovered
a new
to be
for
Biconnected^Components
algorithm
the extra
Clearly,
the DFS, is
it
recent
elegant
determined
algorithm
fully.)
Complexity
must
the
to
When a vertex is found
all the top edges going back in the
we can remove from the stack
articulation point,
We can
until that vertex is reached. This is exactly
the biconnected
component!
in the
and continue
same way. The complete
remove
those edges from the graph
is given in Fig. 7.28.
for
biconnected
(The algorithm can be
components
in
for completeness, we
terms
of
and
of
DFS,
but,
preWORK
postWORK
merely
presentit
O (|
is also an
there
but
point where
an
call to
the
articulation
recursive
point.
line
gives
the graph in
the vertices
Fig.
and
procedure
is made.
vertex
7.25,
High
is
which
second
the
line
numbers
is circled
when
\342\226\2
Graph
Algorithms
(G,
Algorithm BiconnectedjComponents
connected
G = (V, E) (an undirected
Input:
the
root
Output:
n)
v,
computed.
begin
for
vertex vofGdo
every
v.DFS Number
{the DFS numbers
serve to
also
indicate whether
corresponding
DFSN
0 ;
:=
will
not the
or
:= n ;
DFSnumbers;
{ we use decreasing
the explanation
see
in
the
text.}
BC(v)
end
procedure BC(v) ;
begin
:= DFSN
v.DFSNumber
DFSN := DFSN -1 ;
v into Stack ; { Stackis initially
insert
v.High := v.DFSNumber ; { initial
for all edges (v, w) do
insert (v, w)
into
Stack
twice (for
{ each edge will be inserted
is
not
the
then
ifw
parent
ofv
if w.DFS Number
BC( w) ;
if w.High
{v
= 0
disconnects
wfrom
and mark
reached,
{
v.High
else
ofw's
:= max
{ (v, w)
v.High :=
of the
the
is a
until
Stack
back edge
, w.High
Algorithm
is
or a forward
, w.DFSNumber
edge
end
Figure 7.28
they form
subgraph
( v.High
max ( v.High
graph
from
biconnectedcomponent
back into Stack ;
is part
directions)}
then
the rest
remove all
insert
both
then
< v.DFSNumber
as a
empty
value
Biconnected
Components.
7.9
Decompositions
of Graphs
J
9e
16
15
14
13
16
16
16
15
16
15
16
15
16
15
16
15
16
15
16
15
16
15
16
15
00
16
15
16
15
16
15
16
15
16
15
16
15
16
15
16
15
16
15
14
14
14
14
14
14
15
15
15
15
15
15
15
15
15
15
15
15
16
15
16
15
16
15
16
15
15
15
15
15
15
15
15
15
15
15
15
7.29
An
example
00
b
a
m
16
16
16
16
15
16
15
15
16
15
16
15
00
16
15
\302\251
Figure
e
12
11
10
J
7
15
CO
wl
wk
13
13
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
14
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
14
14
14
14
15
14
15
13
15
14
14
14
14
14
14
14
14
15
14
14
14
14
14
14
14
14
14
14
14
of computing
16
15
16
15
16
15
16
15
16
15
16
15
16
15
16
15
15
15
15
16
16
16
16
8
8
8
8
8
8
8
8
8
8
8
8
15
16
16
15
16
16
15
16
16
15
16
16
16
15
16
16
16
15
16
16
16
16
15
16
16
16
16
biconnected
components.
Graph Algorithms
226
we discuss
section,
this
Definition:A
vertices such
no subset that
that,
Notice
as a set
connected
An
in a
components
in the
edge
components.
We
are similar to
the
(we
can
vertices
A circuit is
contained
and a
only
path
necessarily
a circuit.
in
Proof: If
combining
contained
into
may
the
strongly
one component.
two separate
connect
two
which
claims,
is a simple
itself
circuit.)
connected.
strongly
of the
vertices of a circuit,
vertices of the
to the
circuit
connected
of
component).
component
not be
it would
component,
connectivity,
strong
connected
strongly
since
Now,
maximal
given
we claim
that
there is a path
results
in a circuit
any
two
they
are
v to w
from
(but
not
\342\226\2
7.12
it
by the following
section.
previous
a cycle,
\342\226\241
Lemma
there
w to v.
from
in
by
a subset
w from
and
graph).
be partitioned
vertex belongs to exactly
in
of the
there is
strongly
both
is
that
(i.e.,
can
graph
partition
case
same
once. A cycle
than
more
include
subset
7.11
closed directedpath
Proof:
the
component
cannot
of
maximal
component, or
to one
belong
the existence
prove
Each
way.
unique
biconnected
Lemma
\342\226\241
vertex
of every
w to v.
from
path
connected
connected
a strongly
components,
may
graph
is a
strongly
induces a strongly
vertices
The
vertices.
and a
component
is
connected
is strongly
graph
v to w
from
from
subgraph
it and
contains
biconnected
unlike
of
induced
its
a path
vertex
connected
strongly
that
A directed
graphs.
u\\ there is
v and
it is
words,
In other
directed
only
of vertices
Components
a vertex
circuits
those
in only
v belongs
containing
circuits
one strongly
results
connected component.
one strongly
exactly
to more
and
in
connected
than
vertices
another
one
from
circuit,
component.
strongly
connected
component,
then
7.9 Decompositions
Figure7.30A
As was
directed
biconnected
with
case
the
and its
graph
connected
strongly
graph.
component
we can
components,
of Graphs 227
design an
by
algorithm
induction.
Induction
hypothesis: We
of graphs with
components
how
know
<
m edges,
to find
and
the
connected
strongly
construct
to
how
their
SCC
graphs.
The basecaseis
remove x from
now
have
case is
and find,
graph
to determine
what
effect
adding
and pick
m edges
with
x would
the
have on the
same
an
arbitrary
edge
x. We
connected components.
the strongly
induction,
by
x connects
when
a graph
Consider
trivial.
the
Again,
partition.
In this
component.
We
the easy
case,
adding
is when x connects
the SCC graph. The difficult
case
partition
in Fig. 7.31, in which
an
verticesfrom two different components. This case is illustrated
x
is
in the SCC graph
of Fig. 7.30.
edge
Clearly, x merges
connecting two components
has no
effect on
these two
if and
components
or on
the
the
components
and we
component,
plays
a major
Let's
algorithm,
try
and
to
are
only
if
it completes
corresponding
done. If x does
the component.
to
nodes
cycle
the
in
in
cycle
SCC graph.
are combined
In
the
into
by considering
algorithm
a (directed)
to the
the
in a
edges
particular order.
Again,
role.
the same steps as we
them when necessary. When
follow
modify
it
of a circuit
the DFS tree. The
is part
it in
did
we
with
notion
in
visit
of High
values can
be used in
Algorithms
Graph
228
Figure 7.31
Adding
a similar way.
of the graph
from
can
the
We
identify
an
edge
connecting
two
different
\"breakpoints\"
in a similar
way
is no
there
that
We
to
connected
strongly
need a
to
way
components.
the articulation
other parts
reach
mechanism
by
which
points. Consider
we
the
connected
of the tree
parts
strongly connected components occupy
in
That
all
the
a
connected
vertices
must
is,
(Exercise7.88).
component
strongly
belong
to one connected subtree of the DFS tree.
For a given component, consider its highest
in the tree; we call this vertex
vertex
the root of the component. The root
is the first
vertex of the component
to be visited by the DFS.
(For example, the roots in Fig. 7.30
are a, d, #, and /.) If we can identify
the
roots
similarly to the way we identified
articulation points,
to
then
we can find the partition.
We will see that the roots are similar
DFS
The
tree.
articulation
The
points.
algorithm
is based
on induction
that
follows
the
order of
root
removeit
but
from
this is
7.9 Decompositionsof
Figure
must consider
cross
the
e. If
/ and
before we
take
that
means
to go
way
that
be a
vertex
higher from e.
not a root.
/is
only to
is not
it
which
it would
then
the component
as a
the effect
considering
of /,
that
fact
the
must
It
been an ancestor
before /implies
visited
account
into
be /.
it cannot
In particular,
that there is
/.
was
that
not
had
it
reached
discovered yet
a vertex
root.
we claim
then
yet,
as well.
to
point
first
the
edges
edges.
effect of
is the
What
cross
229
Graphs
is an
have been
discovered
e has
containing
not been
So, a crossedgefrom
But
this
is just
important
this
g to
as easy to
of
ancestor
When
edge
is a
back edge. Only the DFS number of e (and its value relative to that of/) is important.
We can define the High
for an
values
as in the biconnected component case by looking
edge leading to a vertex with the lowest DFS number. The High value of a vertex is the
those of its children
is
and among its back edges or crossedges.A vertex
highest
among
the first root if it is the first vertex whose High
Notice
that
is not higher than
value
itself.
the High values do not really point to the highest
The High value of g will be
vertices.
the DFS number of e, even
it is possible to reach b from
e (and thus from g). We
though
care
vertex
whether
we
reacha
can
than
it
is not important
to know
higher
g (or /);
only
do
the identity
we
vertex.
of the highest
want
to
chase
once
a
back
(Nor
pointers
edge is
encountered.)
Once
consists
component
edges,
and
the
we find
of all the
from
the
all the
first
root,
we can
descendants of
graph.
edges
that
the
This is
point
the
find
done
by
to them
first strongly
in the
root
deleting
DFS tree.
all
We
can
ignore
edges
Graph
230
vertices,
other
from
Algorithms
be done by
induction
all the
that
to
of
get outside
a smaller
the
graph!
Notice
valid.)
component.
The
reader
should
(The
that the
of
definition
rest can
carefully
the
High
discovered
pointing to the newly
later.
of
the
values
in
the computation
(This is
High
they will play no part
component,
values for the biconnected component
the \"static\" definition of the High
from
different
there is no
case, which did not depend on any of the previous components.) In practice,
vertices or edges. We can simply mark the vertices of
remove
either
need to actually
to marked
later
on ignore edges pointing
as they are discovered, and
each
component
is
in
7.33
connected
The
vertices.
(we use
algorithm
given
Fig.
component
strongly
The algorithm
Complexity
is
complexity
to avoid
again
the
The
same.
the
edges
confusion).
is similar
time and
An example of algorithm
StronglyjConnectedComponents
The
in Fig. 7.34.
which is repeated here, is given
second
line their
In
it
this
to be
of the
call to
whether
determine
seen
that
a cycle
Given
for
procedure
connected
Use of Graph
must be
line presents
is made. A
the
High
is
vertex
\342\226\2
component.
Decomposition
where
problem
the use
involves
undirected
connected
gives
successive
recursive
a root of a strongly
it contains
line
first
Each
the
section,
The Problem
We have
a new
short
second
when
is discovered
Examples
significantly
the
as updated
circled when
7.9.3
its
7.4
\342\226\241
Example
numbers
and
a cycle of
even
contained
in
of
graph
decomposition
undirected
graph G=(V,
graphs
and
\302\243),
length.
Hence, we
component.
then consider each
components,
that the graph is biconnected!
assume
at least
it contains
than one edge, then
a biconnected
7.9
Input:
of
(G, v, n)
Strongly_Connected_Components
Algorithm
G =(V, E) (a directed
the DFS tree), and n
the
marking
Output:
the
of vertices
number
connected
strongly
v (a
graph),
(the
of Graphs
Decompositions
in
components,
root
G).
and computing
values.
High
DFS,
all vertices
begin
for
every vertex
vofGdo
v.DFS Number
:=
Current
{ we
while
:= 0
Component
DFS_N := n
0 ;
:= 0
v.Component
see
that
the explanation
v.DFS Number
in Section
7.9.1.}
= 0 do
SCC(v)
end
SCC(v);
procedure
begin
:= DFS_N
v.DFSNumber
:= DFS_N
DFSN
insert
v into
:=
v.High
1 ;
STACK ;
v.DFS
Number
; {the
initial
value
0 then
SCC(w);
:= max
v.High
( v.High
, w.High
else
> v.DFSNumber
if w.DFS Number
{ (v, w) is a cross edgeora back
, w.DFS Number
) ;
=
v
is
root
a
v.DFS
Number
then
a
v.High
{
of component}
Current
:= Current Component + 1 ;
Component
{ mark the vertices of the new component}
repeat
remove xfrom the top of STACK ;
v.High
if
edge
:=
max
( v.High
x.Component := Current
until
x =
Component;
end
Figure
7.33
Algorithm
Strongly
Connected
^Components.
<\302\273\"
>
a>
&)
61
0-
-\302\251
o a. a a. o
3
\"2.
cd*
\342\200\242-h
O
O
3
\342\200\242a
oooooooooooooooooooo
ooooooooooooooo
sO
5'
CFQ
oooooooooooooooo^
3Q
IS\"
ooooooooooooooooo
<
CD
-J
&3
-J
-J
^J
-\302\253J
-J
a.
~~-&>-^-&>*\302\273>.P>*\302\273>
OS.
3CD
o
cd
a.
\342\200\242a
CD
\342\200\224
\342\200\224
z
z
^J
-J
^J
^J
sO
sO
sO
a*
o\302\273
7.9
at,
of Graphs
Decompositions
233
7.35). Clearly, v, ^ v7. The path v,, u>,..., w, v7 defines two cycles, as is
It is easy to see that one of the three cycles in the figure must be
the following theorem.
proved
v7 (see Fig
say,
in
shown
even.
7.35.
Fig.
have
We
7.13
Theorem
\342\226\241
The
whether
it
a (directed)
contains
that a
we know
Again,
similar
one,
cycle
but for
directedgraph
cycle of odd length.
be
must
merely
an
cycle.
even-length
Given a
Problem
The
than one
more
has
\342\226\
directed graphs.
G =
(V,
in a
contained
determine
\302\243),
connected
strongly
so
component,
of these
Both
worthwhile
in
questions
to
are
problems
Since both
decomposition.
usually
graph
is strongly
connected.
assumption.
connectivity
the graphs
that the
assume
well
as
might
start
more
much
thinking
either
are
about
a given
biconnected or
to
difficult
efficiently
problem
with
strongly
connected.
the
v*
vi+\\
Figure
7.35
Finding
an
even-length
cycle.
solve
without
in linear
extra
time,
the
it
is
that
assumption
is especially
This
Graph Algorithms
234
for
true
that involve
problems
determining
efficiently
Bibliography
cycles.
It
a directed
whether
is
to note
interesting
graph contains
an
the
that
problem
cycle
even-length
of
is still
section).
Matching
= (V, \302\243),
is a set of edgesno two of which have
a matching
Given an undirected
graph G
is that an edge can be thought
of as a
The reason for the name
in common.
a vertex
We insist that no vertex belongs to more than one edge from
match of its two vertices.
A vertex
to any
that is not incident
so that it is a monogamous matching.
the matching
We
also
that
the
does
not
is
called
unmatched.
vertex
the
in
matching
say
belong
edge
are matched. A
is one in which
all vertices
A
to the matching.
perfect
matching
is one with the maximum
number of edges. A maximal
maximum
matching
matching,
is a matching
of an edge.
that cannot be extended by the addition
on the other hand,
occur in many situations (besides social). Workers
be
Problems involving
matching
may
matched to jobs, machines to parts, and so on. Furthermore,
many problems that seem
in terms of matching
formulations
to matching have equivalent
unrelated
problems.
In this section, we limit
in
is
a
difficult
our
general graphs
Matching
problem.
The
first
to two specific matching
is
so
not
it
discussion
problems.
problem
important;
in special very dense graphs. The solution
involves finding
to this
matchings
perfect
an interesting
problem, however, illustrates
an important problem concerningmatching
Perfect
7.10.1
In
this
Matching
we consider
example,
a very
approach,
in bipartite
in Very Dense
restricted
case
which we
then
to solve
generalize
graphs.
Graphs
of the
G = (V,
perfect matching
Let
problem.
is at
and the degree of each vertex
E) be an undirected
graph such that | V \\ -In
least n. We present an algorithm
As a
in such
to find a perfect matching
graphs.
exists.
corollary,we show that, under these conditions, a perfect matching
always
We use induction
on the size m of the
The base case, m = 1, is handled
matching.
of size one. We will
that we can extend
show
by taking
any arbitrary edge as a matching
any
edge
result
with
two
is not
that
matching
new
another
size
of the
edge or
by
replacing
an existing
matching is increased,
and
the
follows.
Consider a
matching
in G
with
m edges
We
first
check
all the
whether
If we find such an edge,
edges
any of them can be added to M.
then we are done. Otherwise,
there are
M is a maximal matching. Since M is not perfect,
at least two nonadjacent
vertices,
v, and v2, that do not belong to M. These two vertices
have at least In distinct
out of them. All of these edges lead to vertices
edges
coming
that
are covered
an edge
could be added to M. Sincethe
by M, since otherwisesuch
number
of edges in M is < n and there are 2n edges from
v, and v2 adjacent to them, at
\342\200\224
least
one edge from M \342\200\224
is adjacent
to three edges from
(\302\253i,
w2)
say
v, and v2.
of generality,
that those three edges are (\302\253,,v,), (\302\253i,v2), and
loss
Assume, without
the edge (\302\253,,w2) from M
(w2, v,) (see Fig. 7.36(a)). It is easy to see that, by removing
not
in M
to see
7.10
235
Matching
v,
----.Jw'
^#
\342\200\242^
fa)
7.36
Figure
a matching.
Extending
v i), we get a
this algorithm
and
algorithm is
in
each
but,
may
affect
of a
example
extension of one
is
general,
finding a good matching
choices of other edges far away
in
this approach to other matching
problems.
in
generalize
7.10.2
\302\243
is a
The
\302\243,
U)
set of
larger matching
as an exercise
This was
the graph.
This
(7.21).
three
difficult.
more
7.36(b)).
(Fig.
in this
sufficient
A choice of
Next, we show
one
edge
to
how
Matching
Bipartite
Let G = (V,
and
of
in the
step
case,
another
(m2,
be a
bipartite
that
such
graph,
edges connecting
V and U
V to
from
vertices
Find a maximum-cardinality
Problem
are two
vertices
in
disjoint
sets of
vertices,
U.
in
matching
a bipartite
graph G.
in terms
of real matching: V is a set of girls, U is a set of
of
we want
to match boys to girls so as to
boys,
\"possible\" pairings;
maximize
the number of matched boys and girls.
A straightforward
no
until
approach is to try to match according to some strategy
more matches are possible,in the hope that the strategy
will
guarantee
optimality, or at
least
come
close. We can try different
For example, we can try a greedy
strategies.
with small degrees, hoping
that the other vertices
approach by first matching the vertices
will
be more
later
on. (In other words, first match
the
likely to have unmatched partners
boys that are the most difficult to match, and worry about the rest later.) Instead of trying
We
this problem
formulate
can
and
\302\243
is
a set
problem.
such strategies
Suppose that we
maximum
matching.
to
analyze
Can
(which
we
is
hard),
we
try
the
approach
maximal matching,
somehow improve it? Consider
start
with a
which
Fig.
used
in
is not
7.37(a),
the
previous
necessarily a
in
which
the
236
Graph Algorithms
matching
replacing
applied
edges.
in
is depicted
by bold
2A with the
edge
the
the
If we
is
and
clear
2B.
can improve
similar to the
that we
This is
the
matching
by
we
transformation
previous
find
lines. It
edges 1A
a similar
two
then
the
we can improve
further by
matching
5E
and
edges iC, 4D,
(Fig. 737(b)).
replacing the edges
these
Our goal is to add more matched vertices. We
transformations.
Let's study
a match
for it. If we already have a
with an unmatched vertex v and try to find
start
so we must try to break
then all of v's neighbors are already
matched,
maximal
matching,
which
We
choose
to
was
another
vertex
match.
w,
v,
a
adjacent
previously matched
up
v
match
to
break
the
match
between
u
and
w.
We now have to
We
u
and
w.
to, say,
up
w
If
w.
is
to
an
unmatched
then
we
are
for
connected
done (this was
match
a
find
vertex,
if
this
matches
we
can
continue
and trying
the first case above);
not,
way
by breaking
we have to do two things.
into
an algorithm,
To translate this attempt
rematches.
First,
we have to make sure that this procedure
terminates, and second, we have to show that, if
then this procedure will find it. First, we formalize this idea.
there is an improvement,
M is a path
from
a vertex v in V to a
An alternating
path P for a given
matching
in A/, such that
are
the
unmatched
of P are
vertex u in U, both of which
edges
E
-M
in
and
in
the
first
P
M.
That
of
does
not
to M
is,
(v, w)
alternatively
edge
belong
v
not
to
the
to
does
second
and
so
until
the
on,
A/),
(since
belong
edge (w, x) belongs A/,
last edge of />, (z, w), which
Notice
that alternating
does not belong to A/.
paths are
The
number
of edges in P must be
exactlywhat we used already to improve a matching.
odd since P starts in V and ends in U. Furthermore,
there is exactly one more
of P
edge
in E-M
than there is in A/. Therefore,
if we replace all the edges of P that belong
to M
by the edges that do not belong to A/, we get another matching with one more edge. For
the matching
in Fig. 7.37(a) was
example, the first alternating
path we used to improve
which
was
used to replace the edge A2 with the edges 1A and 2B\\ the
(M, A2, 2\302\243),
second alternating
was (C5, 3D, D4, 4E, \302\2435),which
was used to replace the edges
path
3D and 4E with
the edges
C5, D4, and E5.
we have
an
For
improvement.
3D and
12
4E
with
example,
the
12
(b)
(a)
Figure7.37Extending
a bipartite
matching.
7.10
It
then
now
be clear
should
M is
not maximum.
It
out that
A matching is
will be
claim
The
matching
a matching.
proved, in
maximum
with the
of a
context
the
theorem
path
is not
only if it has no
if and
maximum
alternating
that
is an alternating path
the opposite is also true.
for
a given
matching
A/,
Theorem
\342\226\241
Alternating-Path
This
if there
that,
turns
237
Matching
has
more generaltheorem,
suggests
immediately
an
path and
alternating
\342\226\2
paths.
alternating
an
any
in the
algorithm,
alternating
next section.
because
any
can
extend
path
We
start
until
we
complexity of the
An
greedy
We
is O
algorithm
(|
as many
adding
algorithm,
(| V \\ + \\ E
for
search
then
|)).
Improvement
in
Since a search
can traverse
the whole graph
traverses one path, we might as well try to find
We have to make sure, however,
that these paths
the independence of such alternating
guarantee
the
same worst-case
several
do
that
paths
12
(a)
(b)
Figure 7.38
Finding
alternating
it
not modify
12
time
running
paths.
Graph Algorithms
238
disjoint. If
the
vertex
are
paths
disjoint,
improved
applied concurrently. The new
First, we perform BFS in G'
following.
different
modify
they
algorithm
the set
so
vertices,
for finding
alternating
of all
unmatched
U are
found.
they
can
be
is
the
paths
in V, level
we
extract
from
Then,
level,
by
in
a
set
of
vertex
G'
the
BFS
maximal
induced
(which are
the
disjoint paths
by
graph
in G). This is done by finding
any path, removing its vertices,
alternating
paths
finding
is not a maximum set, but
and so on. (The result
another path, removing its vertices,
set in order to maximize the
number
a maximal
of
set.) We choosea maximal
merely
with
one
search
to
the
added
adds
(each
vertex-disjoint alternating
matching
path
edges
we modify the matching
one edge to the matching).
Finally,
using this set of alternating
no more alternating
the new
This process is repeated until
(i.e.,
paths can be found
paths.
in
V
from
the
the unmatched vertices
unmatched vertices
G' disconnects
directed graph
a level
until
vertices
unmatched
which
in
from
in
vertices
inU).
Complexity
0(v\\V\\)
[1973].
The overall
7.11
Network
The
Furthermore,
E)
G=(V,
indegree
weight
0,
that
1.
of
/ (the
an
edge.
0 to
satisfies
the
proof,
which
is thus
O ((|
+ \\ E
\\
|).
| )V|V
of
the
two
capacity of e.
We
call such
The
flow
through
is a
the amount
function / on
an edge cannot
the
with
source)
(the
an associated
exceed
the
total
flow
that
can
we assign a
the network
of
edges
positive
of flow
For convenience
a network.
graph
0<f(e)<c(e):
measures
E has
network-flow
as follows. Let
is defined
vertices, s
distinguished
capacity
posed as
can be
problem
extensions.
and
variations
0. Each edgee in
The
35 years,
many
problems
network-flow
and combinatorial
and many
algorithms
theory
graph
last
the
It has
it.
for
with
graph
in
extensively for
called the
pass through
capacity
directed
be a
of the improved
algorithm is
is due to Hopcroft and Karp
iterations
basic problem
unrelated
seemingly
basic variation
and
c (e),
flows is a
been studied
many
The
time
data structures
problems.
the
Flows
It has
optimization.
and
worst-case running
network
of
problem
We omit
case.
worst
of
out
turns
It
the
in
of
capacity
that edge.
2.
is equal to
/),
the
\302\243/(m,
u
total
flow
v) = \302\243/(v,
w
exiting
w):
The
this vertex
(exceptfor
the
entering a
source
and
sink).
as
area,
much
water through
the pipes
will
burst.
the
pipes
as possible.
If too much
flow
water
then
entering
it is not
exist.)
pipes.
One
The
is pushed
7.11
be
how
to solve
The
reason
posed
problem
the
Given a
problem of
network-flow
problem
the
(namely,
similarities
matching
the
is in
reduction
the wrong
direction).
is that
the
solving
239
in the previous
to
seem
be a fruitless
may
but
we
do not know
problem,
This
problem.
to solve
how
Flows
discussed
matching,
bipartite
present
Understanding
know
network-flow
the
we
network-flow
as a
we already
since
exercise,
that the
show
we
First,
section, can
Network
in understanding
can be helpful
in which we
G = (V, \302\243,
U)
the
matching problem.
network-flow
algorithms.
bipartite
want
to find
a maximumtwo
s
new
vertices
connect
s
to
all
vertices
in V, and
and
f,
cardinality
connect all vertices in U to t. We also direct all the edges in E from V to U (see Fig.
7.39, in which all edges are directed from left to right). We now assign capacities of 1 to
on the modified graph
and we have a valid
network-flow
the edges,
G'.
Let
all
problem
in G. There is a natural
M and a flow in G\\
between
M be a matching
correspondence
of 1 to all the edges in M and to all the edges
s or Mo
We assign a flow
connecting
in M. All the other edges are assigneda flow
of 0. The total flow
vertices
is
matched
of edges in the matching.
It turns out that M is a maximum
thus equal to the number
if and only if the corresponding flow
is a maximum
flow in G\\ One side is
matching
If
flow
is
to
a
the
maximum
and
it
clear:
matching, then we cannot have a
corresponds
since
it would correspond to a larger flow.
For
the other side of the
larger matching,
have to adapt the idea of alternating paths to network
and to
claim
we somehow
flows,
flow
is
maximum.
We
show that, if there are no alternating
then
the
paths,
corresponding
to
that.
do
proceed
just
from
s to t
An augmenting
to a given flow
/ is a directed path
path with respect
which consists of edgesfrom G, but not necessarily in the same direction; each of these
edges(v, u) satisfies exactly one of the following two conditions:
we add
matching,
1.
(v, u) is
in
Reducing
left
same
the
Figure 7.39
edges are from
graph
bipartite
(v,
direction
u) is called
bipartite
to right).
as it is
a forward
matching
in G,
and /(v,
edge.
to network
flow
forward
(the
w)<c(v,
edge
w).
In
this
directions
of all the
240
Graph
more
2.
Algorithms
is in
u)
of
paths
Augmenting
for network
flows
extensions
alternating
as
with two
Fig. 7.40. Each edgeis marked
the current flow.
It is clear that no more
from
s to t that consists of only
be pushed forward, since there is no path
can
flow
is
a
to
flow.
there
extend
the
forward
However,
way
edges.
An additional
is an augmenting path.
flow of 2 can reach
The path s-v-u-w-t
u
slack
over all forward edges until
from s through this path (2 is the minimum
w). We can
is now satisfied for w, since
deduct a flow of 2 from / (w, w). The conservationconstraint
in through
of 2 coming
the augmenting
u had an additional flow
path, and a flow of 2
deducted from the backward edge. We now have an extra flow of 2 at w that needs to be
can continue pushing flow
We
from
w in the
pushed, which is exactly what we want.
same way, pushing
it forward on forward
and deducting
it from backward edges.
edges,
In this
case, there is one forward
(w, t) that reaches f, and we are done. Since only
edge
forward
to
is equal
r, the total flow is increased. The increase
edges can leave s and enter
the minimum of either the minimal
flow
slack of forward edges or the minimal
current
7.41
shows
the same network
with
the modified flow.
through backward edges. Figure
edges is a
little
more
complicated.
(This flow
is in
a is
Consider
the capacity
and
b is
fact maximum.)
Figure 7.40
An
example
of a
network
with
a (nonmaximum)
flow.
Network Flows
7.11
Figure7.41The
The
not
The
The
\342\226\241
A
path,
is a
a cut
V such
sum
and prove
set of edgesthat
e A and 14
of the
w) e E]
capacities of
if it
only
not
augmenting
the flow
then
path,
is
edges.
v e
path.
augmenting
of the
direction
theorem \342\200\224
if the
now
assume
Let's
maximum.
that
a flow
flow admits
/admits
no
that/is
that
admits no
one
shown
separate
its
is an
Fig. 7.40.
Theorem
A. Denote
such
there
if
of
flow
is also true:
maximum if and
that s
of edges {(v,
that
establish
We have already
path, then it is
augmenting
augmenting
opposite
Augmenting-Path
f is
flow
Proof:
an
above
arguments
maximum.
the
of augmenting
result
241
the
rest
A and
It is
of the
vertices
we B. The
clear
that
no
capacity
flow
by
B =
V -A.
of the
can exceed
cut
is the
cut is defined as
the
capacity
set
the
of any
Algorithms
Graph
242
We have provedthe
Theorem
Min-Cut
Max-Flow
\342\226\241
The
\342\226\241
//
the
theorem
implies
maximum
matching,
augmenting path
there is a maximum
vertex
the
to the
the
is a
there
then
theorem.
augmenting-path
to an integral flow
a flow of 0,
lead
paths
will
since
we
with
start
if
In fact,
all the
each
and
total flow.
\342\226\
Clearly,
versa.
problem.
bipartite-matching
in
path
alternating
any
The augmenting-path
augmenting
path
in the previous
the alternating-path theorem given
section. If M is a
which
then there is no alternating
for
it,
path
implies that there is no
in G', which
is
the
flow
maximum.
On the other hand,
that
implies
to a matching
since
each
flow, and it clearly
integral
corresponds
in
of
flow
by only one
1. The
same
since, if
it
vice
and
G\\
edge (with
only
be maximum
must
matching
obvious
an
connected
V is
in
can support
integer
return to
now
We
G corresponds to
from
directly
is
This
are integers,
integer.
capacity
theorem.
following
network
the
in
is an
follows
integers.
path adds
augmenting
edges
value
theorem
that
algorithm
capacities are
minimum
Theorem
whose
flow
Proof: The
any
of all
capacities
\342\226\
Integral-Flow
maximum
in a
flow
theorem
augmenting-path
theorem.
fundamental
following
could
of V
s; hence, each vertex
the vertices of U. This
be a larger
then there would
1) to
capacity
for
holds
argument
be extended,
flow.
The augmenting-path
0, searchfor
no more augmenting
flow. Searching for
flow of
residual
theorem
paths,
augmenting
We
paths.
augmenting
with respect to
graph,
and augment
network
in an augmenting
the
possible
belongs to F
or a backward
directed
regular
requires
\\E
the
augmenting
it is either
if
edges
possible
flow through
forward
edge,
flow
Their
path.
those edges.
case
More
its
/,
the
precisely,
is c(v,
in which
graph
edge
w)-/(v,
path
the
with
to
correspond
an
increasing
residual
capacities
capacity
are
We define the
as the network
the
in
with
start
until there
and the
The edges
capacities.
in which
and a
sink,
are
way.
following
E)
G=(V,
correspond to
different
the
in
We
accordingly,
progress since we
= (V,
flow
the
an algorithm.
suggests
immediately
residual
(v, w)
w),
is thus
graph
lead to a very
in an arbitrary
way
may
selecting augmenting paths
be a
The worst-case running
time
of such an algorithm may not even
function of the size of the graph. Consider the network
flow
in Fig. 7.42. The maximum
is obviously
a
2Af. However, one might
start
with the path s-a-b-t,
which
can support
one might take the augmenting
1.
flow of only
which again
Then,
path s-b-a-t,
Unfortunately,
slow
algorithm.
Figure 7.42
augments
even
very large,
of M can
case,
the
size of
number
of
(IV 13-
IV
next
has only
be
edges.
I
)/4
polynomial
An
this algorithm is
and Karp
upper
They
Some
this
We
started
the chapter
by taking
path
proved
chapter
with
Problem
that includes
worst
the
in
precautions to avoid
in
path
to speed
order
an
complexity
not
describe
these
worst
whose
algorithms
network
most
at
then
simple
of
up
things)
algorithm
other
maintained,
different
Many
input.
are complicated;
is
policy
have been
are really
(none
been
algorithms here
flow has
Bibliography section).
Tours
with a
a discussion
discussion of a tour
of a
tour
containing
all edges
containing
all the
The
to take
augmenting
This leads to
required.
the
the
this
if
that,
on the
of 0(|V|3)
of these algorithms.
We
in the
have
we
of augmentations
bound
Hamiltonian
end
exponential,
are
augmentations
in the
size of
achieved by several
(referencesare given
repeated
vertices
four
bits,
O(logM)
augmenting
suggestedsince then.
7.12
processcan
input.)
Edmonds
the
selecting
the
we want
algorithm.
simple).
graph
of network flow.
example
the scenario
Furthermore,
case is
the
though
Although
it.
1. This
by only
be representedby
the
in
flow
the
bad
243
Tours
Hamiltonian
7.12
Given a graph
find
G=(V,
\302\243),
once.
every vertex of V exactly
vertices of
Sir
a simple
of a graph.
a graph.
William
cycle
R.
in
We
This
Hamilton,
is
Graph Algorithms
244
Hamiltonian
consider
the
Unlike
characterizing
interesting
Eulerian-tour
reversed
induction.
7.12.1
Reversed
We
have
(e.g., S =
set S
the
theorem
that
proving
an
algorithms
solve
that
the
do
[2k),
base
on
= 1,
affect
in
In
of finding
this
It
the
Hamiltonian
to
belongs
we present
graphs that are
section,
special
only
is
example
use
as
hand,
base
values of
is much
case
the
it
Section
in
induction
2,...)
from n to
the other
problem
not
we
version;
will
an interesting
of
cycles
(or
of NP-
the class
a simple example
The
dense.
very
called
technique
Induction
k
going
infinite
cycles
this
mathematics,
11.
in Chapter
seen reversed
already
are called
such cycles
undirected
an
is very
graphs)
of
the problem
difficult.
problem,
discussed
part
directed and
version.
find Hamiltonian
we
which
most
containing
Graphs
cycle.
has a
problem
Hamiltonian
problems
complete
in
Hamiltonian
The
case
that
P(n)
implies
- 1 is not easier
more
is almost
difficult
Then, we go
of
validity
can
n-\\ to w,
from
many
that
\"backward,\"
and
in
proving
When designing
namely, to
nton-\\,
introduce
in
infinite
Usually
P(n-\\).
one.
simple
always easy to go
use an
is, we prove
That
induction.
to S.
belong
the
The idea is to
2.11.
for the
\"dummy\" inputs
cases to design the
this principle.
7.12.2
Finding
of the
let d(y) denote the degree
and
G=(V,
E) be a connected undirected graph,
vertex v. The following
involves
problem
finding Hamiltonian cycles in very dense
is
We will show that the conditions of the problem
that the graph
graphs.
guarantee
the
to illustrate the principle of reversedinduction.
We introduce
Hamiltonian.
problem
Let
Given
The Problem
n > 3 vertices, such
that
satisfies d(v) + d(w)>n,
The
algorithm
is based on
the
complete
graph.
Hamiltonian
and
cycle
connect them
in a
Induction
satisfying
a connected
each
find
a Hamiltonian
reversedinduction
Every complete
it is easy to find
cycle
with
one
with
E)
v
245
and
G.
the number
on
graph
in
Tours
Hamiltonian
7.12
(put
at
least
cycle).
hypothesis:
the
given
how to find a
with > m edges.
know
We
conditions
Hamiltonian
cycle
in graphs
to show
have
satisfies
the conditions
Hamiltonian
cycle.
two
conditions
vertices
of the
jc, and
edges
in G
problem).
jc/+1
, which
of
out
xh
the
edges
not
*/_!,...,
v and
n-2
G contains
are neighbors
and w is connected
to xh Using
does
Hamiltonian
cycle that
v(=x!), xi+l,*/+2,...,w(=xn\\
coming
But
in
the
cycle,
(v, jc/+1)
use
v (see
the
edge
Fig. 7.43).
Hamiltonian
(v,
w).
cycles.
can now
find
to jc,+1
a new
is
the
cycle
is connected
It
Graph Algorithms
246
Implementation
The
complete graph
much smaller graph
and
edge at a
one
replaces
of this
implementation
straightforward
as follows. Take
time.
graph G,
proof
do better
can
We
with
starts
by starting
the
a
with
and
edges
from
(not
edgesto
replace;
7.13
Summary
hence,
the
input
a large
find
G) necessary
has
G', which
the algorithm
in time
runs
0(n2).
used
are
Graphs
examination of
than
for
have
for
useful
major difficulties
directed acyclic graphs are
It is
important
different
many
for
much
usually
cycles (Exercise
7.54).
even
with
involving
to design
a small number
easier
that
Algorithms
graphs.
weighted
Therefore,
algorithms.
graph
that graphs
realize
to
problems
optimization
usually cause
Cycles
trees or
execute.
It is
search.
regular
whole
the
components,
and
consideration
may
properties
not have
by
algorithms.
We
Network-flow
discuss
problems
also
discuss a class
help
of
connectivity),
reasonably
biconnected
them.
this
and
us to
components.
(such as
technique
matrices
there
represented
Reductions
connected
useful
Another
in
strongly
certain
assume
all or
checking
require
to
faster
of edges can
a graph
can be very slow for most graphs.
cycles
useful.
it is also usually
Graph decomposition is very
Fortunately,
We have seen decompositionsinto
connected
inexpensive.
components,
large
of the
fraction
algorithms
and
for graph
is a
called
running
reductions
and
whether
Since graphs
is reduction.
relationship
problems
matching
natural
relationship
determine
problems,
algorithms
between
in general
are excellent
a problem
graph
in
and
can be
matrix
10.
Chapter
is difficult.
In
11, we
Chapter
which probably
NP-complete
problems,
in the
size of the
times
are polynomial
be
cannot
input
in
the
numerous
The differences betweeneasy
graph problems.
sometimes seem minuscule. For example,we have seen an
efficient algorithm
whether a directed graph
to determine
a simple cycle of odd
contains
the same
vertex
length;
problem with the extra constraint that the cycle contains a given
to understand and develop
an intuitive feeling
(or edge) is NP-complete. It is essential
problems
and
hard
problems
for
differences.
these
Thus,
is a
theory
only
this
field,
and
[1979],
for
important
very
now
graph
books
the
algorithms,
graph
and related
Even
results
basic
were
theory is a developedand
on graph theory have been
[1969],
[1973], Deo
Berge
Murty
Tutte
books devoted to
Most of
in mathematics.
field
new
Nevertheless, by
century.
with thousands of results. Many
them
among
[1974], Bondy
Bollobas
relatively
in
well-understood
published,
11 is
Chapter
Notes
Bibliographic
Graph
in
graph algorithms.
understanding
discovered
material
the
247
Reading
Molluzzo
[1979], Golumbic
[1978],
also
are
several
[1980] (which
classes of
Gondran
and Minoux [1984]
graphs),
Gibbons
Nishizeki
and Chiba
[1985],
(which
[1988]
emphasizes
optimization
problems),
and a survey paper by van Leeuwen
(which is devoted to planar
[1986].
graphs),
The notion
of Eulerian
graphs is due to Euler [1736], and it is regarded as the first
An algorithm
in graph theory.
for finding Eulerian paths
result
can
be obtained
quite
Even
from
the
for
or
Ebert
search
(see,
[1979]
[1988]).
easily
proof
example,
Depth-first
was first described by Lucas
work by Tr6maux)
and Tarry [1895],
[1882]
(describing
where it was used to design algorithms
to traverse
a maze. The importance
of depth-first
in
the
of
who
also
the
search was made evident
work
algorithms
presented
Tarjan [1972],
for biconnected and strongly
connected
components.
tree
has been studied extensively. The
The minimum-cost spanning
problem
is due to Prim
in Section
7.6 (although
not
its implementation)
algorithm
presented
is
the
of
Exercise
is
due
to Kruskal
7.59)
[1957]. Another
(which
algorithm
subject
were
for
the
minimum-cost
tree
Other
[1956].
finding
algorithms
spanning
developed by
and Gabow, Galil,
Yao [1975], Cheriton
and Tarjan
[1976], Fredman and Tarjan
[1987],
emphasizes
perfect
graphs
Tarjan
[1986]
Spencer,and
in Section
7.5 was
for single-source shortest paths
presented
a heap is due to Johnson [1977]
The implementation using
[1959].
the case in practice, this
When the graph is sparse, as is usually
(see also Tarjan [1983]).
is a fast algorithm. If the number
time
of edges is proportional
to | V |2, then the running
of this algorithm
with
a
is O (| V | 2log
V
for
dense
A
better
|
\\.
graphs,
implementation
The
time
of
is
the
of
Exercise
7.43.
best-known
0(\\V |2),
running
asymptotic
subject
time
for this
data structures) is
running
(using
problem
quite
complicated
due
to Fredman
and Tarjan [1987]. The all-pair
0(\\E | + |V|log |V|), a result
in
Section
7.7 is due to Floyd [1962]. It works
algorithm
shortest-paths
presented
The
algorithm
developed by
correctly
Dijkstra
for weighted
negative weight
on the average
0(|
and
time
survey
V
an
|2 log2
\342\200\224
Spira
|V
earlier
is 0(\\
graphs
(Exercise
cycles
|),
and
by Deo
possibly
7.73).
[1960] to obtain
|). For more information
and Pang [1984] (which
includes,
|2 log
|V
there
and
by Dantzig
algorithm
V
[1973]
Moffat
with
an
on
algorithm
shortest-path
among
are no
paths faster
other things,
is
time
algorithm
the
222 references).
closure
transitive
The
and Plummer
and
theory
in
Hamiltonian
finding
[1986] covers
for various
algorithms
algorithms
matching
[1982], and
and Steiglitz
Papadimitriou
[1978],
Lovasz
bipartite
in dense
cycles
mathematical
problems.
matching
and
and
Gondran
the
both
Galil
Minoux
[1984]. A book
foundations of matching
general
graphs
[1986] presents
in
graphs. The algorithm
is based
on a theorem (and
of
survey
7.12.2 for
Section
its
by
by Ore
proof)
[I960].
Two important
subjects
isomorphism.
The
developed by
and Tarjan
Hopcroft
a graph
decompose
polynomial-time
isomorphism is
[1974].
It
a linear-time
uses
3-connected
the
NP-hard) is still
see, for example,
unknown
(more
Hoffman
[1982],
on that
or Luks
in
11).
Chapter
here:
and
algorithms
Lempel,
a graph
and
them
for this
planarity
embedding
and
Even,
is planar was
to
(DFS-based) algorithm
and Tarjan [1973]). This
and data structures. A
components
(Hopcroft
of many
other
development
algorithms
for graph isomorphism has not been
algorithm
one of the very few major problems whose status
into
motivated
algorithm
in graph
problem
For a
found
(either
Graph
yet.
or
polynomial
discussion on
this
topic
[1982].
on de
discussion
Exercise7.46 is
Drill
7.1
from
Exercises
Considerthe
problem
7.2
LetG = (V,
E) be a
and
let T
be a DFStree
of G
rooted
at
5.8.
v.
a. Let H be an
b. Let R be
Prove that
7.3
7.4
7.7
7.8
Characterize
all
and
identical
traversed
is
H is
let S be
tree of S.
the
of G
subgraph
induced
vertices
in R.
the
by
graphs
at v that
if
immaterial
is identical
algorithm
Modify
know
or
whether
Sorting (Fig.
not the
Consider algorithm
Source Shortest Paths (Fig. 7.17) Prove
Single
consisting of all the edges that belong to shortest paths from v, found
of the algorithm, is a tree rooted at v.
the
that
Prove or show
works correctly
there are
a counterexample:
for
weighted
no negative-weight
7.11
Prove
MCST (Fig.
algorithm
graphs some of
weighted
7.20) to
7.12 a. Give
such
rooted
b.
an example
that
graph.
minimum-cost
find
a counterexample: algorithm
have negative
edges
or show
the
whose
of a weighted
minimum-cost
Prove that,
tree.
if
all the
a maximum-cost spanning
connected
undirected
graph
spanning tree of G is
the
same
(Fig.
7.17)
weights, provided
then
tree.
for weighted
correctly
at v.
connected
undirected
an example of a weighted
graph G
the minimum-cost
spanning tree of G is very different
tree rooted at v. Can the two trees be completely
disjoint?
=(V, E)
that
7.14 a. Run
a constant
spanning
Give
such
Paths
have negative
edges
tree rooted at
by
cycles.
Algorithm
subgraph
execution
the
during
7.10
7.13
of T and
undirected
rooted
tree
trees are
that
7.9
7\\
the intersection
input
no longer
7.6
of
be a DFS
R could
that
Show
of H.
tree
The
spanning
7.5
a subtree
G.
subgraph of
induced
arbitrary
249
Exercises
Drill
the
biconnected
the
biconnected
components
in Fig. 7.25.
and
from the
and biconnected
tree
a vertex
shortest
resulting
from
the graph
components
algorithm on
the
graph
in Fig.
7.44. The
v,
path
algorithm
250
Algorithms
Graph
should
b. Add
7.15
that
7.16 a.
the edge
there
are no
the
Run
(4,8) to the
given
the
High
values
as
graph
changes this
to the
makes
algorithm.
of a biconnected
tree in Section 7.9.1 is valid.
You have to show
are connected.
cycles, and that the set of all biconnected
components
strongly
should
algorithm
b. Add the
connected components
the DFS numbers
follow
values as computed
7.17
are
that
in each step.
the algorithm
the definition
that
Prove
by
DFS numbers
the
follow
computed
by the
edge (4,1)to
algorithm
the graph
in
each
algorithm
the graph
on
are given
that
in
the
figure.
step.
changes
this makes
to the
algorithm.
connected
Let G = (V, E) be a strongly
graph and let T be a DFS tree in G. Prove that, if all
from G, the resulting graph is still
the forward edges in G, with respect to 7, are removed
strongly
connected.
Figure 7.44 An
undirected
graph with
DFS numbers
for Exercise
7.14.
Figure
7.45
A directed graph
with
DFS
numbers
for Exercise
7.16.
Creative Exercises
7.18 a.
the
Prove
correctness
an odd
finding
length cycle
in
a directed
251
graph
(Section 7.9.4).
b. Show
7.19
7.20
of a graph
an example
that is
not
connected
strongly
does
work.
not
This
generalizes
you
Exercises
Creative
encountered.
7.21
an undirected
Given
H of G
such
that
induced
subgraph
includes
all edges
induced
subgraph
graph G = (V, E) and an integer k, find the maximum
that it does not exist. (An
each vertex in H has degree > k, or determine
F
// = (\302\243/,
and
of a graph G = (V, E) is a graph
F) such that (/cV,
E both of
in
7.22
is discussed
in
U.)
The algorithm
should run
in linear
undirected
(V, E) be a connected
graph. We want to pick a vertex of degree 1 of G,
another
vertex
it and its incident edge from
G, and continue this process (i.e., taking
of degree 1 in the remaining graph, removing
all edges are removed. If
it, and so on) until
this procedure
is always
for certain graphs, then designing
possible
algorithms
by induction
for these graphs
easier.
that
these
be
Characterize
connected
undirected
may
graphs
satisfy
In other words, find necessary
conditions.
and sufficient conditions for a graph G on which
the procedure
described above is possible.
Let
G =
remove
7.23
Describe
7.2.
an efficient
implementation of the Eulerian
graph algorithm discussed
The algorithm should run in linear time and space.
equal to the
7.25
A directed
E)
be an
algorithm
in
Section
undirected graph such that each vertex has an even degree. Design a
to direct the edges of G such that, for each vertex, the outdegree
is
indegree.
Eulerian
circuit
is a directed circuit
that
contains
once. Prove
252
Graph Algorithms
an efficient
Let G = (V,
E) be
a directed graph
that
is equal
vertex
7.26
a. Prove
b.
7.27
graph
of odd degrees.
with k vertices
even.
k is
that
Design
an
exactly
one of these
to
algorithm
k/2
find
paths such
open
that
each
in G is
edge
included
in
paths.
Design an algorithm to
de
A binary
s of size
unique index i
somewhere
is represented
s =
that
such
sequence of 2\"
is a (cyclic)
sequence
Bruijn
string
binary
find
the
disconnect
7.28
an undirected connected
at
ai+x
\342\200\242
\342\200\242
\342\200\242
in
(where
fl/^-i
the
bits
\342\200\224
\342\200\242
a2\" such
axa2
that is,
sequence;
taken
there
modulo
that each
exists
2\.") For
11010001 is a binary
de Bruijn sequence for n =3.
Let Gn = (V, E)
as
follows.
The
vertex
V
defined
set
to
the
of all binary
set
corresponds
graph
\342\200\242
\342\200\242
- 1
A vertex corresponding to the string axa2 \342\200\242
(| V \\ =2\"~l).
strings of size n
an.x has
'''
to a vertex corresponding to the string bxb2
an edge leading
if and only if
bn_x
\342\200\242
''' an-\\ -bxb2
\342\200\242
\342\200\242
Prove that Gn is a directed Eulerian
and discuss
bn_2.
#2^3
graph,
the implications for de Bruijn sequences.
example, the
be a
7.29
sequence
directed
= 1 for 1
ik
integers
n vertices of
positive
with
dn.
d2,...,
dx,
that
such
dny
degreesexactly
7.30
Given
algorithm for the following
problem:
\342\200\242\342\200\242\342\200\242
construct a tree
+dn = 2n-2,
dx+d2+
efficient
an
Design
dx>d2,...,
be a sequence
of pairs
of integers
such
that
< k <n
b.
Find
= n-\\
^Oj
tree
a rooted
An
greater
undirected
than
the label
such labeling
of vertex k
algorithm
is
is ikt and
its outdegree is
(n).
of
at
is possible,
graph G = (V, E)
k different
colors
Design a linear-time
7.33
indegree
v is
colored using
are not
the
that
run in time O
vertices
that no
determine
7.32
should
E)
to label the
each vertex
such
n vertices
with
to
said
Design
to | V
\\
an efficient algorithm
such that the label of
least
be ^-colorable
if
all
the vertices
of G can be
such
that
no
sufficient.
Let G =
two colors.
It may be
graph that can be colored with
colors
in
Exercise
several
in
different
the
Use
possible
ways.
algorithm
7.32 to prove that the coloring of G is unique
for interchanging the colors, which
(except
can always be done) if and only if G is connected.
(VyE) be
to
color
an
undirected
G with two
Creative Exercises
7.34
T be an
Let
associated
7.35
253
binary)
necessarily
(not
a character
with
finite
a connected
Given
edges such
the
that
since
injective
algorithm?
7.36
7.37
Given
a.
b.
edge,
Any
does not
7.38
direct
\302\243),
a rooted
contains
graph
G =(V,
to the
belong
its edges
such that
tree above,
the
completesa
it is
two
following
whose edges
all of
directed
cycle
is
it
point
with
algorithm?
in G
path
should
that has
run in
the
linear
time.
7.39
a. Solve the
problem in
c. Will
your
each two
E)
simple
shortest
not
7.42
7.43
(not
necessarily
v and
The algorithm
in the
are now
acyclic) graphs?
number
be a directed acyclic graph, and let k be the maximal
to divide the vertices into at most k + 1 groups
algorithm
vertices
graphs.
paths.
cost edges?
negative
case of weighted
of edges in a
such that for
Design an
w to v.
from
is the
weight
b. Will
algorithm
Exercise
looking
and
there is
no path
should
the following
property. G consists of an acyclic
back edges, such that every
vertices, and additional
a linear-time
path in G contains at most one back edge. Design
algorithm to find all
all
source
to
other
a
fixed
of
from
vertices
G.
that
the identity of H is
(Note
paths
//,
E)
which
known.)
requires
an
implementation
running time of
of algorithm
\\2) in the
O (| V
Paths
of E).
(Fig.
7.15)
which
254
Graph Algorithms
7.44
Let
7.45
7.46
E) be a weighted
minimum weight. The algorithm
G = (V,
an
to find a cycle
algorithm
(|
in
G of
|3).
graph is an undirected
plane and the weight
Euclidean
in the
weighted
of an
a.
How
would
you implement this algorithm? You
from the implementation of Dijkstra's
algorithm.
b.
Explain
c.
Give
an example
running
The
is a
input
cost
positive
so
if
(v, //)
path
not
this
an
heuristic
is much
example
where
number
g E,
the
cost
as
(a,).
\302\243<
to mention
have
the
only
differences
(non-Euclidean) graphs.
faster
it is
(by
than
more
faster.
not
What
a constant) than
is the worst-case
of vertices?
a*, // is defined
|, a2
minimum-cost
where
and
algorithm,
time in terms of the
Dijkstra's
7.49
should
point
7.47
Design
graph.
run in time O
directed
vertex
a distinguished
The costs
of
the
two
\\\\
that there
such
The cost of a
w.
vertex
endpoints
an
efficient
v and
is a
directed
path
are
ignored,
algorithm
to
find
the
paths from
There is a large
form. The input
this case, shortest
edge
in
class
of problems,
is a
the path).
a. Design an algorithm
to solve the single-source shortest-paths
costs
are defined as above. Can you say something special
paths obtained
by this algorithm?
b.
Design
an algorithm
defined as above.
to solve the
all-pairs
shortest-paths
where
problem
about
the
the
path
tree of shortest
path
costs are
7.50
(V, E) be a
G =
Let
a linear-time
source v.
directed
acyclic
weighted
to solve the
algorithm
graph
with
single-source
weights. Design
from a given
negative
possible
problem
shortest-paths
7.52
of a
corresponds
7.54
cycle.
disjoint
an efficient
Design
to solve the
an algorithm
Design
Input:
algorithm
the
S from vertices
Output: A function
conditions are satisfied:
1. If(w,
2. Ifw
Consider
any
the
(and
words, prove
7.57
that
in
(no
n vertices and
n + \\ vertices
with
edge
to
(w,
subsets
S(u){j{X(u,
and
w)
is labeled
of
{1, 2,...,
whose
n edges,
a unique
with
n)
such
underlying
integer X(u, w)
that
w)}
S(u)\302\261S(w)
subset
of {1,
2,..., n)
including
the empty
set or the
whole
set.)
labeling)
any
in a directed
are
an edge,
V\\
by
E) be a
=0
2 f(tk)
each edge
\302\243,
even
connected
The input
7.59
S(w)
can be any
again
graph
then
then
*w,
S(u)
(Note:
7.56
w)e\302\243,
problem:
following
A directed graph
G = (V, E)
a
is
where
each
tree,
graph
range 1, 2,..., n.
find
\302\243),
a graph
with
that
undirected
in
which
into
Given
an
number of cycles;therefore,
inefficient
for general graphs.)
7.55
under
conditions
partitioned
these conditions.
that satisfy
graphs
7.53
the
denote
adjacency lists
if
e i,...,
(v, w), we
Here is a sketch
one edge at a
of a
time,
e* is a circuit
have/(v,
in
G.
function
Find a function
on all
defined
on
the
edges of G such
of G
vertices
such that
no
that
for
w)=p(w)-p(v).
of keeping one
different MCST algorithm.
Instead
of disjoint trees (which are all
we keep a collection
tree
part
enlarging it
of the MCST)
and
256
Graph Algorithms
them, adding one edge at a time. Initially, all the vertices are considered as
the minimum-cost
of size 0. In each step, the algorithm finds
edge that
the edge. Prove that
these two trees by adding
two separate trees, and combines
connects
Describe an implementation of an algorithm
based
such an approach is feasible and correct.
on this approach. What is the complexity of your algorithm?
(Hint: The Union-Find data
combine
and
trees
disjoint
structure is helpful
7.60
here.)
Let G = (V, E) be an
(i.e.,F doesnot
contains
G that
undirected
weighted
algorithm
any cycles). Design an efficient
has minimum
cost
the edges of F, and that
contain
all
that is a forest
tree in
of G
a spanning
find
among
all
trees
spanning
containing F.
Suppose
under which T is no
MCST or to determine
conditions
a new
find
7.62
be a connected
E)
tree of G.
the
longer
that T
undirected
cost of one
an MCST.
is still
and let
graph,
edge e
Design
an MCST.
(e
in
an
may
T be a
minimum-cost
is changed.
efficient
or may
Discuss
algorithm
the
either to
not belong to
T.)
a.
has just
cost
whose
U.
In
other
the
modify
b.
under the change in e's cost is guaranteed not to affect the MCST.
that can be checkedwith the information
known to the sites in
that no action
needs
to be taken to
words, how can the sites in U determine
MCST after the change?
under which
from the original one
the modified
MCST is different
that belong to U (hence, the change can be handled locally). Consider only
that can be checkedwith the information
known to the sites in (J.
the
in edges
conditions
c. Describe
the
conditions
conditions
Find
only
within
changed.
conditions
the
Find
Consideronly
7.63
weighted
that
an algorithm
in parts a and b (again only
to check for the conditions
to modify
does not need to handle
the MCST accordingly. The algorithm
case where the change to the MCST may be outside
U.
briefly
U), and
is
Consider the problem of broadcasting
interest
now that the main
in a network, but assume
fast dissemination of information
rather
than minimum cost. In other words, the costs
to the time it takes to forward a message, and we want to minimize the elapsed
correspond
time
of broadcast.
A message can be sent concurrently on separate
links.
Assume that one
messageis sent from a fixed source and is forwarded to all other sites such that each site
receives only one copy of the message. You can assume that you are a controller
with full
of the network.
information about the topology
Creative Exercises
a. Design
b.
to determine
an algorithm
are associatedwith
7.64
of t(v)
Let G = (V, E) be a
connected
weights are
and
of G
7.65
7.66
positive
units
to find T(e)
undirected
of
variation
only delays
there
are also delays
forwarding when
site v to forward a message to one
then it takes kt(v) time. (The
neighbors,
an
edge
for all
should
run in time O
Solve a
the
that
of time for
weighted
Let e be
distinct.
minimum
has
that
algorithm
optimal
It takes t (v)
neighbors.
values
the
determine
associatedwith
of its
assuming
forwarding,
optimal
the links.
an algorithm to
the sites.
Design
the
257
the problem
in
7.65
Exercise
directed
for
(|
|2).
tree of a
spanning
is defined
are
asked
The
graphs:
to
as the
a
find
is a
input
a distinguished
vertex v. Find a rooted spanning
G = (V, E) with
weighted directed graph
tree, with v as the root, such that the maximum-cost edge in the tree is minimized. (Recall
that
in a rooted tree the directions
from the root.)
of all edges are away
7.67
Let G = (V,
that
is it
how difficult
*
7.68
Let G = (V,
Suppose that
in
G are
to modify
weighted graph,
weighted
we now
and
increased by a constant
T into an MCST?
be a connected
E)
of G.
vertices
7.69
be an undirected
E)
Design a linear-time
undirected
to G,
some
with
to find a new
algorithm
and let
graph,
together
T be
weighted
MCST that
an
MCST
now
of G.
edges from
includes
not,
to
v.
that
the cost of a spanning tree is not the sum of the costs of the tree's
edges but
an efficient algorithm to find
rather the product of their costs (all costs are positive). Design
this assumption.
a maximum-cost
(You can assume that all costs are
spanning tree under
Suppose
distinct.)
E)
vertices numbered
components contain
edges incident to it.
7.71
be a connected
an efficient
(Hint:
data
union-find
structure.)
E)
Prove
weighted
cycles.
that
algorithm
graphs
undirected graph
in Exercise
set (defined
be a weighted
to find a feedback-edge
7.73
Use the
All_Pairs_Shortest
with possibly
negative
with
positive weights.
7.71) of G of minimum
Paths given
weights
provided
in
feedback-edge
find
set if
a minimum-size
Design an
algorithm
weight.
for
258
Graph Algorithms
directed graph
be a weighted
E)
no
cycles
negative-weight
that some
such
tree
is negative). Let T be a spanning
T
contains
whether
the
to
tree
determine
algorithm
vertices of G. You need to output only yes or no.
edge
7.75
weights
The following
are hints
with
graphs
weights
negative
to compute
an algorithm
for
but
no negative-weight
no
of the
may be negative
in which the sum of the
weights
cycles
of G
rooted at
only
shortest
single-source
v.
to all
paths in
shortest
cycles. The
a linear-time
Design
paths from
weighted
with an
starts
algorithm
other
exact algorithm.
detail the
more
in
terminates after
to improve
c. Suggest a way
not change
need
improvement
7.76
Let G = (V,
Design an
need
a. LetG = (V,
includes
7.79
(The
Algorithm
one (the
a good
\"common\"
tree.
initial
The
case.
whether
determine
path.
The algorithm
Transitive
Improved
outer
one)
If
is a simple path
path
to
Hamiltonian
7.80
selecting
the
the cycle
has to
include
the special
height
Hamiltonian
algorithm
algorithm
alternating
(red) neighbors.
without
problem
a connected
height.
v, with
b. Solvethis
Given
by
case, only
\342\200\242
let v be a
graph, and
Design a linear-time
cycle,which
find
7.78
be a directed
E)
or red.
cycle
the algorithm
the worst
\\
weighted directed graph such that some of the weights may be negative.
to determine whether
the graph contains a negative-weight
algorithm
to output only yes or no.
black
either
E) be a
efficient
cycle. You
7.77
O(|
chooses
that
vertex.
minimum
includes
a given
should run
Closure given
a column, the
loops.
and
The first
the third
one
row. Suppose that we exchange the first two loops such that the first
row
In other words, we simply
and the second one choosesa column.
the
is shown in
first
as
two lines in
the
exchange
algorithm
program,
WRONG Transitive Closure in Fig. 7.46. Show that this modification
does not work,
by
the transitive
closure is not computed.
giving an example for which
operates on the
one chooses a
7.81
chosen
was
for the transitive closure algorithm
(which
Exchanging the order of scanning the matrix
7.80) is desired for the following reason. If the matrix
attempted unsuccessfully in Exercise
is very large and thus cannot be stored
it from
in main memory, we need to access
that the matrix is stored
secondary memory. Assume
by rows such that each row occupies a
page. We want to minimize the number of pages that need to be fetched from secondary
(A) ;
WRONG_Transitive_Closure
Algorithm
begin
x :=
for
to n
for m := 1
do
to n
for
do
then
A [jc, m]
if
:= 1 to n
if
A [m,
do
y ] then
[jc, y ]
:= true
Closure.
Transitive
WRONG
loop scans the matrix by columns, then we need to bring all the rows to
On the other hand, if we exchange the first two loops and we find that
a certain entry
(jc, >>) is false, then there is no need to fetch the >>th row in the next step.
if the matrix is sparse (i.e., if it contains
Therefore,
only a few Is), fewer pages need to be
fetched. The algorithm in Fig. 7.46 is wrong as is, but it can be fixed.
the first
memory. If
look at each
a.
Show
7.82
if we
that,
closure
b.
column.
Show
this
times,
0(\\ogn)
algorithm
the
it computes
then
Let G = (V,
is sufficient
to run
the
only
algorithm
twice.
an undirected
namely,
graph that may have more than one
E
a
and | E | is the total number of
vertices.
is
in
this
case
multiset,
edge
E
+
v of degree 2 by replacing
an
V
O
to
delete
each
vertex
\\
(|
\\
|)
algorithm
edges. Design
the edges (w, v) and (v, w) by an edge (w, w), and to eliminate
multiple
copies of edges by
that removing
them with a single edge. (Note
multiple copies of an edge may
repacing
of degree
has to be removed, and removing a vertex
create a new vertex of degree2, which
which
must
be
removed
2 may create multiple
too.)
edges,
be a multigraph,
E)
a pair of
undirected
connected
if removal
graph G = (V, E) is callededge-biconnected
to
determine
a
linear-time
Design
algorithm
of any
one
whether
7.84
Given
should run
in
Let G = (V,
E)
Prove
of edgesthat
that
7.86
a connected
there exists a
7.85
transitive
correctly.
that, in fact, it
between
7.83
run
cycle in
be a connected
intersection
the
forms
undirected graph
of F
a spanning
and
\302\243),
with
the
tree of the
three edges,
b but
and
and
set of edges
does not
let T = (V,
of any
a, b,
contain
F) be a
biconnected
and
c\\
c.
find
whether
The algorithm
spanning
component
tree
of G.
is a set
component.
G = (V, E) is a biconnected
G' = (V, E') such that
graph
=
an undirected
biconnected extension;
graph G (V, E), find the minimum
the minimum
number of edges. (Hint:
Start
that
is, find a biconnected extension with
by
work
and
to
your
way up
general graphs.)
considering very simple graphs,
A biconnected
Given
E<^E'.
extension
of a graph
Algorithms
260
Graph
7.87
Suppose that
Show how
are given
you
undirected
an
biconnected
find the
to
with a list of
graph
without
components
all
resorting
articulation
the
to
points.
whole
the
running
DFS tree
and let T be a
Let G = (V, E) be a directed
graph,
of the edges of T with the edges of any strongly
Prove
of G.
intersection
the
that
connected component
of G form
a subtree
of7\\
7.89
A High
value
serves
linear-time
connected.
a. Prove
the
only
may
appear
an
G=(V,
undirected
connected
theorem:
following
if there
b. Design
is a
in G
circuit
more than
efficient
should find
such
A directed graph
that includes every
once in
that
to
algorithm
graph.
be oriented
can
E) is
G =(V,
edge
connected
strongly
if and
an edge
at least
once.
a given
strongly connected
(Note
that
circuit.)
find
a circuit
such
in
graph
\302\243).
BcV
with the
subset
E) is a minimum-size
is a vertex b in B such that there is a path of
a
length 0 or more from b to v. Prove the following two claims, and then use them to design
linear-time
algorithm to find a vertex basis in general directed graphs.
basis of a
A vertex
property
that,
a.
b.
An acyclic
directed
each
for
graph
vertex
not on
that is
vertex
a cycle
G=(V,
in V, there
vertex
indegree
basis, and
cannot
it
be in
is easy
to
any
find
vertex
basis.
it.
strongly
7.94
(based on
the edges of G
The algorithm
whether
strongly
7.93
algorithm
E) be
Let G=(V,
determine
7.92
It
v.
from
7.91
not
\"highest\"
DFS number
largest
7.90
the
an indication
as
only
Design a
computed
point to
actually
one
simple
given graph
7.95
Design
7.96
Prove the
Given
w is reachable
from v, there is
if, whenever
=(V, E) is calledunipathic
a
to determine whether
from v to w. Design an efficient
algorithm
=
G (V, E) is unipathic.
Solve the problem first for acyclic graphs.)
(Hint:
graph G
directed
only
components graph.)
path
a linear-time
alternating-paths
two
matching
M\\
theorem
and M2,
set of all
edges
a maximum
directly
study
the
that appear
matching
in
a tree.
without
properties
the use of
of the
(Hint:
between
Creative Exercises
7.97
G be an
Let
a.
the
Prove
undirected bipartite
graph,
M be an
and let
arbitrary
the
b.
edges
a maximum
matching
following theorem: There exists
M covers.
(A vertex is coveredby a matching M
of M.)
the
Convert
in part a to
proof
an
for finding
algorithm
in G.
matching
vertices that
G that
in
if
261
it is
such a maximum
when
matching
7.98
Prove
the
that
7.99
time of
running
improved algorithm
to
and
Hopcroft
7.10) is O
Section
in
Karp
((m +n)w)
in
bipartite
matching
the worst
case.
(the
algorithm
Suppose
want
we
that
a.
the
Find
a nonmonogamous
find
for disjoint
of
instead
are
we
constraints.)
or maximality
minimality
error
by exhibiting a
by
pointing
out the
wrong argument
and
counterexample.
solve
b.
Design
an efficient
(and correct)algorithm
for this
problem.
7.101
following
aniVxAf
Consider
Prove, by using
and one arbitrary
size 2 x 1).
7.102 Prove
the
the
board
bottleneck
n vertices
board of
alternating
black
and white
theorem, that if
then the rest of the
the alternating-path
white
square,
theorem
squares (such as
one arbitrary
a chess
one removes
board
can be covered
a Hamiltonian
cycle
and
in
by
square
dominoes
a graph
any
board).
black
defined
(of
by
two neighboring
the
differs
E)
shortest
from the
be a connected
undirected graph.
sequence of trees
previous one by
Given
T0,Tl9...,Tk,such
an addition
of one
edge
262
7.104
Graph Algorithms
that a round-robin tournament
is played among n
other players. There are no draws,
Assume
is not
in general
possible
to sort
each player
results of all games
That is,
players.
and the
since A may
the players,
plays
are
B, B may
We are
beat
C may
C, and
interested in
7.105 Given
a \"weak\"
0<d,
<d2
we have dj < dx
integers
2</<\302\253,
vertices of
in
implies,
*
A (in
beat
<
\342\200\242
\342\200\242
\342\200\242
^dn,
degrees exactly
+ dj_x.
+d2+
' ''
dn is
the
for
every
with
multigraph
which
algorithm
exists.
always
multigraph
even, and,
an undirected
Construct
...,dn.
d\\,d2,
that such a
particular,
such that dx
7.107
an undirected
graph G =(V, E) is a set of edgessuch that each vertex in
to
one
is
incident
at
least
an efficient algorithm to find
edge from the set. Design
graph
minimum-size
edge cover for a given bipartite graph.
cover of
An edge
the
a
7.108
7.109
cover
vertex
in
the graph
in
Chapter
11.)
Let G =(V, E) be a
each vertex is equal
efficient
Design
an
graph.
(Hint:
to
the
associated with
degree of that vertex.
weights
minimum-weight
7.110
with
tree
Find
E) be an
to
algorithm
a relationship
find
set
in a given bipartite
graph.
11.) (Hint: Find a relationship
An
an efficient
Design
(Independent
in the
cuts
minimum
the
of
weight
to
the
find
weight.
for
a given
bipartite
graph.)
set in G
independent
that
an algorithm
vertex cover
a minimum-size
to minimum
undirected graph.
are connected.
Design
cover with
vertex
such
vertices
the
is a set of vertices no
two
independent
algorithm to find a maximum-size
sets in general graphs are discussedin Chapter
to Exercise 7.110.)
7.112 Design
an algorithm
to find a maximal
in a given
independent set (see Exercise7.111)
undirected graph G = (V, E). The set need not have the maximum size over all independent
sets. It is only
be extended by the
to be maximal in the sense that it cannot
required
addition of more vertices to it and still remain independent.
E)
be a tree
algorithm
such
that
to find an
each
vertex
v has
independent set
in
an
weight w(v).
Exercise 7.111) with
associate
G (see
Design a
maximum
weight.
7.115 Design
an
to determine
algorithm
vertices U,
U is a
that
such
cover
vertex
find such a
should
undirected
an
whether
minimum
set if
and a
graph
E)
G=(V,
maximum
has
a set
of
set at the
independent
it exists.
algorithm to find
independent set in
a maximum
G.
7.117
An undirected graph
7.118
a. Design
an
Can
b.
7.119 a.
algorithm
your algorithm
Design
square
as a subgraph.
triangle
find
all the
in G?
to determine whether
an algorithm
a given undirected graph G = (V, E) contains
a
as a subgraph (i.e., a cycle of length
should
4). The running time of the algorithm
beO(|V|3).
b.
Improve
algorithm
your
the
to
run
in time
matrix
adjacency
O (|
V |
\342\200\242
| E |).
representation
or
the
list
adjacency
representation,
7.121 Let
each
T be a rooted
vertex,
finds
all squares
time in the
that
are
is a weight
There
tree, not necessarily binary.
of a vertex is greater
than the weight of the
with the minimal
weights satisfy the heap condition
the
be
associated
vertex's
designated
same as its
as either a
regular
vertex
or a
weight
with
parent
on
top).
cost of a
\342\200\224
however
their cost
vertices get discounts,
is
vertex.
ancestor
that
a
Thus,
pivot
weight
selecting a vertex as a pivot vertex may increase its cost, but it will also decrease the costs
of some of its descendants.
vertices.
There is no limit on the number of pivot
Design an
or a pivot vertex, such
efficient algorithm to designate
every vertex as either a regular vertex
is minimized.
that the total cost of all vertices
pivot
vertex
is the
\342\200\242
| E |).
the weight
that
of a given
subgraphs
directed
such
that
running
is their
minus
binary
of
the
closest
want
to embed
T in
point (i.e., a
point with integral coordinates), adjacent vertices are connected
by straight line segments,
and
no two line segments intersect.
Embedding
graphs in the plane in this way is an
and especially VLSI design.
in
Our objective
important
problem in integrated chip design
the area enclosing the layout.
this exercise is to minimize
We define this area to be the
lattice
minimum-area rectangle along
that contains the
points (which are not occupied)
with k vertices would be enclosedin a rectangle
of
So, for example, a straight chain
layout.
area is of
area 2(k + 1). It is clear that for any graph with n vertices the minimal
possible
the
Regular
weight.
the weight
size Cl(n).
lattice
264
Algorithms
Graph
a. Describea
layout
completebinary
new
shared
root.
layout of a tree
b.
Design
obtained
T that
for
tree
consists
Assume
of height
an algorithm
in part a.
that
h.)
to
compute,
for
each
vertex
in
7,
its coordinates
in
the
layout
CHAPTER
ALGORITHMS
GEOMETRIC
paint
objects
Pablo
8.1
as I
think
Picasso
them,
not as I
see them.
(1881-1973)
Introduction
role
of computer science,
areas
VLSI
and
design, robotics,
computer-aided
design,
including
computer
graphics,
circles
or even millions of points, lines, squares,and
There may be thousands
databases.
in a computer-generated
picture; a robot may have to make thousands of moves;a design
involve the
All
these
of
a computer
problems
chip may involve millions of items.
of geometric
manipulation
objects. Since the size of the input for these problemsmay be
for them.
quite large, it is essential to developefficient
algorithms
areas in which
There are two somewhat
arise;
algorithms
geometric
separate
of
them
deals
are
both
One
called
computational geometry.
mainly
unfortunately,
they
with
deals
discrete
with
of
the
other
one
continuous
mainly
aspects
geometric objects;
similar
is not strong, and there are many
of geometric objects. The distinction
properties
will
be
on
and
Our
discrete
problems
computational geometry. In
techniques.
emphasis
this
As in other chapters, the
we discuss several basic geometric
chapter,
algorithms.
that
of this chapter is necessarily limited.
We include
some of the basic algorithms
scope
in the design of more complicated algorithms,
and
that
blocks
appear as building
illustrate interesting techniques. We will limit the discussion to two-dimensional objects.
in this chapters are points,
The
and
line segments,
lines,
objects
appearing
these objects and compute certain propertiesof
manipulate
polygons. The algorithms
and a discussion of data structures
definitions
We start with
them.
basic
used to represent
A point p is representedas a pair of coordinates
the
different
(jc, y) (we assume
objects.
this chapter).
A line is represented by a pair of
a fixed coordinate system
throughout
Geometrical
algorithms
play
an important
in
many
266
Geometric
points
p and
-p-q-.
line
by
p-q.
can
(which
segment
we assumethat
A
Algorithms
the
are the
points
P is a
path
by
points
on the
a pair
of points
segment's endpoints,
sequence
of
we
and
We
denoted
it is
and
and
q, but
denote
the
the
and
Pi,P2\302\273\342\200\224\302\273P/i\302\273
points
them.
line),
by
in this
case
line segment
line
segments
the line
segments
connecting
Pk-\\~Pk
P\\-Pi- Pi~P?>i\342\200\242-\">
last point is the same
as its first point. A
in a path
edges. A closed path is a path whose
The
are also called the
closed
points defining the polygon
path is also called a polygon.
is a polygon
with
three
For example,
a triangle
vertices.
A
vertices of the polygon.
a
set
of
the
rather
than
as
because
as
a
order
is
in
points
sequence
represented
polygon
call
sometimes
will
which
p!, p2,...,
that connecting
such
pn
We assume
that
\\
with
pn results
is familiar
reader
the
in
basic
with
a convex
polygon.
analytic
geometry.
For example,
we will
given
discuss
in Section
issue
this
One
8.3.)
perform a
of many
characteristic
inconvenient
on
computation
two
example,
two
geometric
lines
in
the
algorithms
plane
is the
usually
intersect
given
We
Whether
Determining
start
with
a simple
This
problem
the
Is Inside a
a Point
manner,
of
geometric
essential
for
Polygon
problem.
The Problem
whether
at
one
are parallel
lines
More complicated
8.2
of
existence
point
may seem
Given
a simple
polygon
trivial
at
first,
and
a point
q, determine
polygon.
but when
8.2
as is
considered,
a problem
the
by
in Fig.
case
hand
8.1,
is always
first
a Point
Whether
Determining
the
is definitely
problem
a good idea.
The
Is Inside
not simple.
intuitive
first
a Polygon
Trying
approach
267
to solve
is to
try
somehow
this
case,
pictures.
we clearly
When
the
as a
given
is different.
task
is
input
we see
intersections)
the
with
polygon
from the
Figure
point
8.1
of
list
For example,
Determining
as is
coordinates,
whether
find
not
a point
when
a good
an
easy
we
the
usually
do the
work
is inside a
polygon.
case
by
in a
hand,
with
few
when
the
(P, q);
Point_in_Polygon_l
Algorithm
Input: P (a simple
en), and q
Inside (a Boolean
e!, e2,...,
Output:
with
polygon
{first attempt}
and
.,/?\342\200\236,
verticesP\\,Pi,
edges
(a point).
that
variable
is set to
if q
true
is inside
and
false
otherwise).
begin
Pick an
q -s ;
segment
0 ;
:=
count
line
the polygon ;
s outside
point
arbitrary
LetL be the
for all
nor
vertex
line L overlapping
the
with
intersection
is not
at a
the text}
e{; see
increment count;
if
count
else
is odd
Inside
:=
then Inside
true
:=false
end
Figure 8.2
8.3
Figure
is stored
polygon
to
do
visually,
of
polygon
Fig.
Point_in_Polygon_I.
Algorithm
whether
series of coordinates.
but again, it
8.1 is given
at | 368 308 | .)
8.4. Clearly,
is
not
as a
of
when
coordinates
of intersections is easy
coordinates
are given. The
the number
Counting
as easy
list
is inside a polygon.
a point
only
the
in Fig.
point
is
now by looking
to solve the problem
This
work is computing all the intersections.
work can be substantially
for
if the line q-s is parallel to one of the axes \342\200\224
simplified
the vertical axis. The number
of intersections
with this special line may be
example,
much more than that with the optimal line, but we do not need to find the optimal
line
\342\200\224
more difficult
a much
(which is actually
see Exercise
8.3), and computing
problem
The modified
each intersection is much easier.
algorithm is presented in Fig. 8.5.
centered
only
at
Fig.
The
the
reader
bulk
is encouraged
of the
8.2
Determining
Whether a Point
Is Inside
a Polygon
269
320.00368.00
380.00
384.00 | 288.00 384.00 | 288.00380.00| 308.00
| 320.00
|
376.00 | 280.00 376.00| 280.00392.00
392.00 | 332.00 364.00 |
| 332.00
364.00364.00| 364.00 352.00 | 256.00 352.00 | 256.00 404.00| 224.00404.00 |
308.00
288.00
332.00 | 352.00 332.00 | 352.00288.00| 224.00
| 224.00 312.00 |
320.00 312.00| 320.00300.00 | 256.00 300.00 | 256.00 296.00 | 328.00296.00|
280.00
328.00
320.00
340.00 |
| 384.00
| 208.00 320.00 | 208.00 280.00| 384.00
240.00
396.00
248.00
248.00
416.00
240.00 340.00 |
396.00
348.00 |
348.00|
|
|
416.00272.00 | 320.00 272.00 | 320.00 256.00 | 448.00256.00|448.00 320.00 |
224.00 | 256.00 224.00 |
432.00 320.00 | 432.00 340.00| 452.00340.00
| 452.00
256.00244.00| 320.00 244.00 | 320.00 248.00 | 248.00 248.00| 248.00216.00|
256.00
224.00
216.00
256.00 |
| 288.00
| 224.00 240.00 | 232.00 240.00| 232.00
272.00
288.00 264.00 | 224.00264.00| 224.00
| 192.00 272.00 | 192.00 416.00|
428.00416.00
| 428.00 384.00 | 416.00 384.00 | 416.00 400.00| 424.00400.00 |
384.00
424.00
408.00 | 384.00 408.00 | 384.00384.00| 400.00
| 400.00 396.00 |
388.00 396.00| 388.00404.00 | 408.00 404.00 | 408.00 372.00 | 352.00372.00|
404.00
352.00
| 264.00 404.00 | 264.00 368.00|
224.00
P (a
simple
en), and
e!, e2,...,
coordinates.
{ secondattempt}
p {, p2,...,
y0) (a point).
variable
that is set to true
vertices
with
polygon
(P, q);
Point_in_Polygon_2
Algorithm
Input:
of Fig.
polygon
and edges
p\342\200\236,
= (x0,
if q
is inside
and
false
otherwise).
begin
count :=
0;
for all
polygon do
intersects e{ then
line x=x0
that the intersection
assume
edges
if the
{ We
e{
of the
Let y{
be the y coordinates
the linex=x0 and et ;
ifyi
if count
else
is odd
Inside
then Inside
is not
at a
vertex nor
is the
with et }
overlapping
linex=x0
of the
intersectionis below
:=
between
intersection
q }
true
:=false
end
Figure 8.5
to
the
Algorithm
determine
polygon
Point_in_Polygon_2.
whether
given
in
Fig.
the
8.4.
point
with
We count
the
coordinates
number
of
Geometric
270
intersections with
the edges,
a line
at all
280)-(384,
(208,
segment
for those
check,
and
x coordinate
the
Algorithms
280);
(416,272)-(320,272);
(320,256)-(448,256);
Hence, the
and
Comments
hand
by
such
other
is O
algorithm
In
(or eye)
an
The algorithm
performs
this
many
is not
operations
two
between
n is
(where
the
time. Hence,the
line
size
total
(n).
a simple
cases,
calculations is
approach
to perform an intersection
computes n such intersections
time
constant
takes
It
plane.
polygon),
time of
running
the polygon.
is outside
point
Complexity
segments in the
of the
224).
224)-(256,
(452,
not
approach
efficient
only simple,
but
for
large inputs.
efficient.
also
In
obtained
a solution
from
originating
some
Starting
however,
cases,
an
with
\"easy-tothat
can be
a good
idea. There are several observations
visualize\" method is always
we could
In this case, by looking
at the picture, we observed that
achieved this way.
to the outside, disregarding
some path from the point
solve the problem by following
that led to the algorithm.
else. This was really
the main observation
everything
8.3
A
set
of
the
of points
points.
in
the
In this
Polygons
Simple
Constructing
on the order
plane defines many different polygons, each depending
a simple
section, we concentrate on finding
polygon defined by a set
of points.
The Problem
simple
There are
a set
Given
of n points
in
the
plane,
connect
them
in
closed path.
methods
several
corresponding to
to
construct
we would
simple polygons.
We
present
if we
method
had to solve it
of C by a
area
probably approach
a large circle C that contains
all the points. Scan the
line originating
Let's
assume for now that
from the center of C (seeFig. 8.6).
rotating
the rotating line never touchesmore
that,
than
one point from the set at a time. It seems
if we connect the points
in the scan, we get a simple
in the order they
are encountered
Let's try to prove this claim. Denote the points,
as they appear in the order
polygon.
the
is chosen arbitrarily). For
first
line,
imposed by
by P\\,p2* >,pn (the
rotating
point
is included
all /, 1 < / <n, the edge p,\342\200\224p,-+i
in a distinct
of
the circle; hence, it does
region
not intersect with
other
this
is
not
to
edge.
However,
any
prove that the resulting
enough
by
hand.
Consider
the
way
this
problem
8.3
Figure8.6
Scanning
Constructing
the
Simple Polygons
271
points.
is simple \342\200\224
in fact, it may not be. The angle
between
some p, and p/+1 may be
to the edge P/-p,+i
in
which
case
the
180
degrees,
region
corresponding
greater
consistsof more than one-half of the circle. Thus, the edge Pj-pi+i cuts into the other
other edges. (To see that it may, we can consider a circle
regions, and it may intersect
that
is centered
somewhere
of
outside of the circle of Fig. 8.6.) This is a good example
in geometrical problems. We have to be
the kind of ''special cases\" that
often
arise
careful to make sure that all cases are considered. (Of course, we must do that for any
kind
is more prevalent in geometric
of algorithm, but this problem
algorithms.)
For example, we can take any three
We can overcome this obstacle
quite easily.
formed
points from the set and choose, as a center of the circle, a point inside the triangle
a segment
that the circle does not contain
by these three points. This choice will ensure
of more than 180 degrees without any points from the set. Another
which is the
solution,
one we will use, is to choose one of the points from the set as the center of the circle. We
will
if
the point z with
the largest
x coordinate (and the smallest
choose
y coordinate,
there is more than one point with the largest x coordinate). We now use basically the
same algorithm.
We sort the points according to their
in the circle centered at z.
position
These
can
be
a fixed line (e.g., the x
the
between
positions
angles
computed by sorting
lines
z
and
the
from
If
to the other points.
two (or more) points
the same
have
axis)
z to the
from z. We then
connect
angle, they are further sorted according to their distance
with
the smallest angle and to the point
with the largest angle, and connect
the other
point
lie to the left of z, the
Since all other points
bad
case we mentioned
points in order.
is still one more special case that
earlier cannot occur. (There
occurs
when all the points
the points
lie on a line; in that case,
will have overlapping edges.)
any polygon through
in Fig. 8.6 is given in Fig.
obtained by this method for the points
The
simple
polygon
polygon
than
8.7.
Geometric
272
Algorithms
Figure 8.7
we
do
this
We can improve
not have to compute
It is
the
computing
same
exact
to
share
the
angles
compute
-iPti
Output: P (a simple
(points
the
in
(p {,p2,..., pn
in the plane).
whose
polygon
have the
two points
distances!
is presented
algorithm
Simple_Polygon
Pi,P2\302\273
when
distances
of
the square
Algorithm
polygon.
But the
argument,
sufficient
Input:
method
the
a simple
Constructing
there
Therefore,
Fig
same
slope.
is no need to
8.8.
)\302\273'
vertices
are p
\342\200\242\342\200\242\342\200\242\302\273
Pn m some order).
{,p2\302\273
begin
for i := 2 to
n do
{it
P is
the angle
is sometimes
a,
the
between
line -p
more desirable to
the
by
list of
points
in
sorted
order
end
Figure 8.8
Complexity
requires
0(n
logn) time.
Algorithm
of
this
Simple_Polygon.
algorithm
is dominated
by the
sorting,
which
8.4
Hulls
Convex
8.4
set of points
of a
hull
convex
The
273
Hulls
Convex
listed
be
The Problem
smallest convexpolygon
enclosing
as
a
represented
regular
polygon,
cyclic order.
as the
is defined
convex
the
like
in
hull of
convex
the
Compute
all
be
to
hull
given
in the
points
plane.
Dealingwith
serves,
hull
convex
The
belongs
to the
as
and
many
consequently,
hull
of
if it
as all
numerous
the
the
usual,
we
hull.
as
vertices.
points
been
have
region
of the
The convex
polygons.
arbitrary
handling
\"convenient\"
hull
convex
algorithms
start with a
than
smallest
the
is a vertex
A Straightforward
8.4.1
As
sense, as
vertices
points.
is easier
polygons
in some
set.
can
encompassing
We
contain
Convex
developed
a set of
uses,
and
Approach
inductive
straightforward
approach.
We can
easily
find
the
hull of
hull of three points. We assumethat we know how to compute the convex
< n points, and we try to find the convex hull of n points. How can the Arth point change
extra
the convex hull formed
are two cases: Either the
by the first n -1 points? There
the
is
outside
in
which
the
hull
or
case
is
is
inside
the
convex
hull,
unchanged,
point
point
to reach that point (see Fig. 8.9). So, we
the hull, in which case the hull is \"stretched\"
whether
a point is inside the hull,
need to solve two subproblems. We have to determine
convex
\\pi
Pi>\\
p5
Figure 8.9
-_
Stretching
Pa
a convex
polygon.
Geometric
274
Algorithms
to be able to
are
point is outside of it. These
problems
with
the
now
inductive
straightforward
not easy.
enough
experience
by
is to choose a
right away. The first improvement
approach to try some improvements
is
to
choose
a
rather
an
It
inside
than
one.
the
nth
tempting
point
arbitrary
special point
of
some
will
to
the
hull.
work
be
extend
must
But,
course,
points
hull, so no
required
in some cases all points
to the hull. Another
and
to the hull,
belong
possible
belong
\342\200\224
is an extreme point
which worked well for the previous
choice,
problem,
namely,
some sort of maximal or minimal
point.
with the maximal x coordinate (and
the
minimal
We choose again the point
y
are
with
the
same
maximal
x
Denote
there
several
if
this
coordinate).
coordinate,
points
to be a vertex
of the convex
hull. Thus, the
q is guaranteed
by q. It is clear that
point
to
the
to
We
first
is how
hull
include q.
need
to find the
modify (stretch)
problem
only
vertices of the old hull that are now inside the new hull (p3 and p4 in Fig. 8.9) and to
remove them; then, we must insert the new point as a new vertex between two existing
is a line that
line of a convex polygon
vertices (p2 and p5 in Fig. 8.9). A supporting
at exactly
one vertex of the polygon.
The polygon
thus lies
intersects the polygon
of
a
one
side
line.
Consider
now
the
lines
on
and
-q-pi~
supporting
supporting
entirely
two
vertices
of
the
have
lines
to
(see Fig. 8.9). Usually,
q which
only
-q-p$polygon
lines. (We will ignore
the special case of two
or more
are supporting
points that are on
same
line with q.) The polygon
lies
the two supporting
the
between
and that is
lines,
lines have the maximal
and
exactly the way we want to modify it. The supporting
minimal angles, with,
in
the
x
all
other
lines
from
the
axis,
say,
among
points
polygon to
To
find
these
two
we
to
the
need
consider
lines
from
to
all
the
vertices
vertices,
q.
q
t0
the
the
maximal
and
to
and
minimal
Pi\302\273P2\302\273
(see also
\342\200\242\342\200\242\342\200\242\302\273P/i\302\273
compute
angles,
pick
Exercise 8.4). Once the identity
of the two extreme verticesis known,
the modified
hull
can be constructed. (There
are several
other approaches to modifying
the hull, and this is
not
the best one; we chose it for its simplicity.)
We omit the details
necessarily
we have
and
We
this algorithm
concerning
For each
Complexity
find
the
the
work
solution
running
running
maximal
involved
of the
the hull
stretch
if
the
have
because we will
present
a faster
and minimal
angles,
in processing
recurrence
the
is
to delete
point is O
= T(nT(n)
fah
relation
the
and
0(n2).
other
algorithm
anglesto
and insert
(/:), and
\\)
+ 0(n)
The algorithm
we
all
points
have
shortly.
already
is 0(n2).
seen
that
Therefore,
but
the
the
the
operations.
Gift Wrapping
8.4.2
point by point, we
algorithm? When we extend the polygon
to the
be internal
that
convex polygons containing
may
points
final convex hull.
of
Can
we avoid doing that?
convex hulls
of considering
Instead
subsets
of the set of points,
we can start with
the hull directly.
the whole
set and build
That is, we can start with an extreme point (which
find its neighbors
must
be on the hull),
in the
hull by finding the supporting
and continue from these neighbors in the same
lines,
as the gift wrapping
is known
for obvious reasons. We
way. This algorithm
algorithm
How
can
spend a lot
we improve this
of time building
vertex of
one
start
with
after
neighbor.
The
the
wrap the
and
\"gift,\"
is given
algorithm
in
Fig
around
hull
be modified
It can
8.10.
275
Hulls
Convex
8.4
dimensions as well.
The
induction
Induction
With
this
hull.
Instead
the following
of
application
straightforward
(on k):
hypothesis
convex
is a
algorithm
gift-wrapping
hypothesis:
path of
hypothesis,
of
finding
length
of n points in
is part of the convex
a set
Given
that
k<n
hull of
we can
this set.
rather
than on
the
a path
sets, we
plane,
a part
find
find
the
extending
of the
final
convex
hull.
\342\200\242
\342\200\242,
Gift_W'rapping (p x, p 2, \342\200\242
pn) ;
in
set
of
the plane).
(a
Input:
p!,p2\302\273 \342\200\242\342\200\242\342\200\242\302\273P/i
points
P
hull
of
convex
(the
Output:
P\\,pi,
..,p\342\200\236).
Algorithm
begin
set P
to be the
be the
(and the
Let p
empty
set
the
with
x coordinate
largest
if there
y coordinate,
smallest
the same
with
set;
in the
point
are severalpoints
largest x coordinate) ;
AddptoP ;
L be
Let
while P
the line
is not
is parallel to
p which
containing
the
x axis
do
complete
and L
be the point such that the angle betweenthe line -p-qall points ;
(in counterclockwisefashion) is minimal
among
let q
add qtoP;
L := line
-p-q-
p:=q
end
Gift
Wrapping.
and
angles
8.4.3
We
now
algorithm
Grahams
maximum
is
algorithm
Scan
show
an algorithm
starts
by ordering
the
to compute
the
convex
hull
in time
to angles, similarly
according
8.3.
Section
Let p x be the point
if
there
are several
other
O{n logn).
to
points
with
points
The
the construction
the
with
maximal
the same
Geometric Algorithms
276
coordinate).
and
the points
sort
Induction
hypothesis:
to algorithm
according
the first
among
path
Given a set
of
in the
points
plane, ordered
can
find
a convex
encloses
polygon
The caseof
points
then
and
can
pk+l
are
by removing
becausethe
points
have
now
be added to
the
obtained
path
to extend
than
we
We
the convex
the lines
between
angle
is less
polygon),
convex),
obtained
the
Denote
...,qm.
by P=q\\,q2,
Consider
angle
k = 1 is trivial.
and
existing
qm
the
to
hypothesis
-qm-pk+\\-
is measured
from
&+1
Fig. 8.11).
(see
the
inside
with
first
points.
If the
of
the
it is still
convex polygon
Otherwise, we claim
from
to
and
This is so
/>, addingpk+l
P,
connectingpk+l
top{.
ordered according to their angles.
The line -p\\-pk+\\- is on the
done.
were
from the
(inductively)
Figure
8.11
Graham's
scan.
above, qm
can
qm
Although
8.11shows
qm_x
180
be removed
as well.
For example,
points may have to be eliminated
the
modified
We
defined
must
continue
by
polygon
path.
last two edges of the
until we find two that form an angle of less than
path
The path is then convex,
the hypothesis
has been extendedto k + 1 points,
the
degrees.
be inside
the
detailed
is presented
algorithm
Graham's_Scan
Algorithm
Input:
other
that
clearly
now
may
checking
of the
side
\"left\"
277
(p \\,
p2,...,
in
8.12.
Fig
pn);
(a set
Pi,P2\302\273 \342\200\224<>Pn
Output:
..,pw).
begin
Let p
the point
be
(and
the same
with
Usealgorithm
order
largest x coordinate)
; let
the
to arrange
order
be P\\,p2,...,
px
;
p\342\200\236
<7i :=P\\
<72
:=Pi
<73
'=P3
{P
set with
the
Simple_Polygon
in sorted
m :=
in
y coordinate
smallest
consists
initially
ofp\\,p2,
andp3
3;
fork :=4
while
m
m :=
qm
ton
do
is >
180 degrees
do
+ 1 ;
\342\200\242\"=
Pk
end
Figure
8.12
Algorithm
Graham
s Scan.
is dominated
algorithm
by the sorting. All the
in
Each
the
set
is
considered
O(n)
point
exactly once in
steps require only
the induction
At
that
the
is
as
to
the
convex
added
time,
always
step
Pk+\\.
point
path.
later (possibly more than once) to verify its inclusion
The same point
will be considered
in the current
We call this phase a backwardtest. The
convex path.
number
of points
Complexity
other
time.
involved in a
point and the
backward
that
current
So, we
Overall,
is thus
278
Geometric Algorithms
8.5
Closest
Pair
Supposethat we are given the locations of n objects and we want to check that no two of
for example, to parts
the objects are too close to each other. The objectsmay correspond,
in
a
In this section, we
to
stars
or
to
in a computer
galaxy,
irrigation
systems.
chip,
is
which
of
this
an
of
a
a
variation
discuss
problem,
example
large set of proximity
problems.
of n
Given a set
Problem
The
in the
points
plane,
a pair
find
of
closest points.
Other proximity
problems
closest points to
include
or the k
A
A straightforward
minimal one.
This solution
comparisons.
The
solving
distance,
then
for
and
points,
it
to
point
the
additional
efficient
In fact,
these two
for
algorithm
A Divide-and-Conquer
would
of
to
point
of distance
(7(2)=
other
n -1
is the
are identical.
We
must
want
be
the
satisfies
we have already
and
1),
minimum
points
computations T(n)
solutions
straightforward
all
if
However,
point.
case
n-\\
the
the
- l)/2- 1
removing a
by
proceed
the extra
considering
take
to
and
and n(n
computations
induction
the solution
from
obtained
distance
using
n-\\,
= 0(n2).
a more
solution
recurrence
T(n)
requires
- l)/2
all pairs
between
distances
the
n (n
straightforward
the problem
information
check
is to
solution
the only
checked.
in the
to a
Approach
Straightforward
point,
finding,
seen
that
to find
n.
large
Algorithm
induction
of
considering
hypothesis
one point
at
a time,
we divide
can
the
set
of reducing
with
problems
into
8.5 ClosestPair
279
d2
\342\200\242
+
choosethis
For
subsets,
the
Identifying
If the
algorithm.
set of
of
work
needs to be performed
simplicity, we concentrate on
The sorting
problems.
points.
the
to minimize
division
8.13
Figure
n points,
\\
and
actual
and assume
P2,
two
two
that
points,
n is
as described
a power
above.
We
of 2.
find
the
will
points
then we
of
the
smaller
once.
only
finding
closest
the solutions
combining
only
find
We
minimal
be
first
straightforward
distance
their
distance among
divide
directly.
P into two
distance
P2 be d2, and
the closest
in
each
the
from
the
Let P
be a
equal-sized
subset
by
in Px be d\\, and in
without
induction. Let the minimal
distance
assume,
loss of generality,
that
We need to find the closest
distance in the whole set;
d\\<d2.
in
there
is a point in Px with a distance <d\\ to a point
namely, we have to see whether
a
of
that
it
is
to
that
lie
in
we
notice
sufficient
consider
the
First,
P2.
points
strip
only
width 2d i centered
around
the vertical separator of the two subsets (see Fig. 8.13). No
other point can be of distance less than d { from points in the other subset. Using
this
in
worst
can
from
the
we
eliminate
observation,
consideration, but,
usually
many points
can still reside in the strip,
and we cannot \"afford\"
to use the
case, all the points
straightforward
algorithm
for
them.
less obvious observation is that, for any point p in the strips, there is only
a small
number of points
on the other side whose distanceto p can be smaller than
d\\.
This is so because all the points
in each strip are at least d \\ apart.
If p is a point
in the
on the other side with
a y coordinate
strip with y coordinate
ypj then only the points
yq
such
that \\yp-yq \\ <d\\
need to be considered. There could
be at most six such points
on
one side of the strip (see Fig. 8.14 for the worst
As a result,
if we sort all points in
case).
and scan the points
in order, we need to check
the strip according
to their y coordinates,
of its neighbors in the order (instead of against
number
each
against
only a Constant
point
of the algorithm is given in Fig 8.15.
all n - 1 points). A sketch
Another
280
Geometric
Algorithms
\302\253
\342\226\272
di
The
8.14
Figure
Closest_Pair_l
Algorithm
of n points
Output:
of six
case
>'
(P\\,P2*\342\200\224*Pn)
(a set
Pi,P2\302\273 \342\200\224<>Pn
Input:
worst
plane).
closest
two
apart.
(firstattempt}
the
in
points d x
in
the
part
points
set).
begin
Sort
the
points
{thissorting
set into
Recursively, compute
Let d be the minimal
Eliminate
Scan
if any
of these
lie farther
to its five
distances
beginning
in
each
;
separation line ;
d apart
from the
to
y coordinates
than
according
points in the y
remaining
point
minimal
points
remaining
each
the
the
parts ;
distance
equal-sized
of the
that
points
the
two
coordinates ;
only once at
is done
Divide the
Sort the
to their x
according
and
order
neighbors ;
is less than d
their
of
then
update d
end
Complexity
sorting
is done
It
takes
0(n
only once.
Closest_P
We
then
solve
two
air
according to
the
J.
x coordinates,
of
subproblems
It then
steps.
but this
the
Intersections of Horizontal
8.6
T(n) = IT(nil)
T(n) =
0(n
An
0(n
The
key
0(n log
know
to
how
other
Induction
find
to include
but we
algorithm,
still
takes
so
we
are
the
induction
we have to spend
we
Although
Can we
the closest-pair
too long.
solving
hypothesis
for
the
set of < n
to
know
how
sorted according to
the set
output
plane, we
in the
points
the
y coordinates.
to
according
step
is
relation
recurrence
sorting.
Given a
hypothesis:
this
a quadratic
than
words,
the closest
points
of
solution
The reason
hypothesis.
the y coordinates.
induction
the
strengthen
sorting
problem
closest-pair
the
better
in the combining
the sorting
solve
problem? In
that
verify
Algorithm
here is to
a?) time
1.
that.
than
log n)
idea
T(2) =
is asymptotically
This
log2a?).
better
to do
want
logrt),
the reader to
it to
leave
We
+ 0(n
281
Line Segments
Vertical
and
the minimal
y coordinates.
their
is to
hypothesis
find
to
sort
Hence,
of n
set
the
points
distance
the
thing
only
the
when
if
two
the
points
that
subsets
are sorted
in
each
we need to do to extend
(of size nil) are already
is exactly
of this
sorting
advantage
mergesort (Section 6.3.2). The main
\342\200\224
is
that
we
not
time
we
the
solutions
we
do
have
to
sort
combine
every
only
approach
have to merge. Since merging
can
the recurrence relation
be done in O(n)
steps,
becomes
that T(n) = O(n logn). The
T{n) = 2T{nl2) + 0{n\\ 7(2)= 1, which
implies
sorted.
But, this
revised
algorithm
8.6
is given
in
Fig
8.16.
of Horizontal
Intersections
Segments
Intersection
are
problems
applications.
objects, and
we
common
in
are sometimes
We
are
computational
interested
interested only
sometimes
in
geometry,
the
computing
in
whether
detecting
have many
of several
the intersection is
and they
intersection
easier,
nonempty. Detection problems are usually
although not always
substantially
easier.
In this section, we present one intersection
that illustrates an important
problem
of computational geometry. The same
can be applied to other
technique
technique
intersection problems (and
to other
are given as
problems as well), some of which
exercises.
The Problem
segments
in the
plane,
a set
Given
find
all
of n horizontal
the intersections
and
vertical
among them.
line
Closest_PairJl
Algorithm
Input: P\\,P2*
Output: d (the
(a set
\342\200\224<>Pn
the two
between
distance
version }
{&n
improved
Pn)\302\273'
(P\\,p2^ \342\200\242\342\200\242\342\200\242>
in the plane).
of A2 points
closest points
in the
set).
begin
Sort the
to their
according
points
{thissorting
is done
set into
Recursively do the
two
Divide the
Merge
Notice
that
supply
the
in
sorted
two
the
we must
beginning
parts ;
equal-sized
distance
each
each
in
part;
to
according
part
y coordinates
their
lists into
to
set sorted
whole
the
following:
the minimal
compute
x coordinates;
only once at
need
we
to
of the recursion }
level
next
the
distances
d be the minimal of the minimal
;
Eliminate points that lie further than d apart from the separation
Scan the points in the y order and compute the distances of each
Let
to its
point
if any
of these
update
five neighbors ;
distances is less than
line ;
d then
end
This problem is
contain hundreds
for
important,
of
problem
Finding
is a
lines
in
Fig.
in the
example,
of \"wires,\"
thousands
is
and
also
A circuit
design of VLSI circuits.
the designer
has to make sure that
is usually
simply
lines
more complicated,
For simplicity,
vertical.)
section.
this
in
may
there
of hidden-line
context
the
in
important
problem
or
air J..
Closest_P
when
there
An example
because
is no
of
the
8.17.
all intersections
simple problem,
which
the
vertical
as an exercise.
lines
or all the
We assume,for
horizontal
simplicity,
that
resulting
as
will involve
and
the
O(mn) comparisons. In
algorithm
may
require
0(mn)
there
general,
time
just
to
may be as many
them. But
report
an
We would like to find
mn.
may be much smaller than
well
when there are few intersections and
that performs very
not too poorly
algorithm
We
achieve
when there are many.
it by combining two
of our favorite
techniques:
choosing a specialorder of induction and strengthening the induction
hypothesis.
the
number
of intersections
Intersections
8.6
Figure 8.17
The order of
segment)
the
order
that
Intersectionsof
is determined
induction
the plane
\"sweeps\"
in which
intersect
they
of Horizontaland Vertical
not a
line,
by an imaginary line (an infinite
to right;
the line segments are consideredin
from
left
with
this imaginary
intersections, we also
so far. This information
will
be
about
helpful
283
Segments
lines.
vertical
and
horizontal
Line
line.
the line
for computing
addition
In
segments
the
next
that
to computing
we
intersections
have
seen
more
One
feature of
knowledge of all
sweeping
line.
Is
the
it
the
induction
intersections
better
line is
to check
considered?
line is
a vertical
to choose
considered or
When
we
lines
that can intersect it are still under consideration
line, the horizontal
On the other hand,
when
we look at
(since we have not yet reached their right endpoint).
of a horizontal line, we either have not yet
or the right
either the left endpoint
endpoint
them.
about
intersect
Assume that the
seen the vertical lines that
it, or we have forgotten
a horizontal
when
look
at
a vertical
It
seems
better
the
former.
Geometric Algorithms
284
sweeping
information
of
Induction hypothesis:Given
list of
k sorted x
coordinatesas described
x coordinates),
we know how
xk being the rightmost
lines that occur to the
intersections among the corresponding
lines
that are to the left of xk.
to eliminate those horizontal
to
(with
We call
right
endpoints
structure
be
lines
containing
discussed
under
still
are to the
of the
shortly.
The (k
+ l)th
detected
the
The (k + 1 )th
line
the
reached
3.
to
the set
endpoint
of
a horizontal
than
comparisons,
stretch
against
all
even
if no
line,
hypothesis,
case
which
in
of candidates. As we said,
considered, so we lose no
step thus extends the induction
the set
intersections
lines
are
intersections
is a
left
This
of a
endpoint
of candidates. Sincethe
should
not
horizontal line,
line's
so,
be eliminated,
by
right
the
in
endpoint
part
by
we add
case
has
not
the y
been
above, this is a
of the algorithm.
checking
we
are
hypothesis.
which
arguments
we
We
the y coordinates
coordinates of the
line.
The algorithm is
smaller
the
from
line.
horizontal
endpoint
right
To extend
lines
line
the
vertical
when
eliminating
is a
endpoint
simply eliminate
2.
and
consideration
left
of xk,
candidates.
(These are the
current x coordinate, and whose
are to the right or at the current
x coordinate.)
We maintain a data
the set of candidates. The implementation
of this data structure will
that are
lines
horizontal
the
horizontal
all
report
left
now
The
complete.
of comparisons
number
will
usually
be much
Unfortunately,
algorithm still requires 0(mn)
even
if the actual number of intersectionsis small.
If all the horizontal
from
left to right, for example, then
each
line must be checked
vertical
horizontal
lines, resulting in an 0(mn)
This bad case will hold
algorithm.
mn.
in the
worst
a horizontal
case,
this
line.
285
8.7 Summary
To
y coordinates
candidates.
Let the
and
yR,
that the
\342\200\224\302\273y/cSuppose
y 1\302\273J2\302\273
order according to
can find
one for
also perform
find
one
only
for
yR.
the
with
problem
yL,
the
be yL
considering
currently
in
be
of candidates
set
candidates are
in sorted
given
order). We
increasing
line by performing
vertical
two
binary
The
yi<yL^yi+\\^yj^yR<yj+\\-
exactly ^/+i,^/+2,\342\200\224.)>_/.
We
then
and
of comparisons
in the
set of
number
is in
\342\200\224..y*
line are
involves
are
set of
the
that
Suppose
original
the
lines
y\\,y2*
vertical
the
with
binary
though the
Even
yj.
intersect
that
lines
horizontal
one
and
yL
in
(namely,
horizontal
the
searches,
horizontal lines
y coordinates
their
the horizontal
of
coordinates
need to minimize
line and those of
y coordinates
the
let
and
we
algorithm,
of a vertical
the
improve
between the
scan the
until
y coordinates
can
we
for numbers
a one-dimensional problem. Searching
a one-dimensional range query. If the numbers
is called
are sorted,
this case, yL to yR)
time for a one-dimensional range query
is proportional
to the search
then the running
the number of items that are found.
time
to sort the
But, of course, we cannot afford
plus
a vertical line.
horizontal lines every time we encounter
Let's review
the requirements.
We need a data structure
that
allows
us to insert
a
to
a
new element,
to delete an element,
and
one-dimensionalrange query
perform
\342\200\224
there are several data structures
for example,
balanced trees
efficiently.
Fortunately,
\342\200\224
that
of elementsin
found.
elements
Complexity
the
handling
one-dimensional
the
8.7
of
intersections
of
number
O ((m + n) log
is thus
algorithm
(n being
number
insert
each
Since
can perform
the number
line.
The
running time
total number
is the
of
the
of intersections.
Summary
some
sense, geometric algorithms seem less abstract
than,
say, graph algorithms, since
we are used to seeing and handling geometric objects. But, appearances
are sometimes
at small
from
misleading. Dealing with huge number of objects is different
looking
pictures, and we must be careful that the picture that we have in the back of our minds
does not lead us to wrong conclusions. We must deal with many special cases, and make
The algorithm for determining
sure that we can cover all of them.
whether
a point is
inside a polygon
do not think of a polygon
8.2) is a good example. We usually
(Section
In
as being
cases
like
the
one given
in
Fig.
8.1.
Furthermore,
it
must
is
geometric algorithms.
The
techniques
techniques
that
we
for designing
studied in
have
(discrete) geometricalgorithms
the
previous
chapters.
are
Induction plays
similar
an
to the
important
Geometric
286
Algorithms
Intersection
Algorithm
hn)) ;
vm), (hx,h2,...,
v2,...,
((vx,
0y(v/))
I yfi(v/)
bottom
the
of line v,
y coordinates
(top)
begin
all x
sort
V
coordinates
{V
set of
is the
and place
order
increasing
in Q
them
is organized
not empty
Q is
remove the
ifp is the
insert
else ifp
ofhk then
from
left
perform
ofhk then
endpoint
into
hk
is the
Q;
p from
endpoint
right
remove hk
else ifp is the
to the
according
do
endpoint
first
as a balancedtree
horizontal lines }
of the
coordinates
candidates for
are currently
that
lines
horizontal
; it
intersection
y
while
in
0 ;
:=
yB(Vi)toyT(Vi)inV
end
The line-sweep
role.
geometric
(one
algorithms
common.
is
which
presented
Geometric
is common to
on induction,
based
several
The
8.5).
divide-and-conquer
(except for simple ones) seem to
and ingenious data structures
sophisticated
have not covered here any of these special
Section
in
algorithms
many
We
structures.
Notes
Bibliographic
We
was
require
have
data
technique,
Intersection.
have
discrete
literature
seen
small sampleof
geometry.
techniques,
Preparata
examples,
and
15 years.
Shamos
and a
new
relatively
last
the
Reading
[1985]
comprehensive
[1984].
convex
in this
algorithms
geometric
is a
computational
geometry
in this area, spanning
computational
numerous
only a
Further
and
hulls is
[1972]. A
field,
there
chapter.
exists
Several books
and Edelsbrunner
concentrate
due to Chand
and
containing
though
extensive
on
[1987] present
Additional
bibliography.
bibliography
Even
an
Kapur
books
[1970].
268 papers on
Drill Exercises
hull
convex
and other
algorithms
The algorithm
O (n logn)
algorithm
Schorn
and
A probabilistic algorithm
is O(n), was developed by
for
[1988].
time
running
problems of convexity
was
for
finding
Rabin
[1976]
by Ronse
published
Shamos and
[1987].
[1975].
Hoey
287
An
to Hinrichs,
is due
Nievergelt,
whose expected
and
Hopcroft
A general technique
for
involves the construction
of
[1979]).
proximity
problems
A
Voronoi diagrams.
Voronoi
for a given set of points
is a division
of the
diagram
into regions such that each
contains all points that are closest to one of the
region
plane
from the set. Voronoi diagrams can be constructed
in 0(n
points
log a?) time (Shamos
are
useful
for
a
of
and
[1975]).
Hoey
They
variety
proximity
problems.
The algorithm
for reporting
intersections
vertical and horizontal
lines
among
in
Section
is
due
to
and
The
Ottmann
time
of
8.6)
[1979].
(presented
Bentley
running
is the best possible in the worst case (see Preparata and
O (n log n +/?) of this algorithm
An algorithm for determining
Shamos
whether
there
are any
intersections
[1985]).
an arbitrary set of line segments
was developed
It
[1976].
among
by Shamos and Hoey
also uses the line-sweep
This
can
also
be
solved
a
divide-andproblem
by
technique.
with
the induction hypothesis
and Wood
conquer algorithm
strengthening
(Guting
There
a
of
literature
on
is
intersection
and
the
reader
is
[1984]).
large
body
problems,
referred to one of the books
Exercise 8.16 is from
listed above.
and
Faust,
Bentley,
8.17-8.18
are discussed in Preparata
and Shamos
Preparata [1982], and Exercises
[1985].
Drill Exercises
8.1
when the
8.2
is the
Let S be
different
The
an
arbitrary
simple
first
computing
points.
In PolygonJ
a vertex
of the
set of points
are the
polygons
set 5?
Either
in
8.2)
(Fig.
polygon
to determine whether
complexity of your algorithm?
vertices
whose
8.4
L intersects
line
the
points
the
in
Is there only
plane.
the uniqueness,
prove
plane
one
or
are
unique
all
on
arise
one line.
simple polygon
of two
show
an
(Section
8.4) proceeds by
by the rest of the
it is outside the
that
example
n given
an algorithm
Design
What
8.3
Point
algorithm
Complete
thing
we know
the convex
hull
point to the
about the
or anywhere
extra
hull
formed
point q is
We can
still
try to find the
two supporting lines from q to the hull by computing the angles to all other points, but it is
not clear any more how to select the minimum
and maximum angle, becausethese
angles
Find a method to determine
can be in any range.
of the lines from q to points in the
which
hull is a supporting line.
hull.
8.5
It may be
below
the hull,
else.
to a circle whose
according
Graham's
scan to work
Modify
288
Geometric
8.6
Algorithms
Simple
Graham's
Scan (Fig. 8.12) to reject p
by an example, that it is possible for algorithm
value of p. (In other words, the loop can be
in a row, one at a time, for
every
the value of m.)
executed forp steps without
changing
Show,
8.7
points
8.8
of n points
an example
Show
8.9
The
algorithm
The
line
line
take
plane
with distinct x
Q(n log
which
for
coordinates,
algorithm
a) steps.
segments
in time
Exercises
Creative
8.10
horizontal
should run
a set of n
Given
the
in
input
that
is a set
of n
is parallel
equal-sized
subsets
to find a
into two
on
either
side).
8.11
Let P be a simple
(not
be an arbitrary
point
connecting q to any
intersects is minimum.
whether a point is inside
8.12
Let
or outside
Design
a polygon;
P be a
in
to determine
algorithm
algorithm
8.13
R.
Many
convex-hull
asked
to
are based on
algorithms
a convex-hull algorithm
or
are similar
to
cyclic
The
order.
Design
running time of
algorithms.
sorting
an
the
You are
sort. In
to the
be
inserted
and
should
iteration,
point
possibly
current convex hull, which should consist of the convex hull of the points seen so far. The
should be done initially).
points should be considered in an arbitrary order (i.e., no sorting
The algorithm
whether a given
should be based on an efficient data structure to determine
of your
time
point is inside a given convex polygon. What is the worst-case running
to
have
do
not
all
for
all
the
the
details
(You
cases.)
algorithm?
supply
special
develop
one more
each
8.14
Considerthe
with
the
computing
should
that
is similar
to
insertion
efficient
an
be considered
the
an
hull
one
O(nlogn)
at a time
algorithm for
point
Creative
8.15
289
Exercises
Assume
convex
8.16
convexhull
A {/-approximate
set of points
are
from
P,
8.17 Let P be
such that
all
of a
points
in P
of convex
that
hulls
points
in
need to
the plane.
be
8.18 a. A
b.
8.19
8.20
points of a given
all maximal
find
set P
with
three
points.
dimensions
(the
of dominance
definition
is
Let S be a set of points in the plane. For each p e S, we define D (p) to be the set of points
an algorithm
to compute the sizes
in S that are dominated by p (see Exercise 8.18). Design
in
of the sets Dip) for all p e S. The running
time of the algorithm should be O(n \\ogn)
the worst case.
n points
Given
has
them
in the plane,
find
the
the maximal
worst case.
8.21
8.22
The input
The
is a set
or
input
have
8.23 Design
an algorithm
line segments
by
using
horizontal
time
and
of the
vertical
algorithm
290
Geometric
Algorithms
8.24
8.25
as the
in Section
discussed
algorithm
time
in
+n) + R)
0((m +n)log(m
8.6.
That
(where
found).
is a set of
The input
determine whether any
line
the
to
Design an algorithm
The algorithm needs to output
in the worst case.
time of the algorithm should be 0(n \\ogn)
only yes or no. The running
to
the
horizontal
and
method
vertical
the
but
Use
maintain
case,
(Hint:
similarly
line-sweep
different information.)
A
arbitrary
two
of the
line
in
segments
segments
plane.
intersect.
polygon
grid
y axis.
Design
(i.e.,the
area
an efficient
order.
8.26
The
input
set of intervals
to identify
is a
an algorithm
Design
Can
rectangles.
8.30
The
intervals
run in time O (n
should
are
which
that are
\\ogn)
is a
set of
all of whose
rectangles
intersection of all
by their two
represented
in the
endpoints.
in another
contained
worst case.
to find the
algorithm
The
a line,
all
obtained
algorithm
8.29
is a
on
edges are
to the
contained
are
that
to the
parallel
axes. Design
an
the rectangles.
is a set of n circles
in the plane. Design
an algorithm
to detect whether
there are
input
not need to
does
any two circles in the set with nonempty intersection. The algorithm
the intersection,
time of the algorithm should
compute
only output yes or no. The running
be O (n logn) in the worst case.
input
is a set
of n polygons,
polygons in
there
need to
the
compute
intersection,
vertices.
an algorithm
to detect whether
does not
intersection. The algorithm
is the worst-case running
only output yes or no. What
each with k
the
Design
time?
8.31
The
a
8.32
The input
a linear-time
at least
8.33
The
should
extension
The
a cyclic
order). Design
The output,
algorithm
a cyclic
which
is
order.
convex polygons given by their lists of vertices (in a cyclic order). Design
to compute the union of the two polygons (i.e., the area enclosedby
algorithm
is two
one of the
input
algorithm
8.34
is two
input
linear-time
is a
polygons).
set of
edges are
union
be represented by its
to the skyline problem
is a set of
input
algorithm to compute
all of whose
rectangles
to compute the
of all
in
chapter
in the
triangles
their intersection
parallel
union
axes. Design an
a polygon. It
(This problem is an
to the
is obviously
order.
5.)
0(n
logn)
Creative Exercises
8.35
8.36
The
input
linear-time
The
set of
The input
points
by its list
given
polygon
algorithm to find
is a
input
whether
8.37
is a convex
n triangles
in the
points in
vertices
in cyclic
given
order.
Design
polygon.
an 0(n2
Design
\\ogn) algorithm to
that are vertices of a square.
plane.
the set
of
291
determine
is a set of n
there
whether
polygon.
We are interested in
set of n points all of which have integer coordinates.
of parallel
lines such that all the points are contained in at least one of the lines
in the set. The lines must either be parallel to the axes or have a 45-degree angle
with the
axes. Design an 0(n \\ogn) algorithm to find a minimum-size
set of lines satisfying
these
conditions. Again, the lines must all be parallel,
so, in particular, if one of the lines has a
45-degreeangle with the coordinates, then all of them do.
8.39
is a
a set
finding
line
is a
8.40
The input
line),
into
is 2n
such
two
points
in
that n points
the blue points
smaller
problems.)
CHAPTER
NUMERICAL
AND
ALGEBRAIC
ALGORITHMS
One
Two
Four
one is
plus
two is four.
plus
plus
Eight
four
is eight.
eight is
A child's
9.1
we
We are
ten.
poem
complicatedarithmetic
very
that
operation,
numbers
large
phenomenon
However,
granted.
we
that
are
have
in
fact
we
these operations that
whether it is multiplication,
with
familiar
so
usually
operation, we are
an arithmetic
perform
for
algorithms
algorithms
more than
Introduction
Whenever
when
two.
plus
the
or large
seen in
straightforward
sequences of
the
input
previous
chapters
become
inefficient
the
corresponding
or a more
division,
algorithm is
numbers
an algorithm.
executing
take
are
not
always
involved.
the
The
best
same
grows.
of an
have done in previous
we will
measure
the complexity
chapters,
that
of
the
the
number
executes.
For
the
most
\"operations\"
algorithm
algorithm
part
by
we will assume
that basic arithmetic operations (such as addition,
and
multiplication,
is
a
This
time.
reasonable
of
when
the
take
one
unit
division)
assumption
operands can
be representedby one or two computer words (e.g., integers that are not too large,
real
There are cases, however,when
the
numbers).
single-precision or double-precision
In
we
have
to
such
take
2000
into
account
are
cases,
integers).
digit
operands
huge (e.g.,
As we
Algebraic and
294
Numeric Algorithms
operands, or at least to be aware that the basic operations are not simple.
but are in fact very
that look very efficient \"on
to
paper,\"
design algorithms
It is possible
the
are
of
the
sizes
because
ignored.
operands
inefficient,
sometimes. Given an integer
is confusing
of the \"size of the input\"
The
meaning
it
is
natural
to think of the value
an arithmetic operation,
to perform
n on which we want
to our usual convention of using
this is contrary
the
However,
n as the size of the input.
The
distinction
for
its
size.
is
of
the
important.
defining
very
input
requirements
storage
even
can be done quickly,
numbers
hand,
by hand. On the other
Adding two 100-digit
cannot
done
in
a
be
reasonable
number
value
a
to
time
100-digit
by
represented
counting
n
can
be
Sincea
number
fastest
the
bits, its
even by
represented by riog2Ail
computer.
that
an
For
size is defined as |\"log2A2~|.
O(logAi)
operations
example,
algorithm
requires
for computing
when n is the input (for example, an algorithm
2ri) is considered linear,
of the input, whereas an algorithm
size
is a linear function of the
that
since 0(\\o%n)
the
of the
size
O(V>0 operations
requires
We
algorithms.
what
is probably
factoring
n by
trying
for
designing
all
in
how
finding
matrix
beautiful
the oldest
this
to
chapter
compute
on interesting techniques
powers
of a
given
number.
known
We
then
Euclid's
nontrivial
common
the
2200-year
We
(for example,
algorithm:
algorithm for
that
divisor.
It
is
modern
greatest
computers use a
quite amazing
for polynomial
old algorithm.
and
We then discuss algorithms
multiplication
and most
with one of the most
and we end the chapter
multiplication,
important
\342\200\224
the fast Fourier transform.
algorithms
present
9.2
concentrate
first discuss
is the input
is considered exponential.
to
or equal
we
usual,
when
Exponentiation
with
start
The
a basic
Problem
arithmetic
Given
operation.
two
positive
integers
n and k,
compute nk.
nk=n-nk~l.
reduce the problem
since
to that of computing nk~\\
the problem can be solved
the
and
induction
on
k,
straightforward
resulting
by
algorithm is given in Fig. 9.1. We have reduced the value of k, but not its size. The
k iterations.
Since the size of k is \\og2k, the number of
straightforward algorithm
requires
interation is exponential in the size of k (&= 2log2*).
is not bad for very small values
This
of k, but it is unacceptable for large values of k.
this
Another
to reduce the problem is to use the fact that nk = (nk/2)2. With
way
k
to one with
we reduce the problem
n and k/2.
observation,
Reducing the value of by
half corresponds to reducing its size by a constant. Thus, the number
of multiplications
\342\200\224
will
be linear
in the size of k. We now have the skeleton of the algorithm
repeated
=
k
for
2J
is
The simplest case
for some integer j:
squaring.
We
can
Therefore,
easily
9.2
295
Exponentiation
Power
Algorithm
n and
Input:
Output: P (the
of nk).
value
begin
P:=n;
fori:= 1 tok-\\
P :=
do
n*P
end
Figure 9.1
nk =
But
n2' =
if k
what
j times.
is
not
with
two parameters
This
reduction
is
a power
of 2?
and
k, and
not
if k/2 is
an
not
Consider again
reduced
valid since
always
the reduced
integer,
Power.
Algorithm
then
integer,
the
be
not
may
the conditions
satisfy
is
(k-\\)/2
to a
problem
k/2
an
reduction
the
and
integer,
the following
is
reduction
appropriate:
\342\200\236*=\342\200\236L(*-l)/2
We
we
odd,
square
Complexity
progresses,
more costly.
algorithm
We leave it
of
an application
Encryption
of
this
n.
by
is 0(\\ogk).
the
the
in which
algorithm
the
As
the
become
complexity
of the
multiplications.
the numbers do not grow
cost
of
algorithm
multiplications
measure for
is
9.2.
in Fig.
Therefore,
for k/2. If
The number
of this
We
during
now
the
algorithm.
Application
The study
realistic
a more
larger.
the solution
multiply
is given
multiplications
the reader
to
and
(k-\\)/2
become
numbers
the
square
simply
The algorithm
of
number
The
execution of the
An
most 2\\og2k.
however,
under
for
solution
the
is at
multiplications
present
If k is even, we
have an algorithm.
now
to Cryptography
cryptography
schemes
usually
is beyond
rely on
the
of this
book,
on the
two
and
we
discuss
participants
it briefly.
who want to
and must
algorithm
encryption-decryption
We want
this need to exchange
to avoid
of participants. The following
is known
as the RSA
Adleman
who
Rivest, Shamir, and
(after
[1978],
of participants
be used by a group
(e.g., computer users)
agree
to themselves.
scope
296
Numeric
and
Algebraic
Algorithms
Algorithm Power_by_Repeated_Squaring
n and k (two positive integers).
Input:
k)
(n, k
div
(n,
of nk).
begin
ifk = 1 then P := n
else
z := Power by Repeated
if k mod 2 = 0 then
P :=
Squaring
2)
z*z
else
P := n*z*z
end
Figure
want
to communicate
one for
and
who
encryption
9.2 Algorithm
one
has
two keys,
of other
only
number
the
A participant
P in the RSA scheme
participants). These keys are chosen as follows.
selects two very large prime numbers p and q and computes their product n =pq.
He then
chooses another
d and (p-\\)(q-\\)
have no common
very
large integer d> such that
divisor. (See the next section for an algorithm to verify
that
fact; if d is a random
then the condition above is likely
From p, qy and dy it is possible
to occur.)
number,
not easy)
to compute the value of a number e that satisfies
(although
(9.1)
e'd=\\(mod(p-\\)(q-\\)).
As we
be the encryption
key and d the decryption key. The values
can read. (We assume
publicized by P in a central directory that everyone
the
of a trusted directory such that no other person can forge P's keys.) The
availability
value
of d, as well as the values
of p and q, which
are not needed
anymore, are kept
of n
secret
see
shall
and
next,
will
e are
by P.
Let M
message
can
be an
be
integer
translated
that
to a
to a messagethat
of
bits, which can
sequence
corresponds
Assume
that M is smaller than n\\ otherwise
M can be broken into
each smaller than n. The encryption function EP that P uses is very
EP(M)
= Me
= Cd
several
to
small
an
(every
integer).
messages
simple:
(modn).
to encrypt
wants
be translated
(mod n).
encrypt
it can
messages
be performed
by
P> since
P.
the
297
are
valid
DP(EP(M)) = M, hence these
thus
of
consist
one
algorithms
computing
only
and
or
one
division
the
these
are
(Me
Cd)
(for
power
congruence), although
operations
on
numbers.
The
n
modulo
can
be
at
performed
very large
operation
applied
any step of
and
not
the
at
the
end.
This
is
true
because
algorithm,
necessarily
can
One
prove that
and decryption
encryption
n.
and
y,
is very
need
to
the modulo
way
the
of
step of the
grow
in each
operation
values
the
do not
operands
by Repeated-Squaring
Power
algorithm
to
multiplication
this
multiplication
modify
Applying
Power
(or
0(\\o%e)
only
but each
power,
Both
important, since
that
guarantees
functions.
= [^(modA2)-y(modA2)](modA2),
x-y(xnodn)
computation
(9.1)
by
by
Repeated-Squaring
modulo n. Thus,
a multiplication
the
applying
only
changing each
requires
scheme
RSA
only
n) multiplications and
is no known
algorithm
There
key cryptosy
the
unique
integers
that
As usual,
or m
we
divides
GCD(a? -m,
1
Find the
m, denoted
and
both
and
by GCD(az,
m),
m, and
given
integers.
smaller
If k
divisor
Problem
The
positive
of numbers
can factor
Algorithm
The greatestcommon
is
that
stem.
Euclid's
9.3
divisions
try
to
reduce
without
both
m),
changing
the
m, then it
we now have
n and
and
the problem to
problem?
divides
one of smaller
their
difference!
Can
size.
Euclid noticed
the
If n >
again,
we somehow
obvious
m,
positive
then
GCD(h,
we reduced
make n
answer:
m) =
the values
for
factoring
and Numeric
Algebraic
298
Algorithms
to be efficient, we must
and not their sizes. For the algorithm
aw =24,
and
we will
For example, if n is very large (say 1000digits)
This
will
24
from
times.
take
n
az/24
subtract
to
need
computation
O(n)
approximately
in the size of n.
is exponential
steps, which
m from
n and apply
the
Let's look at this algorithm again. We subtract
same
If
we
In other
n
m is still larger
than m,
subtract
m again.
algorithm to n m and m.
the result
m. But this is
m from
n until
becomes less than
words, we keep subtracting
Division can be done
exactly the same as dividing n by m and looking at the remainder.
which
is presented in Fig. 9.3.
leads directly to Euclid's algorithm,
This
quickly.
of the
numbers
in
question,
sizes.
the
reduce
We claim
Complexity
n+m;
its
specifically,
is O
operands)
changed
of algorithm
a mod A3. Then,
b and
into
and
A3 mod
But,
since
(a mod
a >
in two
So,
A3).
have
we
A3,
a mod
in a
In
GCD.
in the
next
iterations,
A3
< all,
the
claim,
first
sufficient
they
number
which establishes
the
are
a is
to
show
Let's look at
of iterations.
number
constant
of
independent
step
it is
and
changed
(a>b)
two
are
into a mod A3
changed to a mod
A3.
claim.
Multiplication
Polynomial
n-\\
n-\\
Let P
half
by
this
size of
in the
time
running
iterations
consecutive
9.4
running
has linear
algorithm
time (counting
of a is reduced
Euclid's
that
represented by
Q =
and
Pix'*
list of
its ordered
GCD
Algorithm
Input:
two polynomials
of degree n
coefficients.
(m, n)
n (two
and
Qi*1
gcd
positive integers).
of m and n).
begin
a :=max(
b :=
r :=
n, m
);
n, m )
min(
1;
{r is
r>0do
while
r :=
a mod b
a := b
A3
:=
gcd :=
the
remainder}
r;
end
Figure 9.3 Algorithm
GCD.
- 1.
A polynomial is
9.4
The Problem
PQ =
We can
then
number
seen
it is
that
algorithms
algorithm
coefficients
the
have
We
better?
not
+ l+P/J-2<7/+2
+ P/ + l<7/i-l
so
surprising
is positive.
the answer
in Section
9.6.
But
we
first,
quadratic
straightforward
A
0(n
complicated
a simple
describe
if we
that,
Can we do
be 0(n2).
will
of
improvements
many
that
to see
is easy
It
+ Po<7o-
\342\226\240*
logAi)
divide-and-
algorithm.
conquer
we assume that
parts. Let P = P, +xnU
For simplicity,
two
+ '\"
(9.2)
by now
discussed
be
will
P/J-l<7/
the
+ <7o
\\<ln-\\XH-X +
+p0
compute
(9.2),
\342\200\242\342\200\242\342\200\242
pn.xxn
Pn-tfn-\\X2\"-2
follow
polynomials of
of two given
product
- 1.
degree
the
Compute
299
Multiplication
Polynomial
equal-sized
\"'
P\\=P0+P\\*+
a?
is a
power of
2.
We
each
divide
+Pnl2-\\*
and Q
into
polynomial
,n/2t
=Q, +xnu
Q2, where
. P2=Pn/2+Pn/2+\\*+ \"' +Pn-\\*
P2,
/i/2-l
and
/i/2-l
*\342\200\242\342\200\242
<2l=<7o+<7l*
+<lnl2-\\X'
(?2=<7/i/2+<7/i/2+1-* +
/i/2-l
\342\200\242\342\200\242'
+<ln-\\X
We now have
PQ =
The
(/>i+/V/2)(Qi+Q2*\"/2) = PiQ\\+(P\\Q2+P2Q\\)xn,2+P2Q2xn.
for
expression
compute
the
product
of
by
degree
induction,
n/2.
We can
then add
the
to complete
7\302\273
The factor
from
+ <9(A2),
= 47>/2)
4 comes
adding
(see Section
the
ni)=l.
from the
smaller
3.5.2),
any
comes
0(n2)
(see
improvement
Exercise 9.4).
To
get
an improvement
four
solving less than
reason we use such an
next
section).
to
the
subproblems.
table
elaborate
by
quadratic
(the
in
the
300
Algorithms
We
have
the
A+
to compute
want
three
only
by
relation
anyway.
Qi
we need
and
D,
contribute
which
only
O(n)
to
to
need
we
Hence,
can be
recurrence
the
is
relation
recurrence
new
only
do not
compute
is that we
their sum! If we
know
to
polynomials: A,
and subtractions,
The
observation
important
then B + C = E-A-D.
of smaller
products
additions
computed
Q\\
+Dxn. The
(B + C)xn
C separately;
to compute B and
= (P { +P2)
E
(Q\\ +Qi),
product
compute
Pi
P\\
7\302\273
implies
Notice that
0(A2,og23) =
the
0(n L59).
are
and Q\\+Q2
P{+P2
polynomials
coefficients
They are formed by adding
a nonintuitive
This is quite
differ
way to multiply
polynomials,
by n/2.
reduces the number of operations significantly for large n.
polynomials
a strange
in
the
original
whose
way.
yet this
indices
algorithm
9.1
\342\226\241
Example
Let P
to
related
-jt
+ 2*2-jt3,
divide-and-conquer
using
the
= 2-x-x2J
A=(\\-x)-(2+x)
(2-x)-(-\\+2x)
= -2
(3-2x)-(\\+3x)
= 3 + lx-6x2.
+ 5x-2x2,
and
E =
From
\302\243,
A,
and D,
B+C = 3
Now, P
A +
(B + C)xnl2
P Q =2-x -x2 +
Notice
that we used
and 12 additions
and
to
a constant
multiplications
have
+ 3x3
3x2
-3x4 -2x4
+ 5x5
12 multiplications
-A -B:
we
instead
carried
n is
large.
when
factor
the
had
of
is reduced by
that
in
about
the
-2x6
5x5-2x6.
to 16
compared
subtractions
9 if
=E
and we
+ Dx\\
+ 2x2 + 3x3-5x4+
= 2-x
multiplications
B +C
+ 3x-3x2.
\342\200\242
=
we can easilycompute
(The
of 9.
number
straightforward
a?04.)
could
(We
recursion
in
the
straightforward
have reduced
one more
of
step.) The
algorithm,
the number of
savings
are,
of
additions
9.5 Matrix
Multiplication 301
C of
product
two
x n
A and
matrices
B is defined as follows:
(9.3)
5>,**V
Cij=
The Problem
of real numbers.
The
Compute the
way (and
straightforward
C =A
product
seemingly
the
only
xB of
way)
two
n x
to compute
n matrices
matrix
multiplications and (n
product
is to
additions.
Notice
that
follow (9.3), which
\\)n2
using n3
requires
of rows and columns in the matrix,
rather than the size of the
n represents the number
which is n2. We now
two different schemes that show
the possibilities
for
present
input,
improvements.
Winograd s Algorithm
9.5.1
for
Assume,
simplicity,
that n is
even. Denote
nil
nil
Ai =
Z
k
After
ai,ik-\\
'ai,2k>
and
Bj=
= \\
rearranging
\302\243
b2k-\\j'b2kj.
k = \\
terms, we get
nil
\302\243
(ai,2k-\\+b2kj)'(<*i,2k
k=\\
C'J=
+ b2k-\\j)-Ai-Bj.
To compute
and BjS need to be computed
only once for each row or column.
of
The
total
number
n2 multiplications.
multiplications
requires
only
BjS
of additions
has increased by about
has
thus been reduced to Vin3+n2. The number
in cases
where
Vm3.
This algorithm is thus
than the straightforward algorithm
better
more
than
additions can be performed
multiplications.
quickly
the /4,s
But
all
the
Afi
and
This algorithm
Comments
make a difference, even for
simple form. The next algorithm
Strassen
9.5.2
use
that
expressions,
carries
order of the
the
rearranging
such as matrix
multiplication,
can
computation
which
have
farther.
s Algorithm
the divide-and-conquer
9.4. For
algorithm in Section
We
shows
method
in
a way
similar
to
a?
the
polynomial
is a
power of
multiplication
2. Let
302
\"^
A\\2
1.1
the
#1,1 #1,2
and
#2,1
C =
#2,2
\302\243/>7s,
^2,1 ^2,2
matrices.
and
AjjS,
Algorithms
B=
^2,2
^2,1
where
Numeric
and
Algebraic
We
can
use
the divide-and-
9.23.)
we
when
The
algorithm
a product
time
each
of elements appears.The
regular
Substituting
each
less
we would get
8 multiplications,
than
an
algorithm
that
is asymptotically
faster
than
cubic.
most important
to compute
part
product
of
similar
the
eg
c d
c d
00a
write
the
product
r
\342\200\224
OOrrf
We
00
b 0
'a
problems.
P s
=
to computing
is equivalent
the recursion
the product
Computing
'a b
of
two
(9.4)
multiplications
as
AX
required
Y.
are
We
to evaluate
Y.
easy to compute.
two
of
which
Let's
look
As
it turns
out, we need four
are very similar). They are as follows:
Product
Type
the number of
to minimize
ways
that are
for special matrix
products
such
of
special products (the last
types
for
looking
No. of
Multiplications
a a
a)
a a
l
a(e+f)\\
9.5
a
P)
a-b
-a(e+f)
ae
ae+b(f-e)
a b-a
5)
We
+ bf
a(e-f)
bf
matrix
look
now
steps of
the
we may be
of multiplications,
number
trial
to reach
error
and
even clear,
~b
B =
it is
but
the
less
somewhat
able to
0 0 0
use
in
less
at the
something
This process is
hardly
(9.4)
into
than the
end.
It
takes
several
nominal
a lot
of
or
straightforward
Let
0 0
a-b
0 d-b
c-b 0 0 c-b
=
given
0000 , c 00cc
00c c
0000
0
save
magic.
0 0
than
product
types of products
combinations.
right
0 0
b b 0
303
Multiplication
a(e+f)
-a -a
Y)
Matrix
b-c 0 0 b-c
E =
and
0 b-c
c-b 0 a-c 0
0 0 0 d-c
0 00 0
All
the products
BX + CX +DX+ EX.
above, except for EX, can be computed with one multiplication
using types a or p. The
= F + G, such
\302\243
two matrices
is to compute EX.
But E can be divided into
only
problem
F is of type y and G is of type
that
8:
A=(B
Then,
+ C
+ D+E)
0 0
0 0 0 0
a-b
F =
c-b 0 a-c 0
0 0 0 0
So,
9.10).
0 d-b 0 b-c
G
0 0 0 0
0 0 0 d-c
overall,
product
AX =
therefore
and
AX =
a,
(see also
of type
and
one
Exercise
Numeric
and
Algebraic
304
Algorithms
the original
7 products of matrices of half
size, and a constant
less important
are
than the products,
of matrices. The additions
in
time
which
is basically a
two
nxn
be
done
of
matrices
can
addition
0(n2),
because
the
is
not
The
term
dominant
factor in the
size
of
the
matrices.
O
the
in
time
(n2)
linear
=
+
The
this
solution
of
which
is
recurrence
relation,
T(n) lT(n/2) 0(n2).
recurrence
If we use the derivation
is approximately
is T(n) = 0(nlog2 ), which
0(n2S[).
relation
to reduce the
18 additions
above, we obtain
(see Exercise 9.10). It is possible
described
to 15 (Winograd
but this reduction does not change
of additions
the
number
[1973]),
use
We
Complexity
of additions
number
running time.
asymptotic
Comments
1.
are
There
Empirical
studies
algorithm
faster
three
algorithm:
indicate
the
than
straightforward
0(n3)
straightforward
100
to make
Strassen's
Roth
and
(Cohen
algorithm
[1976]).
Strassen's algorithm
similar errors in
2.
the
is less
the
algorithm
will
algorithm.
probably
create
much more
is obviously
complicated and
Strassen's
Furthermore,
algorithm.
harder
and
inversion
determinant
computation.
We
will
see
to implement
algorithm cannot be
whereas
the regular algorithm can.
easilyparallelized,
is important.
It is faster
Nevertheless,Strassen'salgorithm
algorithm for large az, and it can be used for other problemsinvolving
matrix
output.
Strassen's algorithm
than the straightforward
3.
Strassen's
input,
the
than
in Chapter
10
regular
as
such
matrices,
that
several
problems
Boolean Matrices
9.5.3
In
this
we consider
section,
These
and
case
special
are 0
or 1, and
correspond to or and
rules (which
definitions
product;
of sum
hence,
defined
and
(both 0+1
with
and
and
of
two nxn
defined by
the
respectively):
product
algorithms
the
elements
Multiplication 305
9.5 Matrix
Strassen's
cannot be used for Booleanmatrices,
it requires
because
algorithm
subtraction. However, there is a trick that allows us to use Strassen's algorithm.
We
an
+
bit
as
n
where
n
is
consider
modulo
the
size
of
the
and
we
1,
matrices,
every
integer
of such
use the rules of addition and multiplication
for
if
So,
integers.
example, az=4,
and 1 + 1 + 1 + 1 + 1 =0.
It turns out that,
if we compute
then 1 + 1 =2, 1 + 1 + 1 =3,
the
matrix
to these rules and if we substitute every nonzero entry
in the
according
product
final result by a 1, then we get the Boolean
This is so, essentially,because we
product.
will
the number n + 1 (we omit the proof). (More precisely,the integers
not \"overflow\"
k form a ring, which
is an algebraic
structure with definitions
modulo
of sums and
that
certain
can
Strassen's
be
satisfy
algorithm
properties;
applied to any ring;
products
Ullman
for
more
see Aho, Hopcroft, and
of
[1974]
details.) Thus, the complexity
Boolean
matrix multiplication
is also
The use of Strassen'salgorithm,
0(n2Sl).
than
Boolean
Next, we present
however,
operations.
requires
integer operations rather
that utilize the properties of Boolean
to improve the running
two
operations
algorithms
matrix multiplication.
These
are more practical in most
of Boolean
time
algorithms
than Strassen's algorithm
for Boolean
matrix multiplication.
situations
k operands
Since Booleanoperands
in
only one bit of storage, we can store
require
n is stored
in
of size k. In particular, since we assume that
one computer word
one
The regular algorithm
for
word, we can storek bits for k <log2Ai in one word.
computer
consists of n2 row-by-column
matrix
(or inner products), as
products
multiplication
Therefore,
each of
which
Boolean
vectors
assume
of
time.
the
and
of k-bit
how
idea
(For example,
to
Boolean
store
of
represented
on,
we
the
by
will
in
time
all the
algorithm
the
four
k bits
not
a
of a
make
time; thus,
computed
the
whether
they involve
(k22k) (we can
in
and
and
two
is stored
ih is
a distinction
Boolean
between
do
and
with
bits
k. There
k. We can
that; see Exercise
of size
of
where
by
ih
another
illustrate
of size
than
better
a (or
idea
Both ideas
products
represented
ia
Nevertheless, we
this
vectors
table of
at entry (iaJh),
the integer
0s.)
further.
a two-dimensional
b
two
not unreasonable to
steps:
combine
Booleaninner
actually
of
assume
even
two
We
of size k
is all
result
on
dependent
arithmetic
basic
multiplication
in
products,
product
integers.
it is
vectors
assumption. Then, we
matrix
results
two vectors
the
check
since
products,
all of them
9.24),and
product
idea
possible
compute
we
two k-bit
Boolean
techniques
The first
are 22k
one unit of
of nlk
sum
for
Assume,
'bmj.
inner
the
Finding
multiplying
can be
product
avoid
takes
(besides
primitives
to improve
interesting
inner
an
say,
a\\m
\302\243
m = \\
into a
product
of size k.
product of two
not
do
computer
simpler than,
integers
inner
each
vectors
an inner
computing
of the
usually
the
that
We can divide
of size k is
of computing
consists
product
Boolean
involves
a multiplication
that
inner
//th
k divides n.
that
simplicity,
now
The
in (9.3).
defined
the
and
ia
The
2*x2*.
size
is
k bits
the
integer
of b.
/?), since
From
they are
of size k, we can
vectors
represented in exactly the same way. Thus, given two Boolean
compute their product by simply looking at the table. If we can access a table of size 22k
time (once
in 0(1) time, then each inner product of size k can be computed in constant
306
the
Algorithms
table is
assumption
we can
then
input,
it requires
constant
k =
store a
number
with
[\\og2n/2\\.
usually
In that case,
We
assume
have
that,
if
size
the
The assumption
that
of the
we can
n is
word (or
log2Ai bits in one computer
we can compute
is constructed,
table
constant
a Boolean
words). Once the
=
Notice
that
the
table
n in time
size
O
O
of
(n/k)
inner
(n/\\ogn).
depends
only
product
the product
of two Boolean
not on the matrices.
So, computing
on the value of k and
and extra storage of O(n). We can also
matrices can be done in time 0(n3/\\ogn)
in
the
table
size is 0(n2), but we save an extra factor
which
case
be
to
k
choose
[log2\302\253J,
However, if we can afford an extra space of size
of 2 in the multiplication
algorithm.
0(n2), we can find a faster algorithm.
Consider two n x n Boolean matrices A and B. The usual way to view matrix
each involves a row
multiplication is as defined in (9.3): We perform n2 inner products,
the two matrices
can also multiply
columns
of A and a column of B. We
by multiplying
of A with rows of B in the following way. Denote the rth column of A by Act^l, and the
and ^[rjasa
lx/i matrix.
Consider
rth row of B by BR[r],
Ac[r] as an n x 1 matrix,
x
The product of Ac[r ] with BR[r ] is an n n matrix, whose ijth entry is the product of the
the 7th entry of BR[r ] (see Fig. 9.4). It is easy to see that
of Ac[r ] with
/th entry
of computer
number
A'B
(9.5)
itAc[r]-BR[r].
r
= \\
(9.5)
expression
are performed,
additions
n/k
A-B
(9.6)
YtAi-Bi.
/
=!
BR[k]
Ac[k]
ABC
Figure
9.4
Multiplying
matrices
columns
by
rows.
307
Matrix Multiplication
9.5
is
now
problem
an example
Also,
is 0(n2).
requirement
storage
find
this
table
depends
row j
to
j of Aj
on Bh and
and see the
of rows
as an
represented
the first row of Aj
be
can
be constructed
must
combination
integer
in Fig.
j of Aj (e.g.,
to 3, the third row corresponds to 4,
9.5 corresponds to 1, the second
row corresponds
This integer is the address
in the table where row j of C, is stored. It takes
and
so on).
time to find a row of C, in the table, and O(n) time to copy this row to the
0(\\)
to the
corresponding
appropriate row
We now
O (n
time
induction,
in C,.
that
show
\342\200\242
2k).
Each
that
we
Assume
in time
of
0(n2).
of Bj can
rows
combination
know
corresponding to integers
trivial.
of row
representation
binary
that
are
less
significant 0 is
of rows corresponding to
xxxx
000000
of / -
is xxxx
We assume,
integer.
combinations
sum
corresponding
011111
\342\200\224
rows corresponding
the row
plus
the
be computed
to
/ is
in
by
of rows
to 0 is
namely,
its least
equal to
the
sum
corresponding to 0000100000.Since
xxxx
000000
0
0
I
0
0
1
0
1
0
1
1
0
I
0
1
1
1
0 1 1 1 0 1
1 1 1 0 0 0 0
1
0
B,
1
0
0
0
1
1
1
0
0
C^Afr
Ai
Figure
multiplication.
1
0
1
0
0
is given
algorithm
in
Algorithms
[1970]), after
et al.
(Arlazarov
algorithm
The
and Numeric
Algebraic
308
the
and
nationality
number
of
inventors.
its
9.6.
Fig.
(A, B, n, k) ;
Boolean_Matrix_Multiplication
Boolean
and
B
A,
matrices),
k(an integer).
(twonxn
Input:
of
A
and
C
B).
(the
Output:
product
| we assume, for simplicity, that k divides n )
Algorithm
begin
C toO ;
the matrix
Initialize
fori :=Otonlk- do
1
Construct
Table,
forj :-
do
ton
be
Let Addr
k-bit
the
A[j, w +
number
\342\200\242
\342\200\242
\342\200\242
l]y4[/', m+2]
] to
row j
A\\J,
in
m+k]
end
Figure
9.6
Boolean
Algorithm
JAultiplication.
JAatrix
as
group
All
only on
it depends
each of
the
size m
a m-bit
steps
(we
integer;
of the algorithm
thus,
We now divide
for simplicity, that
of m.
value
the
assume
again,
each
will
row
of
is
\302\243,
be performed
each
m
represented
row of
divides
n).
Bt
into
by an rt/m-tuple
on these tuples.
nlm
groups,
We consider
each
of integers.
add two
in nlm
rt/m-tuples
corresponding
entry
in constant
performed
use
this trick
for
adding
both for
the rows
approximately
represent n in
one
instead of the
usual
that
the
changing
and, since we
0(n)
running
an
order of
the
As
an
to the fast
introduction
Fourier transform,
we
the hardware.
by
Lipson's excellent
from John
quote
aesthetic
grounds
it
Fourier
is strong
all
on
algorithms
of these
The
FFT algorithm is
We concentrateon
no
product
p(x)-
and
simple,
of
Given two
polynomials
a^x
its
+an_2xn~2
This linear
function
sixties.
p (x)
straightforward.
multiplication.
polynomial
is
above,
not
and q (jc),compute
We
polynomials.
'\342\200\242'
+a{x
well defined.
+ a0
by the
function
corresponds
is
their
to a
line
of
its
of
in the
degree
by the
plane, it
coefficients
it is not
but
1, which is
two
can
the
polynomial
in
increasing
the
only
a linear
coefficients
also
specified
represent
adequate,
specified
usually
list
not
have
We
usually
of degrees.
+ a0.
is not
development
\342\200\224
FFT
the
it
\"super\"
the mid
in
discovery
the
of
q (x).
means
one application
Problem
The
because
perhaps
\"FFT\,")
293.)
by
only
its
grounds
The fast
applications.
(Lipson[1981],page
dramatic
on
on
problem,
practical
or even
the
on technological
grounds;
important
to as
referred
(popularly
transform
of
unexpected areas of
new and
up
opens
an
is elegant,
it
because
a number
solves
it efficiently
because
because
be appreciated on
may
algorithm
grounds
of
manner to other
same
book:
An
be
can
Fourier Transform
Fast
The
trick
The
algorithm.
computing
9.6
and
all possibilities
We also demonstrated
a better
lead to
can
be
We
word. For
in one
number
we
the
is 0(A23/log2Ai).
is needed.
what
only
computation
assume that
two
of computing
method
interesting
of computing
wisdom
their sum).
time
We presented
Comments
22m =
then
[log2\302\253/2j,
computer
the
execution
the
add
the corresponding
m-bit numbers and fetching
of the computer
word is at most 2m.
for the regular four-Russians algorithm,
of the algorithm.
If we select m to
tables
the
during
taking
two
the size
as
constructing
J able to
contains
(which
time, as long
equal to
of m,
choice
this
AddTable
in
Add
309
ax
be specified
one
function
and a0.
by
any
Algebraic
310
and Numeric
Algorithms
of degree n is
on that line.
In the same way,
any
polynomial
the
n
+1
For
example,
second-degree
polynomial
by
points.
uniquely
and it is the only
p(jt)=Jt2 + 3jt+l is defined by the points (1,5), (2,11), and (3,19),
are not the
all those points. These three points
second
degree polynomial that includes
on
the
three
that
define
this
points
corresponding curve
polynomial; any
only three points
two
points
(nonequal)
defined
will
do.
This
representation
of points is
is
for
attractive
multiplication
polynomial
because
multiplying
values
of two
we can compute the
scalar multiplications!
Using this idea,
product
in
this
with
of
O
az,
(n)
degree given
only
representation,
multiplications.
polynomials
with
this approach is that
we
cannot
The main
problem
simply change the
to
fit
one
We
must
be
for
able,
only
application.
example, to evaluate the
representation
at given points. This is much
than it is
harder
to do for this
representation
polynomial
five
coefficients
are given. However, if we could convert
from one
when
the
efficiently
we would have a very
then
representation to another,
good
polynomial
multiplication
algorithm. This is what the FFT achieves.
can be done by polynomial evaluation. We
Converting from coefficients to points
can compute
the value of a polynomial
at any given
p (jc), given by its list of coefficients,
We need to evaluate
5.2) using n multiplications.
point by Horner's rule (Section
p (jc) at
n arbitrary points, so we require n2 multiplications.
from
points to
Converting
here (as
idea
so
in
many
transform
Fourier
can be
interpolation,
free to
choose
a very special
it also
in
any
set of points
that both
such
steps,
and
evaluation
done quickly.
Transform
Fourier
Forward
The
and
other examples
first
the evaluation
consider
polynomial
arbitrary
polynomial
the
P =
n-\\
a{x{
\302\243
polynomials
matrix terminology
P above for the n
of degree
n-\\
are easy to
use
to
points
n distinct
evaluate.
simplify
x0,xu
at
the
..., xn_x
We
assume,
notation.
can
We
points.
want
to
for simplicity,
The
be represented
evaluation
as
the
find
that n
of
following
the
9.6
matrix
vector
by
311
Fourier Transform
multiplication:
n-\\
(x0Y
The question is
this
can
we
P(x{)
(x{y
n-\\
P(*n-\\)
C*\342\200\236_i)J
choose
the values
of
jc0,
Consider
multiplication.
P(x0)
<*0
n-\\
(Xn_\\)
Xn_i
whether
(x0y
Ui)2
*i
simplifies
The Fast
...,
jcj,
and
j.
in
xn_\\
would
We
that
a way
like to
as similar
as possible to save multiplications.
We
make them
cannot
make *,=*,-,
because the values must be different, but we can make (jc,)2= (jc7)2 by letting Xj = -xh
This is a good choice, because every even power of x{ will be equal to the same even
of the multiplications
of
We may be able to save one-half
involved with row j.
power
Xj.
of
do
same
for
other
rows.
Our
is to have n special
we
can
the
Furthermore,
pairs
goal
above
nil
rows for which the computation
vector products. If we can do
requires only
that,
we
then
efficient
may
algorithm.
of half
the
We
Let's
try
cut
to
the
size by half,
problem
this problem
pose
in
terms
which
of two
will
lead
to a
very
separate subproblems
size.
want
to the
according
be able to
scheme
*i
n-\\
(xoY
(x0y
C*i)2
(x{y
n-\\
P(x0)
<*0
P(x{)
1
xni2-\\
(Xnl2-\\)
i.Xnl2-\\)
n-\\
(9.7)
-x0
i-x0f
~X\\
(\"*l)2
(-x0y
n-\\
P(-Xn,2-\\)
0*i-l
1
The nxn
~Xn/2-\\
(-Xn/2-\\)
* *
n-\\
(-Xn/2-\\)
The
\"even\"
matrix
in (9.7)
are
polynomial
can
(\302\243)
be
written
as a regular
polynomial
of
degree
nil-
312
the even
with
Algorithms
coefficients of P:
/i/2-1
\302\243=
The
a2i(x2)l=Pe(x2).
\"odd\"
(O) can
polynomial
be
same way:
in the
written
n/2-1
0=x
a2M(x2)l=xP0(x2).
2
/=o
we have
overall,
So,
the
expression:
following
P(x) = Pe(x2)+xP0(x2\\
where
powers
To
are the
(P0)
Pe
of P.
-jc for x
in
(9.8),
polynomials
degree
substitute
we
When
(9.7), we
evaluate
n/2-
(9.8)
and
need to compute only n/2 values of Pe(x2) and n/2 values of P0(x2), and to perform az/2
and n multiplications.
of size
So, we have two subproblems
additions, n/2 subtractions,
additional
O
and
nil,
(n)
computations.
with
the same scheme recursively? If we could, then
Can we continue
we would
=
+ 0(n), resulting
in an 0(n\\ogn)
T(n)
lT(nll)
get the familiar recurrence relation
of computing P(x) (a
the problem
algorithm. But this is not so easy. We reduced
of degree
n- 1) at n points to that of computing Pe(x2) and P0(x2)
(both
polynomial
of
n/2at
n/2
is
This
a
valid
one
for
reduction,
1)
polynomials
degree
points.
except
small thing.
The
of jc in P(x) can be chosen arbitrarily,
values
but
the values of jc2,
which
are needed,
for example, in Pe(x2),
this
can only be positive. Since we obtained
reduction by using negative numbers, this poses
a problem.
Let's extract from (9.7) the
matrix
that corresponds
to the computation
of/^((jc,)2):
ri
1
If
we
squares
try the
numbers
let
= V-1
xj+n/4
c*o)4
c*0r2
*0
t*i)2
C*i)4
C*i)\"-2
a2
a4
C*/i/2-l)
C*/i/2-l)
same
on
trick
complex
first
c*o)2
partition.
xj9
0<y
O (n)
to
V-1.
We
solve the
a number
=
and z2 V-1).
z such
In
again
general,
partition
problem
of size
1, and
we need
Pe(xn/2-\\)\\
set
the
n/2
we
zj * 1 for
a number
not
impossible
that
into
two
same properties
by
need
solving
two
a number
0 <j
satisfies
Since
(jc\342\200\236/4)2
=-(jc0)2.
the problem
divide
satisfies
that z8 =
But it is
impossible.
This
Pe(*\\)
we need to
subproblem,
seems
PeiXo)
an-i
(xn/2-\\)n
additional computation.
carry this process one step further,
that is,
z4=-l,
< n/4.
Hence, we can
this
include
which
for
this
' '
\"
<8
(which
that
if we
use
parts
and
as did
the
subproblems
is equal
implies
the condition
to
that
above
9.6 The
for
n rather
denoteit
n throughout
co\"
do
= 1,
this
co
1
we
simplicity;
will
313
of unity.
use
conditions:
(9.9)
are 1, co,
or
co
2-2
CO
co\"-1.
co2,...,
Therefore,
*0
P(D 1
CO\"\"1
a\\
/>(co)
CO2
We
the same
for0<j<n.
root
nth
primitive
for
include
not
section.)
and o7^
a number is called
n in the notation
satisfies the following
Such
8.
for
than
by co. (We
Fast FourierTransform
^\342\200\242(n-l)
we
want
/>(co2)
=
This product is
co\"-1
called
co(\"-1)2
CO'|(n-l)-(n-l)
/Mco\"-1)
an-\\
Fourier
indeed for
algorithm
is presented
depends
Output:
(az, a0,
Fast_Fourier_Transform
Algorithm
Input:
9.7.
in Fig.
(an integer),
aQj a \\,..., an_x
on the application), and
V (an array
in
the
| we assumethat
a x,...,
(a sequence
CO
mh root
(a primitive
- 1 ] of
range [0..n
is a power of 2 )
co, var
an_x,
of elements
of unity).
elements
output
).
begin
ifn=
then
V[0]:=a0;
else
a0a2,...,an-2>to2>U)>'
Fast_Fourier_Transform(n/2,
ax
a3,...,
Fast_Fourier_Transform(nl2,
(9.8)
{follow
forj:=0ton/2-\\do
for
V[j]:=U[j]
an_x,
Jt =
co2, W
co7
+ rtW[j];
V[j + n/2]:=UV]-rtWlj]
end
Figure
9.7 Algorithm
Fast Fourier
Transform.
V);
whose type
);
and Numeric
Algebraic
314
Algorithms
D Example 9.2
We
to compute
we
how
show
denote
transform
Fourier
the
where
jk(x0,xj,...,xk),
and jc0,x{, ...,xk denote the
the coefficients
of the polynomials,
denote
JoJi'\342\200\224'Jk
for which we need to evaluate the polynomials.
So, in particular, this
values
example
notation
is quite awkward,
but
co,co2,\342\226\240\342\200\242\342\226\240\302\273co7).
(This
involves
computing /*(),1,2,3,4,5,6,7(1*
we need.)
The main recurrence we use is (9.8).
it contains all the information
To
confusion,
avoid
first
The
co2,co4, co6).
Px 3,5,7(1,
and
P0,40\302\273co4)
4.
We
Pjojxt
co7)
t0
co2, co4,
^0,2,4,60.
and
CO6)
reduce /^^O'co2,
co4,co6) to
recursively
tnen
to
reduced
which
is
P0U).
clearly 0, and
^0,4(1. co4)is
can now combine
the results to get
and
continue
We
co4).
P2,6(U
is clearly
which
P4(\\),
by
subproblems
^0,1,2,3,4,5,6,7(^ co,co2, -.
reduces
step
the
4,
and
P0 4(co4)
Since
co4
manner,
We
= />0(co4)+ co4/>4(co4)
= 0
+ co4\342\200\242
4.
= -1,
we get P04(to4) = -4, and,
overall,
co4) = (4, -4).
P0,40.
we get /^(l* co4)= (8, -4).
now
combine
the two vectors aboveto compute
co2, co4,
Po,2aA^
PoiAfii\302\2562)
/>0,2,4,6(C04)
Po,2aA<\302\2736)
^o,4(co4)
+ co2\342\200\242
/>2,6(co4)
/>o,4(C08)
+ C04-/>2,6(C08)=
+
^0,4(C012)
= -4
\342\200\242
CO6
same
co6):
= 4 + 8=12.
= />o,4(l)+l-/>2,6(l)
/>o,2,4,6(l)
the
In
/>2,6(C012)
+ co2(-4).
/>0,4(C04)
CO2
\342\200\242
= ~*
/>2,6(C04)
~ ^HO-
So, overall
same
In the
CO4, CO6) = (
CO2,
^0,2,4,6(1,
P 1,3,5,7(1,
co2,co4,co6)= (
To
compute
= 12-
co, co2,...,
1-16=-4; P0t
in
the
+co2)) +
= (-4(1
1,2,3,4,5,6,7(co)
we need to
co7),
12+ 1 \342\200\242
16 = 28, and,
= (-^4(1+co2))- co-(-4(l+
manner, ^0,1,2,3,4,5,6,7(co5)
to the reader.
The
Fourier
Inverse
The algorithm
for
the
evaluate
the
multiply
the resulting
two
).
16,-4(l+co2),-4, -4(1-co2)).
/>o,i,2,3,4,5,6,7(l\302\273
P 0,1,2,3,4,5,6,7(1)=
example,
find that
we
way,
12, -4(1+C02),
given
same
compute 8
manner,
P0,i,2,3,4,5,6,7(co4)
co-(-4(l +co2)),and,
co2)),
and
so on.
For
values.
in
the
We leave the
same
rest
\342\226\2
Transform
fast Fourier
values, and
solves
transform
polynomials
find
p{x)
and
the values
q{x)
of
only half of
at the points
the
product
our
problem.
1, co,...,co\"\"1
polynomial
p(x)-q
We
can
quickly,
(x)
at
we still need to
But
those
points.
from
vector
the
Fortunately,
the
and we
polynomial,
points 1, co,co2
at the n
algorithm
to
want
, we
co\"
can
we are
When
compute
out
to
solve
it.
values of
the
matrix
be very
coefficients
the
given
compute
the
polynomial
product
turns
problem
interpolation
notation.
matrix
of the
coefficients
problem,
the
...,0\342\200\236_i)of
(a0,ai,
polynomial
the
interpolate
315
Transform
Fast Fourier
The
9.6
by
the
product:
co
CO\"
2
CO
oy
On
the
(v0,
v j,...,
other
system
following
2-2
co2'\"\"\"
oy((n-l
and we
are given,
v\342\200\236_!)
co
CO\"
of
want
/>(0)2)
)\342\200\242(/.-!)
/>((0\"_l)
an-\\
the
(fO),
f^co), ...,/> (cow_1))=
the coefficients, we need to solvethe
polynomial
to compute
a0, a \\,...,0\342\200\236_i:
of equations for
\"
v0\"
\"tfo\"
o)\"-1
2-2
CO\"
P(l)
a)\"\"1
the values
when
hand,
co(\"_1)*2
co\"\"1
Vl
tfl
v2
a)2c-|>
CO
1 CO\"\"1
Solving
case),
(vo.
where V(co) is
Tne
\342\200\224\302\273
vi\302\273
vn-\\)-
V(co) has
inverse
an
has a
simple
very
of
the
matrix
for a
form
out
that
(we omit
011-1
.
time
quite
in
the
left
can be written
V(co)
the
v\342\200\236-i
-
consuming
Let's write
equations.
always
(9.10)
(0(az3) for
this
matrix
the
general
equation
as
as
proof):
9.1
\342\226\241
Theorem
[V(co)]\"1 =
usually
solution
It turns
inverse.
oy((n-l)(n-l)
CO*\"\"02
systems of equations is
but this is a special system
V(co)-a=v,
v =
the
following
- V(-).
co
\342\226\2
to solve
Therefore,
by vector product.
Theorem
\342\226\241
If (d is a primitive
unity.
only
matrix
one
9.2
nth
root
of
unity,
then
1/co is
also a
primitive
nth
root
of
\342\226\2
and Numeric
Algebraic
316
Therefore,we
the product
V(l/(fl)v by
substituting 1/(0 for co. This transform
can
compute
transform,
Fourier
Algorithms
the
using
for
the inverse
algorithm
is called
fast
the
Fourier
transform.
Notice
operations.
0(n\\ogn)
product of two
that we need to
the
Overall,
Complexity
be
computed with
complex
and multiply
add
to
able
be
can
polynomials
numbers.
9.7
The
Summary
numerical
We have
algorithms.
Strassen's
best.
the
lecessarily
in this
presented
algorithms
nonintuitive
algorithm
examples of
the
for
chapter are a
seen again that
a seemingly
and,
of induction,
use
is
algorithm
small
one
known
straightforward
of the most striking
simple problem.
in
of
sample
the
particular,
of the
use of
of
examples
seen
have
We
algebraic and
algorithms are not
a
more
several
divide-and-conquer
algorithms.
The
on
induction.
10.
Chapter
Notes
Bibliographic
The
include
Aho,
[1980],
for arithmetic
source
best
and
and
Hopcroft,
and
and
probably
algorithm
is Knuth
Munro
and
[1975],
[1981].
Lipson
schemes
encryption
algorithm
algorithms
algebraic
powers by
writings circa 200 B.C. (seeKnuth
scheme
is due to Rivest, Shamir,
encryption
Reading
key
Further
repeated
squaring
is very
old;
it
was introduced
in Euclid's Elements,
first
appeared
it
was
Ofman
the
context
of multiplying
discussion on
Roth [1976].
the
The four-Russians
of
the
Myers
in
and
Cohen
be found
317
four-Russians
probably
Paterson
algorithm can
Strassen's
Exercises
is due
algorithm
The improvement
[1970].
of
implementation
Drill
Exercise
of
solution
(Masek
algorithms
9.26 appears in
comparisons
sequence
[1988]). The
and
and
Atkinson
showed a reduction
between
Boolean
matrix
[1988]. Fischer and Meyer
[1971]
and
the
transitive-closure
multiplication
problem.
The algorithm
for the fast Fourier transform
was introduced
by Cooley and Tuckey
can be traced to Runge
and Konig
[1965],
[1924]. For
although the origins of the method
more information
on the fast Fourier transform,
see Brigham
and Rao
[1974] and Elliott
Santoro
[1982].
Exercises
Drill
9.1
the
Discuss
computing
9.2
9.3
Let x be a positive
Design an algorithm
(a,
then
9.4
not
of computing
example
necessarily
nk
(k>
10)
= a/b.
number that is represented
by the pair (a, b) such that x
of x\\ that is, the representation
to compute the smallest representation
b) with the smallest possible values of a and b. For example, if x = 24/84 = 6/21 =2/7,
of x.
(2, 7) is the smallest representation
Prove
that
nk does
computing
that
the
computes
as
operations
rational
straightforward
all
does
four
the straightforward
algorithm
follows
that
polynomial
makes
(9.1).
multiplication
exactly
Assume
the same
that
n is a
power of 2.
9.5
the
Find
P(x)=x
algorithm
+ 2x2 + 3x* +
\302\243)(.*:)=16+15*
How
9.6
P(x)-Q(x\\
presented
product
multiplication
+14*2
many operations
hand,
Section
using
9.4.
the divide-and-conquer
polynomial
\342\200\242\342\200\242\342\226\240
+
15jcl\\
\342\226\240\342\200\242\342\200\242
+ 2x,4
1jc'\\
are required
A divide-and-conquer technique
and discuss
an algorithm,
such
algorithm.
by
in
overall?
can
the
be used
differences
to
multiply
between
318
and Numeric
Algebraic
9.7
9.8
way to
all the
Design
an
Derive the
in
9.11
two numbers
4679x7114.
base b (not
the
recursion
Carry
in
any
with
only three
scheme
described
(a+bi)(c+di)
of -1.)
root
2x2
for Strassen's
matrix
multiplication
9.5.2.
Section
Suppose that
you
be the
would
square
expressions
explicit
by 10111010.
10011011
multiply
to multiply
(/ is the
9.6 to
Exercise
numbers.
1 -digit
algorithm
multiplications.
9.10
discussed in
divide-and-conquer
technique
b = 2).
Use it to perform
only
down
9.9
Algorithms
complexity of a
What
algorithm?
find an algorithm
value
matrices
4x4
multiply
matrix
general
maximal
is the
to
multiplication
will lead
of k that
to
over Strassen'salgorithm?
Exercises
Creative
9.12
Consider
the
two
for computing
algorithms
Let n
be an
nk
in Section
given
Assume
d digits.
and
repeated squaring).
integer
multiplications
integer
are performed by the regular algorithm, which requires J, \342\200\242
two integers
d2 steps to multiply
with dx and d2 digits.
What
is the number of steps required
to compute
nk by the two
algorithms? (You can assume that k is a power of 2, and that a product of two integers with
with d]-\\-d2 digits.)
dx and d2 digits is another
integer
9.13
9.14
9.15
9.16
to
an algorithm
Design
is a multiple
The Fibonaccinumbers
a.
Prove
of m and
integer
LCM of k given
of all of them.)
> 2
is the
integers.
is a
that
multiple of
integers.
given
(The LCM of
integers
is the
recurrence relation:
by the following
can be
smallest integer
LCM of two
every
that
of k integers.
to find the
are defined
F(2)=l,
that
(LCM)
algorithm
to find the
Design an algorithm
smallest integer that
F(\\)=\\%
the GCD
find
with
(n>2).
written
as
a sum
of
at
most
\\og2n Fibonacci
numbers.
b.
9.17
an algorithm
Design
Let P(x)
to
find
such
a representation
for a given
number
n.
and
exists
a.
Show
b.
Extend
that
the GCD
Euclid's
of two
algorithm
polynomials
to
find
is uniquely
the GCD
defined.
of two given
polynomials.
319
Creative Exercises
9.18
the
Modify
multiplications involving
of 3. What is the
smaller
complexity
power
9.19
the
Modify
polynomial
multiplication
into four equal parts, and
polynomial
9.20
Hamilton's
real
and
i, j,
componentwise, and
= -kj
ki
= -ik
/,
symbols
j\\
a. Prove that
b.
= j
P[i]
Considerthe
by
required
a, b, c, and d are
subtract
quaternions
where
and
is an
x n
with
themselves).
the ordinary
procedure
and
the number
reduces
matrix such
each
of two permutation
to
The outcome should
multiply
also
many
quaternion
to 12.
each column
be represented
row and
is equal to 1. A permutation
matrix can
the ith row contains a 1 in the jth column.
if
How
for
with
that
of multiplications
that
the product
representation.
matrices is another
permutation
has
exactly
by an array
matrix.
two permutation
matrices given
be given in an array representation.
by
the
array
following
to
algorithm
9.24
that
a linear-time algorithm
Design
+ cj + dky
a+bi
form
the
numbers are
matrix
such that
of
to compute
permutation
of
problem is a
size of the
are
k commute
and
Give an algorithm
each
= i
= j
multiplication?
9.22
the
9.4 by dividing
the number
= -l
of real
Show how
minimizing
multiplications
9.21
9.23
and
multiply
/' = y2 = k2
ij = -ji = k
jk
that
Section
(instead
are vectors
quaternions
numbers,
(the
algorithm described in
of two),
and
can
You
assume
that
parts.
of the algorithm?
polynomial
multiplication
into
three equal parts
polynomial
compute
Design an
of size k
9.25
the program
Complete
tables
explicitly,
significant
9.26
Design
submatrices
such
should
and
for
matrix multiplication
to handle the case where
Boolean
how
(Fig. 9.6).
Show
is not an
n/k
how
to build the
integer
without
loss of efficiency.
an algorithm for Boolean matrix
multiplication
of size kxk, and uses the idea of precomputing
submatrices.
require extra
operations on
numbers
of
the
space of 0(n\\ogn).
with up to
bits
log2\302\253
in one
can
assume
step.
all possible
should
algorithm
You
that
into
products between
be 0(w3/(logw)15),
that you can perform
and
it
basic
320
9.27
Let A and B
chosen
be two
(independently)
the product of A and
9.28
nxn
Algorithms
random
to be
By such
either
that the
0 or 1 with
expected
probability
number
entry
in each
matrix is
randomly
of operations
to
find
will be 0(n2).
Let
A and
2.9)
matrices.
be n matrices of real numbers.
The dimensions of A/, are fl,Xfl/+1, so
9.29 Let Af M A/2> \342\200\242\342\200\242\342\200\242\302\273A/\342\200\236
is defined
for each 1 <i <n.
We want to compute the product
the product of A/,-A//+1
Let's
assume
that it takes af-a/+1 a/+2 operations
to multiply an
MlxM2'x '\" xMn.
matrix
The problem is to find the right order in which to
by an ai+l xai+2 matrix.
fl,-Xfl/+1
For example, let n = 3, and let the matrices be of dimensions
carry out the multiplications.
and 5x3. Finding
the product
of the first
two
matrices
takes 10-2-5
2x5,
10x2,
in a matrix of dimensions
10x5.
resulting
Finding the product of this matrix
operations
with the third one takes 10-5-3 operations \342\200\224
250 operations. On the other hand,
if
overall,
we first find the product of the last two matrices and multiply
the first matrix with
that
an algorithm
to find the optimal order
product, we end up with only 90 operations. Design
out the matrix product above.
of carrying
10
CHAPTER
REDUCTIONS
is of two kinds.
a subject ourselves,
know where we can find
Knowledge
know
We
or
we
upon it.
information
Samuel Johnson,
Introduction
10.1
We
1775
this
start
chapter
with an old
joke.
mathematician
and
her husband
to boil water,
and you want
\"Suppose that you are in the basement
following
question:
and boil
The mathematician
what do you do?\"
says that she will go up to the kitchen
asked
the following
Now they are both
answers
water there; her husband
similarly.
what
do you
and you want to boil water,
question: \"Suppose that you are in the kitchen
\342\200\224
the water.\"
boil
I'll just fill the kettle and
do now?\"
husband
The
says \"it's easier
basement
I'll go down to the
The mathematician answers \"it's even easier than that \342\200\224
to solve that problem.\"
and I already know how
will
show that
we will concentrate
on the idea of reduction. We
In this
chapter,
is
another
useful.
Here
reductions
can be extremely
besides being funny
sometimes,
a package
send
time
a real one. When you
of a reduction, this
by Federal
example
New
York City, the package will be
New York City to downtown
Express from uptown
all packages
through Memphis, so
of delivering packages across town
they
In this case, the solution
makes
sense. It
know how to solve the problem.\"
\"already
a special situation and
to identify
to build
a mechanism
to
may be much more difficult
more efficiently. It may be easier, and overall cheaper, to handle
situation
that
handle
true
in algorithm
This is also often
design. When we encounter a
equally.
everything
routed
when
through
they
are
Memphis.
faced
with
the
special
situation
i^
322 Reductions
whose solution is already
problem,
posed as a specialcaseof another
a solution
solution can be used. Such
may sometimes be too
a
solution
is the the easiest,
But
in many
or too expensive.
cases,
general
using
general
use
this
We
solution.
most
to
a
and
the
fastest,
the
principle
every day.
elegant way
get
\342\200\224
\342\200\224
it is usually
a database query
for
not
For some computing
problems
example,
it
is
to
this
sufficient
a program
that solves only
use
problem;
necessary to write
more
The
that
handles
software
general
problems.
general-purpose
general-purpose
but it is much easier to use.
not be the most efficient solution,
solution
may
P
that
seems
but that also seems
we
are
a
t
hat
complicated,
problem
given
Suppose
to
solve
P
from
or we can try to
We
can
to a known
scratch,
similar
problem Q.
try
them
to
P.
to
solve
and
There
of
the
methods
used
a
some
is, however,
borrow
Q
apply
the
two
to
between
We
can
find
a
reduction
third way.
(or transformation)
problems.
try
of one problem using
a \"black
box\" that
is a solution
Loosely speaking, a reduction
can achieve one of two goals depending on the
Reductions
solves the other problem.
black
box is used to solve which
in which they are done (i.e., which
direction
problem).
into an algorithm
for P if we
A solution of P that uses a black box for Q can be translated
for Q. On the other hand, if P is known
to be a hard problem, or, in
an algorithm
know
if
a
same
lower
bound
be applied to
we
know
lower
bound
for
then
the
f\\
may
particular,
is used to obtain information
about
in
f\\ whereas,
Q. In the former case, the reduction
about
the latter case, it is used to obtain information
Q.
For example, in Section
we discuss the problems
of matrix multiplication
10.4.2,
and
matrix
the
matrix
with
itself).
(i.e.,
squaring
multiplying
Clearly, we can square a
matrix
with
a matrix multiplication algorithm;
the problem of matrix
therefore,
squaring
can be reduced to the problem
of matrix multiplication.
We show in Section
10.4.2 that
it is possible to multiply
two
with the use of a matrix
matrices
algorithm;
squaring
is reduced
to matrix squaring.
The
of the latter
therefore, matrix multiplication
purpose
reduction
is to show that
the square of a matrix
cannot
be done faster (by
computing
can be
that
problem
more
that are
We
see
will
between
reduction
lower bounds
on
problems.
problem
than computing
a constant)
than
conditions
both
known
the
then
known,
discussed in
several
the
An effective
examples
is useful
of the
The reduction
may be
of
For
it.
enough
have
arbitrary matrices
some
(under
example,
the
of reductions
use
even
does
if it
used to
find
new
this
Finding
to new upper
helps us to understand
for attacking
techniques
chapter.
directly
reduction
The
problem.
in
lead
not
be used to
or
the
a parallel
design
problem.
way
to
use
of two
product
10.4.2)
of
two problems
the complexity
or variations
algorithm for
the
Section
reductions
Finding
10.3.
is to define a
a general
a wide variety
solution. We
cover
an efficient
such
of
general problem
problem is
easy.
but it must
problems,
discuss one
not
such
problem,
to
which
This
many
problem
also be simple
called
linear
programming,
We have already seen severalexamplesof reductions
for example,
in this book \342\200\224
the reduction of the transitive-closure
to
the
problem
all-pairs shortest-paths problem
are important
(Section7.8). Reductions
enough, however, to deserve a special chapter.
are also the cornerstone of the next chapter.
Reductions
in
Section
Examples of
10.2
10.2
In
of Reductions
Examples
this
we
section,
Reductions 323
present
four
of
examples
reductions
using
to
efficient
obtain
algorithms.
A Simple
10.2.1
with
start
We
The
two
a simple
ai=b(k+i)modn
and
matching),
of the
variation
determine
0</</?-l.
'\302\273
we call
the
solve
and
an_x
B is
whether
We
call
string-matching
the
modifying
a cyclic
this
\342\200\242\342\200\242\342\200\242
B=b0bi
be
bn_x
of A.
shift
an index
exists
there
original
\342\200\242\342\200\242\342\200\242
whether
f\302\260r
aU
problem.
string-matching
Let
Problem
A=a0a{
of
sizen.
Determine
strings
The problem is to
Problem
String-Matching
such
0<k<n-\\,
ky
CSM
problem
(for
cyclic
that
string
can
was
CSM
10.2.2
Let
S i,
a set R
R
to
regular instanceof
S2,
way
to arrive
a collection
of sets. A system
of distinct
(SDR) is
representatives
that, since we require
r, e 5,, for all /, 1 </ <ky (notice
R includes exactly one
be distinct). In other
words,
\342\200\242\342\200\242\342\200\242>
Sk be
r2,...,
a set, the
{r j,
be
better
In other
SM.
of Distinct Representatives
Systems
=
is a
there
representative
such
must
/*,s
each
from
collection of
rk
sets. For
{2,3,4},
53 = {1,3},
set.
that
It is
example,
an
not
to
possible
always
SDR for
the
an
find
of
collection
= {1,2}, S2 =
of sets
The Problem
for
There is
the
collection
very
conditions for
the
elegant
existence
Given
(any SDR
a finite
will
collection
do),
of
or determine
finite
find
sets,
that none
gives
number
of a given
SDR
sets Sx
the
collection
an SDR
exists.
and sufficient
necessary
of elements of S.
324 Reductions
Theorem
Hall's
\342\226\241
Let S i,
S2,..., S*
be
card
It is
are
bipartite
vertex v,
in V
each
for each
Consider
the
there
of
the
if
and
size k.
Involving
\342\200\242
\342\226\240
\342\200\242
bm are
it becomes
\342\200\224
insert,
an
three
different
Another
Each
entry
elements,
apply
the proof
way
is also
condition
be
cannot
they
unfortunately,
subcollections, and
pose this problem
and
network
discussed
the algorithm
of Hall's theorem
there
as
in
Section
be obtained
can
flows.
Sequence
problem discussed
two strings of characters,and
Comparisons
in
we
6.8:
Section
want
A -ax
to edit
A,
a2
'''
an
character
by
delete,
edit steps.
to look at this
in the table
less than
altogether
the
exercise.
our
goal is to minimize
construct a table of size n by my where each entry corresponds
contains
the cost of editing
the first / characters of A into
entry
The
is
thus
to
the
goal
compute
\"bottom-right\"
entry (nm) of
each entry can be computed from only three other \"previous\"
the
with
That
all possible
The idea is to
can
We
matching
sequence-comparison
until
character
every
distinct
Furthermore,
of bipartite
A Reduction
character,
union
G of
in
10.2.3
b2
G=(V,
sets
to check
approach.
Let
problem.
are m
there
representatives.
it as
have
will
uj) e E
is, (v,,
if
set Sh and
in the
element
B=b{
leave
we
We
problem.
containing it;
is simply a matching
7.10 to solve this
from the properties
one
since,
have m distinct
is necessary
cannot
We need another
that
and
other
m
\342\226\
efficiently.
them.
2k of
contain
must
sets
to prove, and
theorem provides
matching
for
they
checked
directly
2, 3,..., k).
altogether at least
In
of {1,
\342\200\242\342\200\242\342\200\242\302\273'wl
{\302\253i,/2\302\273
condition
then
Hall's
words,
>m
us,
is harder
sufficient
if and
\\<m<k.
for every
m elements,
SDR
is satisfied:
\342\200\242\342\200\242\342\200\242
sf us,-u
a collection
condition
problem is by considering
in the graph.
corresponds to a vertex
to a
the first
the
We
table.
showed
that
entries corresponding to
the table
as a directedgraph.
A vertex
thus
corresponds
to
edit. There is an edge (v, w) if the partial edit corresponding to w has one more
edit step than the partial edit corresponding to v. An example
of such a graph is given in
the
10.1, where A =caa and B =aba.
The horizontal edges correspondto insertions,
Fig.
vertical edges to deletions, and the diagonal
edges to replacements. For example, the
shaded
of b,
of c\\ a match of a, an insertion
path in Fig. 10.1 correspondsto a deletion
and another match of a. In the basic problem, the cost of each
for
is 1 except
edge
to equal characters (i.e., no
diagonal edges that correspond
replacementis necessary)
a partial
10.2 Examples
to the
corresponding
graph
325
Reductions
of
sequences A
and B
-caa
-aba.
now becomes
a regular
0. The problem
single-source
shortest-paths
Each edge is associatedwith a cost (which is the cost of the corresponding edit
We
m].
path from vertex [0,0] to vertex [\302\253,
step), and we are looking for the shortest
to
the
have reduced the string-edit
problem
single-source sortest-pathsproblem.
than solving this problem directly.
shortest
paths in general is not easier
Finding
of
is
variations
useful.
for
this
reduction
Consider,
Nevertheless,
example,the following
the sequence-comparison problem. The cost of editing is not necessarily
per character.
of characters in the middle of another string
not be the
The cost of inserting
a block
may
same as that of inserting the same number of characters,one by one, in different places.
The same may be true for deletions. In other words, instead of assigning a cost per
we may want
to assign
a cost per blocks of
and replacement,
deletion,
insertion,
of
their
sizes.
we
or
deletions, regardless
insertions,
Alternately,
may want to assign a
/
is
the
where
cost of say, I+ck, for inserting a block of k characters,
cost,
\"start-up\"
metrics.
We can
character.
There are many other useful
and c is a cost per subsequent
the
formulation than by modifying
model them more easily by using the shortest-path
original problem. We can add edges anywhere we want and assign any cost to them,
whose cost is
problem.
without
the problem.
changing
10.2.4
Finding
strong correlation
can be represented by
is a
There
vertices
and
is a weighted
graph,
to
of edge
to
the
weight
associate
G =(V, E)
and
only
ith
vertex
and
matrices.
matrix
A,
adjacency
if (v,, v7) e E. If G
we define A as an n x
(v,, v;) or to
with
n vertices
with
if the
graphs
its
only
then
a matrix
between
0 if
graph.
to
is undirected,
n
matrix
and m edges is
is incident
the
an
jth
A graph G=(V,
is an n xn matrix
which
n xm
edge.
then
such
in the
the
matrix
that the
graph.
incidence
in
A is
which
E)
symmetric.
ijth
entry
with
in which
If G
is equal
ways
of a graph
the ijth entry is 1 if
matrix
Reductions
326
The correlationgoesbeyond
be
understood
better
Here is one
n vertices
designan
use of
making
by
this
analogy.
running
graph
with
determine
whether
subsets of
vertices,
of the
time
whose
Can we
is sparse.
three
of
subsets
algorithm
an algorithm to
undirected
is to check all
1)(a?-2)/6
so the
be a connected
(V, E)
Design
edges.
solution
straightforward
constant time,
graph
and
G =
Let
contains
(^) = n(n-
be resolved
example.
The Problem
The
by
problems can
algorithmic
many
surprisingly,
representation.
at the
can be discovered
of matrices
properties
mere
looking
by
resulting
time is 0(mn)
running
do better than
and each
subset can
algorithm
is 0(n3).
are
be
checked
in
It is
(Exercise7.118),which
to show
We proceed
that?
There
vertices.
three
an
possible to
if the
is better
is
which
algorithm,
A2[iJ]=idA[iAYA[kJ].
if
of the
[ky
and
j ] are
1.
and
k*j,
terms
In
both / and
and
if there
only
graph,
j are
[/,
exists an index k
j ] > 0 if there exists
to
connected
k.
(We assume
a vertex
ky such
the
graph
that
and
[/, k]
* /,
not
does
that k
that
means
that there exists a
/]=0 for all /.) However,
if / is connected to j and A2[iy j] >0. Thus, there
triangle
involving
only
2
exists a triangle
in G if and only if there
are i and j such that A [/, j ] = 1, and A [/', j ] > 0.
The
discussion
above
A2 and then check
implies an algorithm. We first compute
the
condition
above for each pair / and j. It costs O (n2) to check all pairs,
so the running
time of the algorithm
is dominated
of
matrix
the
time
multiplication. We
running
by
have thus reduced
the problem of finding
to that of Boolean matrix
a triangle
in a graph
contain
self
loops;
two
problems
and
multiplication
0(az281).
We
multiplication,
of
precisely to matrix
are equivalent).
(more
multiplication
these
hence, A [/,
/ and j if and
obtain
also
can
and
an
obtain a
0(n3/(\\ogn)2).
We
use
for finding
algorithm
in
practical algorithm
for
have
multiplication,
so, in
the complexity
of Booleanmatrix
general,
the
reduced
complexity
multiplication.
this
of
will
see
in Section
use Strassen's
now
can
We
algorithm
the
but we
squaring,
triangle
Section
finding
9.5.3
a triangle
for
algorithm
whose
10.4.2
running
for Boolean
with
a running
that
matrix
time
is
matrix
time
matrix
is
10.3
Reductions
10.3
Reductions
Involving Linear
Programming
327
areas of algorithm
examples of reductionsfrom different
one
to
that
we
could
another
so
use
a
known
design.
problem
algorithm.
This section also presents reductions,but with a slightly different approach. Instead of
for a candidate for a reduction whenever a new
we explore some
arises,
looking
problem
4'super-problems,\"to which many problems can be reduced. One such super-problem,
the most important
is linear programming.
There are efficient
one,
perhaps
algorithms
for solving linear programming, although
are not simple. A thorough
discussion
of
they
linear programming is beyond the scope of this book. In this section, we only define
and show several examplesof reductions
some variations
of the problem,
to it.
The
section
previous
tried
We
included
to map
Introduction
10.3.1
and Definitions
a certain
are
function
many
problems that involve maximizing or minimizing
involves maximizing
subject to certain criteria. For example,the network-flow
problem
flow
constraints
and to the conservation
the
function
constraints.
subject to the capacity
in
Linear programming is a general formulation
of such
cases
where
the
problems
and the constraints
can also be written
linear
functions
function is a linear function
using
of variables. An objective
in
the
be a vector
following
way. Let x = (x\\,x2, \342\200\242\342\200\242\342\200\242,-*\342\200\236)
is defined as a linear
function
the variables of x;
function
involving
There
c(x)
^CjXh
where thee,
s are
constants.
(10.1)
some constraints
is to find the values of x that
of linear programming
satisfy
function.
We shall see later that, if
the value of the objective
(listed below) and maximize
with a similar minimization
it is easy to replace the maximization
objective
necessary,
with
of
three
we
a
form
linear
define
First,
types of
general
programming
objective.
will
show
that the
we
all
of
which
needed
for
all
not
are
Later,
constraints,
problems.
with
two
constraints.
to
a
of
can
itself
be
reduced
general problem
problem
only
types
Let a!, a2y...,ak be vectors of real numbers, each of length ny and let b i, b2,.., bk
be real numbers.
The inequality constraints are as follows:
The goal
ax
-x<bx
a2 'X <b2
(10.2)
ak-x<bk.
(Exceptfor
The
jc, all
equality
other symbols
constraints
are constants.)
are similar:
Reductions
328
-x=dx
ex
e2 'X
=d2
(10.3)
em'X=dmy
1 \\, e2>
where
also vectors
\342\200\242\342\200\242\342\200\242,
em are
We also
Xj
> 0,
for all j
a given
function
(10.3),
and the
used in
all
of {1,
nonnegative
the
real numbers.
\342\200\242\342\200\242\342\200\242\302\273
^m are
previous
2,..., n).
can be
problem
to
subject
the
inequality
constraints
as
formulated
follows:
(10.4).
constraints
problem.
we can get rid
must be
that
show
a loss
of either
of generality.
or the
the equality
inequality
constraints,
Let
(10.5)
erx=d;
be
an
the
maximize
constraints
of the
without
both,
dx, d2,
(10.4)
nonnegative
instances
first
We
but not
following
and
/>,
programming
(10.1)
c(x)
subset
linear
The
of size ny
represented as a specialcaseof
can be
P is
where
usually
they
though
the
add
We can
constraint.
equality
arbitrary
for
substitute
(10.5)
the following
two
constraints:
inequality
(10.6)
erx<dh
and
(10.7)
-erx<-dh
we can
Alternately,
Given a general
replace the
constraints
inequality
constraint
inequality
(10.8)
arx<bh
we can
a new
introduce
Such a
yh
and
replace
is
variable
is said
called
to be in
a slack
standard
variable.
will
following:
not
existing
describe
linear
program
with
only
equality
form.
the
with
(10.9)
In both
(10.8)
and^>0.
ai-x+yi=bh
constraints
variable,
set
of constraints
to increase.
with
Therefore,it
is not
always
algorithms
of
set
another
note
and
Linear
just an exercise
is not
329
Programming
a good
but
way to
solve the
problem.
10.3.2
in
introduce
seldom
life are
real
Problems
has to
to Linear Programming
of Reductions
Examples
the
to make
definitions
right
in linear
directly
given
the
problem
One
programming formulation.
fit this formulation. Here is one
example.
in
in Section 7.11.) Let the
detail
variables
A*i,jt2, ...,*\342\200\236
of the flow for all the edges (n is the number of edges here). The
is the value of the total flow in the network
is discussed
problem
values
represent
the
objective
function
c(x)=
\302\243*,,
ieS
S is
where
the
<
where
(10.4),
that
inequality
constraints correspond to
edge /.
The
constraints
equality
to
correspond
the
constraints:
\302\243 x,r
leaves v
whole set
is the
the
maximize
maximum
A Static
for alive
= 0
\302\243 Xj
enters v
.v,
the nonnegative
Finally,
The
source.
the
< a?,
capacity of
is the
conservation
x,
/, 1 < /
for all
Cj
c,
leaving
constraints:
capacity
Xj
of edges
set
the
constraints
{1,2,..., /?}).
function
objective
to all
apply
We
V-[s, t).
leave
under
set
it to
constraints
these
f\\ as
that
correspond
defined
the values
in
of
indeed to a
flow.
Problem
Routing
Let G =(V, E) be
that each
node
messages
in one
undirected
an
v,
unit
of
size). Supposefurther
graph representing
network
the
in
time
that
has a
limited
buffer
corrmunication
space,
network.
and can
receive
Suppose
only
\302\243,
of messages. The
infinite
supply
in one unit
of time in
edge should
carry
on the network.
(This is a static routing
all nodes
since we assume that
transmission
problem,
always want to transmit;
usually,
In a graph-theoretic formulation,
needs are dynamically
the problem
is to
changing.)
that
the
sum
such
of
to
the
the
of
all
incident
to
node
edges
assign weights
weights
edges
and the total sum of weights is maximized.
v, is <Bh
can be easily formulated as a linear programming
This graph-theoretic
problem
variable
with
a
each
can
associate
We
the
xf
edge e,=(v, w), indicating
problem.
The objective
function is c(3c) = \302\243\342\200\242*/\342
The
of messages passing through
number
er
transmitted
through
any
link,
and
that each
node has an
each
messages
of messages
number
many
Reductions
330
Xi<Bh
\302\243
is incident to v,
e,
and
for all/.
jc/>0,
Problem
The Philanthropist
there
that
Suppose
are n organizations
sciencedepartments.
year, as well as a limit
be 0 for
may
a,j
of
limit
a^
/ has
organization
amount
on the
to contributed
want
that
it
of
limit
s, on its
contribution
total
to contribute to department
is willing
j (e.g.,
some departments).
In
general
s, is
smaller
than
each
\302\243fl,;; therefore,
organization
Each
an algorithm
that
nevertheless). The goal is to design
no
to
fairness).
(with
regard
is a generalization
of the matching
in
This problem
introduced
Section
problem
It can be solved by matching
but it also has a simple linear
7.10.
techniques,
There are nk variables xijy 1 </<\302\253, 1 <j<k,
formulation.
the
programming
representing
amount
of money
/
is
to
contribute
to
The
organization
willing
department j.
objective
unrealistic,
maximizes
it
but
total
the
is interesting
contributions
is
function
c(x)
^Xij.
'j
are the
constraints
The
Xjj
<
for all /,
a,j
following:
y,
/t
\302\243
Xjj
<
Sj
for
all /,
and
n
2>,7<0
In
addition,
The
Let's
for all j.
of course,
change
the
must
be
nonnegative.
Problem
Assignment
donate money
all variables
problem
one department and
philanthropist
to only
slightly
that
that
insisting
by
total
usually
one
bipartite
problem.
weighted
matching
problem,
each
organization
each
or, as
it
is
to it,
and we
donations.
called,
want
to
only
with
find not
This problem is
an assignment
10.4
The variables for
problem.
one edge is
exactly
problem
must
be
the
notion
to capture
connectedto
the
so
previous
and of
jt/; for
0 otherwise.
The
becomes
c(x) = ^aijxij.
(10.10)
constraints
The
that
insist
must
a variable
assigning
by
is selected,
edge
of the
matching. We
of a
331
Bounds
Lower
those
from
different
We do
node.
each
of 1 when
a value
with
function
objective
this
have
somehow
We
for
Reductions
= 1 for all
\302\243
Xjj
/,
and
= l
2>//
for all/
These constraints
one
guarantee
all variables
addition,
is selected for
edge
each node.
In
a yes or no
deficiency. The variablesrepresent
real
We
have
that
be
numbers!
to
add
constraints
optimal
may
limit
the values of the variables
to either 0 or 1. This
is generally
hard
to
do.
Linear
very
must be integers are called integer linear programs. Solving
whose
variables
programs
of the problems
discussed in the previous
them
involves
integer programming.
Many
as integer linear programming problems. However,
formulated
chapters can be naturally
linear programs are usually
linear
can be efficiently
solved,
integer
although
programs
We
issue
in
the
next
discuss
not always)
difficult.
this
(The
(but
chapter.
very
can
linear
the
be
solved
see,
programming;
assignment problem, by
way,
efficiently
by
for example, Papadimitriou
and Steiglitz
[1982].)
This formulation
choice,
much to
the
applies to
lower bound
common
A as
problem
proofs.
errors
in the
A Lower
Consider
the
problem
well. We
Another
that,
under
be modified
can
problem
then
\302\243,
present three
examples
example
is presented
in
\342\200\224
without
too
adding
a lower
of
next
the
use of reductions.
Bound
of connecting
problem A
\342\200\224
to solve
time
running
10.4.1
also
major
can
we
one
values
Reductions
10.4
If
has
their
but
certain
have
a set
seen
assumptions,
of
points
how to
this
in the
solve this
problem
Polygons
plane
problem
cannot
by
using
be solved
closed
simple
sorting.
more
It is
quickly
we
the algorithm
for the simple closed polygon
Therefore,
presented
sorting.
without
be
cannot
(When we say \"improvement,\"
improving
sorting.
improved
problem
a
than
constant
more
we mean an improvement
factor.)
by
than
Reductions
332
10.1
Theorem
\342\226\241
It is possibleto
simple
Proof: Consider
in time
sort
that
problem
polygon
in time
runs
a (black-box)
algorithm
for
the
O(T).
these points
the model
that was
proved
bound for
must
the
include
assumption.
the decision-tree
bound
decision
for
way
sorting
tree.
for
the
simple
that
we
assume.
problem,
model. In
have
then
this
assumed
6.4.6
Section
simple-polygon
in a
this
in
that the
assume
comparisons
points
a lower bound
of computation
obtain
To
about
a simple
into
case,
the
the
simple
the
on
the
the
model.
from
sorting
lower
reduction
The theorem
is also
first
or
coordinates
numbers
to points.
must
with
consistent
\\/yi
conversion
this
That is, we
use
decision-tree
to show that
To
careful
of
the lower
in
the
the
point
corresponding
the
the
the decision-tree
connecting a set of
in the
comparisons
model, the
tree.
the decision
a decision
into
height.
the
in
a simple polygon
of finding
problem
points
given
plane
n)
case.
worst
the fact
establishes
reduction
This
in
change
significant
in
can be transformed
10.2
\342\226\241
Corollary
Under
as one comparison
problem
simple-polygon
without
sorting,
still counted
y,- is
angle
that solves
tree
decision
333
Bounds
Lower
\342\226\
that
at the
is really
sorting
of
heart
the simple
solving
polygon problem.
10.4.2
In
Section
in
we saw
9.5,
matrices
practice.
to multiply
is
entry
equal to the
it is easier to multiply
nonintuitive
very
matrices
in which
It is natural
to ask
(i.e.,
the
two matrices.
ways
ijth
whether
entry)
y/th
symmetric
Symmetric
occur commonly
It is
matrices.
3
helps in finding better expressions for multiplying,
say,
for multiplying
symmetric
by 3 matrices. This may lead to a better asymptotic algorithm
that this is not
the
case.
We prove that
two
matrices. We now
show
multiplying
a constant factor, as is multiplying
two arbitrary
to within
symmetric matrices is as hard,
that
possible
entirely
symmetry
matrices.
Let's denote
ArbM,
the
the product of
of computing
problem
the product of
of computing
that
and
two
two
symmetric
matrices
arbitrary
matrices
by SymM.
by
is
It
obvious
Suppose
algorithm
arbitrary
every
exchanging
involving
ij
entry
of two
a product
the entry
with
In x In matrices,
ji).
utilize
We
is easy
which
the
following
expression,
to verify:
\"
(The 0s
the
fact
using
stand
two
algorithm
AB
n xn
for
that the
the
Br
\"0
AT 0
ATBT
whoseentries
matrices all of
for
side are
on the left
matrices
the
problem
SymM.
are
// there is an
real matrices
timeO(J(n)
can
the
following
follows
to
algorithm
that
time
0(T(n)),
in
compute
+ n2).
computes
the product
of
two
arbitrary
nxn
from
algorithm
theorem.
symmetric
then
real
there
nxn
is an
matrices
by
product
10.3
Theorem
\342\226\241
algorithm
The reduction
0.)
We
symmetric.
in
for
Reductions
334
Proof:
two
Given
arbitrary
as shown in
product
compute
and to construct the
their
two
symmetric
nxn
to
matrices, we use the assumed
algorithm
(10.11). It takes 0(n2)
steps to compute AT and BT
them.
The theorem
matrices, and T(2n) to multiply
follows.
\342\226\
The
polynomial
do not
to
suggest
the
utilize
is faster
it. This
it in
using
symmetric
0(T(n)),
the
compute
Proof:
following
such
that
the
computes
= 0(T(n)),
T(2n)
of
product
the
in
As
enough
A
two
proof of
a matrix
multiplication
that
algorithm
reduction.
arbitrary
real matrix
square ofannxn
then there is an algorithm
nxn
real matrices in
to obtain
information
in
to
time
~AB
0\"
BA
arbitrary
matrix
matrices.
whose
square
This is
done
\342\226\
Errors
Common
Reductionsshould
to find
expression:
10.5
for
similar
overly
+ n2).
0(T(n)
that
algorithm
time
by
to multiply.
properties of a matrix
practice
10.4
If there is an
the
is good
reduction
Here is another
asymptotically.
Theorem
\342\226\241
contains
= O(T(n)) is
that T(2n)
assumption
satisfies
The following
are examples
of common errors one
a reduction. The most common
error is to apply
the reduction
in the wrong
order.
This mistake is more prevalent
in reductions
for lower bounds. The
reduction
should
in this case that one problem
establish
P is at least as hard as another
whose
we
with an arbitrary
know.
We
need
to
start
problem
Q
complexity
already
instance of Q and to show that it can be solved with
a black-box
solution for P.
can
make
when
Consider,for
be
used
with care.
attempting
the following
of data compression
example,
attempt to reduce the problem
via Huffman's encoding(Section6.6)to the problem of sorting. The goal is to prove a
lower bound of Q(n log a?) for the complexity of Huffman's
encoding.
The main observation is that,
are wide apart,
of the characters
if the frequencies
then
the tree becomes
so unbalanced that it can be used for sorting
(see
Fig. 10.3). In that
the
case, the characters
will appear in the tree in decreasing order of frequencies
(with
at
the
Huffman's
of
character
means
that
But
that
the
tree).
highest-frequency
top
the tree is at least as
encoding can be used to sort these frequencies. Therefore, building
of \302\243l(n
hard
as sorting,
and a lower bound
be
seems
to
log a?)
implied.
The error in this argument comes from the fact that we started with a special case
We considered only
of the sorting
those
that that are wide apart.
problem.
frequencies
Figure10.3 A
To prove
a lower
After
the proof
all,
sorting.
by
the
We
must
should
start
Huffman's
we
for sorting,
bound
with
show
for frequencies
tree
Huffman
arbitrary
numbers
and
We
will
encoding algorithm.
wide
apart.
start
must
Huffman's
that
are
that
335
chapter.
The trick
As it turns out, we can modify the arguments
above and save the proof.
is to spend some time
the input of the
(which must be
sorting
problem
changing
of distinct
Let the input
be a sequence
so that it conform with
our
arbitrary)
goals.
=
are
X
We
can
assume
that
the
numbers
distinct,
(jcj, jc2,...,jc\342\200\236).
integers
positive
the lower bound for sorting
to distinct numbers as well (in fact, the lower
because
applies
The Huffman's
bound was proved for distinct
encoding corresponding to
integers).
in
X
can
be
tree;
thus, the
any
general
frequencies that are equal to the numbers
above
cannot
be used. However, we can replace each
jt, with,
say, y; = 2X'.
arguments
tree will have the
for any positive integer m, we have 2m > \302\243
2', the Huffman
Since,
i <m
form
sort
in Fig.
shown
the
reduction
y,-s.
We
(computing
the yp from
the
jt,s,
to use
that
the
in this
the
Huffman's
extra
encoding
computation
case) is
not
prohibitive.
algorithm
involved
in
to
the
Computing
be quite expensive,
but that is irrelevant
case, because the lower bound
the number
of
involves only comparisons.
We made no assumptions about
other operations (see again
the
we established that building
Section
Therefore,
6.4.6).
Huffman's
in the worst
case
the
under
encoding requires \302\243l(n
comparisons
logn)
decision-tree model. (It may be possible to build the tree more quickly with an algorithm
that does not conform
to the decision-tree model.)
have
to be careful that
the
We
also
reduction
does not impose significant
the
Consider
in
discussed
Section
and the
5.11,
knapsack
problem
inefficiency.
5.17. (The extension was to solve the knapsack
in Exercise
extension to it addressed
an unbounded number of
problem where each item can be included in the knapsack
the
of
extended
to
reduction
A
(in
times.) straightforward
problem the original
problem
Let the size of the knapsack
be
at most once) is the following.
which each item appears
powers can
for sorting
in this
Reductions
336
K.
item
An
in the
item
10.6
It
is
Summary
idea to look for
a good
always
problems
between
similarities
problem,
known
only
apparent
reductions betweenmatrix
several examples of reductions
and
after
all
\"Is this
cases),
complicated
graph algorithms
in this chapter,
into both
insight
the
Sometimes,
problem?\"
studying
By
problems.
between
have
seen
the
next
in
chapter.
Linear
are very
and
important
and should
be
Notes
Bibliographic
Hall'sTheorem
were described
programming
integer
studied
in detail
and
Further
too
by anyone
in
briefly
this
interested in
They
chapter.
algorithms.
Reading
detailed
algorithm,
is due
to P. Hall
almost
in
found
discussion and
has been
numerous
in
However,
ellipsoid
by
Dantzig
[1972]
Minty
time
running
algorithm
in 1947
(see Dantzig
algorithm, called
examples).
Dantzig's
use since the 1950s.
extensive
is exponential.
simplex algorithm
for linear programming whose
also
algorithm, which is known
the
first solved
was
programming
is
It
worst-case
the
is polynomial
as the
ellipsoid algorithm,
has other applications
first
in the
works
fast
worst
very
(see, for
running
case.
poorly
example,
the
and
to exhibit
for
[1963]
simplex
practical.
time
of the
an algorithm
Khachian's
in practice.
Grotschel,
linear
a
received
programming was introduced
[1984]. Karmarkar's algorithm
by Karmarkar
lot of attention,
for being
and
extensive
research, because of its potential
triggered
in certain cases. For more on linear and
superior to the simplex
integer
algorithm
and Steiglitz
see, for example, Papadimitriou
programming,
[1982], and Schrijver [1986].
10.7 can be found
A solution
to Exercise
in van Leeuwen
[1986]. Exercise 10.8is
from
Exercise 10.11 is from Maggs and Plotkin
Even
Exercise 10.22 is
[1979].
[1988].
from Aho, Hopcroft,
and Ullman [1974].
Lovasz,
and
Schrijver [1981]).
Another
polynomial-time
algorithm
for
337
Exercises
Exercises
10.1
and
10.2
10.3
by using the
theorem
Hall's
Prove
techniques developedin
the
sections
on network
flows
matching.
bipartite
Solve the
variation
of the sequence-comparison problem. The input
is two
following
the
is
B
so
it
becomes
to
A.
The
A
and
and
to
edit
that
edit
\302\243,
equal
goal
steps are
sequences
the
usual ones: insert, delete and replace
(or match). The cost of a step, however,
depends
of the corresponding characters.
The cost of inserting a
on the position in the sequence
the yth
character at the z'th position in B is d, where c is a constant, and the cost of deleting
character of B is cj. The cost of replacing a character with another character is still 1. The
edit sequence.
algorithm should find the minimum-cost
(in some
a reduction
Find
8.18) and
plane (Exercise
between
direction)
of marking
that
in the
points
intervals
(Exercise
8.26).
10.4
Department D at
The
examination
n
in area
= Q).
(\302\243<7,
/=i
questions for
write
who
There are
that can
questions
be
used,
P \\,P2
professors
examination.
the
>
that
\302\243/?,
/=i
Q. A
(these
Suppose
and
Pk
/>,
committee is responsible
has
names)
overall
for selecting
/?,
the
We assume
from the questions supplied
by the professors.
and that they are all good.
Assume, furthermore, that each
questions are unique,
of
r is a constant independent of the professor)
professor insists that no more than r (where
in later
will be used (so that he or she can use the remaining
his or her questions
questions
for the examination under these
years). Design an efficient algorithm to selectthe questions
so.
is
to
do
or
to
determine
that
it
constraints,
impossible
for the examination
questions
that
10.5
all the
variation of
Consider
the
following
In students
who
the
to be admitted
want
variation
of the bipartite
matching
problem.
Suppose
now
that
there
are n
problems.)
10.7
Let
G =(V,
E) be
an
undirected
graph
such
that
each
vertex
v is
associated
with
an integer
338
Reductions
than
in G
A ^-matching
incident to
edges
b(v)
it.
matching problem.)
Reduce
of finding a
the
problem
(If
maximum
^-matching
maximum
for
is one
is
of
to that
the
exactly
number
maximum
with
^-matching
each vertex
that
has
no
regular
of edges.
a maximum
finding
matching.
directed
acyclic
graph.
that include
paths
all
Design
an
to find a minimum
algorithm
of G.
vertices
Let G = (V, E) be a network with source s and sink t. Assume that G is planar,
it
namely,
that such a
can be laid out in the plane such that no edges intersect. Assume furthermore
and that both the source and the sink
layout is given to you (in a reasonable representation),
an algorithm to find a minimum-cost
cut in G
of the layout.
lie on the outside
Design
using the
without
maximum-flow
algorithm.
10.10
Exercise 8.40 can be solved by reducing the problem to that of minimum-weight matching.
leads to an inferior
is
(In this case, the reduction
algorithm since minimum-weight
matching
and prove its validity
harder than a direct solution.) Show the reduction
(i.e., prove that the
satisfies the conditions of the problem).
matching
corresponding minimum-weight
10.11
10.12 The
v, x],
u is
that
variables.
allowed
10.16
The
endpoints
u are
and
is a
path
ignored,
you
have
that
a linear
in our
one
Section
in
programming
definition
Show
how to
exhibiting
in a
of linear
constraints:
can be reduced
(Recall
by
\302\243<:(*,).
i=i
formulation
inequality
restricted to be nonnegative.)
10-15 Show,
w.
defined as
the \">\"
imposes
type
formulation
10.14 Suppose
vertex
the
\302\243,
a distinguished
with
each
but here we
second
E)
associatedwith
cost c(w)
so if (v, u)
minimal-cost
10.13
is a
input
positive
of
algorithm
linear
reduce the
constraints
different
Show
coefficients).
that
10.3.
that
programming
general
of the
not
problem
type
all
nonnegative
variables
were
to this one.
ax*b
should not be
Exercises
339
Supposefurther
10.17 Consideragain
global
the
of h
bound
philanthropist
amount of
on the
limits
of
the number
that
is one
there
that
each
money
problem of Section
is willing
department
10.3.2. Supposethat
to accept.
there
Solve this
variation
are no
of
the problem.
10.18 Considerthe
round-robin
problem
by
a good
finding
10.19 Let S be a
given
in
of n points
order.
set
an
that
problem.
vertices
are
Prove
arbitrary
standard polygon
10.20 We
for this
algorithm
that
of an arbitrary
it takes
Q(a7
(i.e., in consecutive
representation
log a)
order).
have
to compute
matrix
with
10.4.
Theorem
contradict
is a square matrix
10.21 A lower triangular
matrix
(fl;/) such that, if j > /, then a,, =0 (in other
on
or
all
are
below
the
main
An upper triangular
nonzero
entries
words,
diagonal).
that
the
nonzero
on
or above the main
is defined similarly,
entries
are
matrix
except
if
there
exists
an
to
an
nxn
Prove
lower triangular
that,
algorithm
multiply
diagonal.
matrix
whose running time is O (T(n)), then there exists
matrix,
by an n xn upper triangular
two arbitrary
nxn matrices whose
time is 0(T(n) + n2).
an algorithm to multiply
running
You
can assume that T(cn) = 0(T(n)) for any constant c.
10.22
if there
that
Prove
time
running
matrices
any
where /
running
time is
there
0(T(n) + n2).
You
can
lower
to
triangular
multiply
assume
matrices
two
that T(cn)
arbitrary
whose
nxn
= 0(T(n)) for
c.
closure
+A +A2+
is the
algorithm
then
whose
constant
exists an
is 0(T(n)),
n x n
of an n
xn
matrix
A is
defined as
follows:
\342\226\240\342\226\240\342\226\240
+A\"~\\
identity matrix.
if A is a Boolean matrix
to an adjacency matrix
of a graph,
that,
corresponding
matrix of the transitive
closure of the graph.
then A* corresponds to the adjacency
is performed according to the Boolean rules.)
(Assume that multiplication
a. Prove
340
Reductions
b.
Prove
if the
that,
polynomial
in n,
assume that
T(cn)
10.24 Let S be a
transitive
then
matrix
= 0(T(n))
closure
multiplication
for any
constant
T(n) is a
You can
c.
a weighted
undirected graph in the
points in the plane. The points define
is the complete graph (i.e., every
two vertices
are connected),
way. The graph
to the Euclidean distance between
the two corresponding
the weight of an edge is equal
and
time of an MCST algorithm for
points. Show a lower bound of Cl(n log/?) for the running
set of n
following
this
case.
10.25 Let
S be a set of
two points
in
S.
points
Denote
^k-)-:-^.-:-l^.-.-.-.-.-.-.-.^
f1''\\\342\226\240
'-^.-:-l->:^l^J;-
-r-: -r-\342\200\242\342\226\240-'
-:-r-r:- - -: -V-CC->\\-V^
11
CHAPTER
NP-COMPLETENESS
Give
me where
to stand,
Archimedes
and
(287-212
I will
B.C.)
Introduction
11.1
This chapter
different
for
studied
the corresponding
is efficient if its
algorithm
size of the input n. Recall
and
call
required to represent
algorithms
is
denoted
that
input.
is 0{P{n)\\
time
running
The
by P (for
size of
the
input
class of all
polynomial
words,
where P(n) is
is defined as
problems that
time).
other
This
can
we say
a polynomial
the
in
an
the
of bits
number
be solved
that
by efficient
may seem to be
strange
that run
in
time
O (nl0)
NP-Completeness
342
times of
we encounter
in practice
are mostly smallThe
is
also
true:
opposite
usually
quadratic).
degree polynomials
not
are
times
whose
are
than
usually
any polynomial
practical
running
larger
Algorithms
for large inputs.
is known.
no
are many problems for which
There
algorithm
polynomial-time
that are yet to be
of these problems may be solved by efficient
Some
algorithms
We strongly
that many problems cannot be solved
however,
discovered.
suspect,
like to be able to identify
so that we do not have to
We would
such problems,
efficiently.
In
this
we discuss how to deal
for
a
nonexistent
time
searching
algorithm.
chapter,
spend
with problems that are not known to be in P. In particular, we discuss one specialclass
in one class
these
called NP-complete
of problems,
problems
problems. We can group
running
above
(seldom
becausethey
any
all equivalent
are
one NP-complete
problems.
complete
for
The
chapter
problems
and
show
several techniques
These solutions
a strong
sense
\342\200\224
there
an efficient
exists
proof
problems,
they would
Even
known.
be
surely
very
for
algorithm
of that belief is
no
NP-complete
have been
problems
but
any
were efficient
since they
complicated,
if there
of
(maybe even thousands)
which
is
is
so
this
why
NP-complete,
subject
important.
consists of two
we define the class
of NP-complete
First,
parts.
how to prove that a problem
Then, we present
belongs to the class.
for
researchers
eluded
have
in
problem,
NP-complete
algorithms
that
algorithms
polynomial
years.
hundreds
to be
found
and
may
many
So far,
for
examples
be optimal,
not
and
solving
they
than nothing.
11.2
We
will
Reductions
Polynomial-Time
restrict
ourselves
those problemswhose
in
this
answer
section
is either
to decision problems;
yes or no. This restriction
that
is,
we consider
makes
only
the discussion
the
theory
simpler.
For example,instead
this
of
chapter.
Definition:
Let L,
We
U2.
say that
polynomial-time
and
Lx
algorithm
L2
be two
languages from
reducible
is polynomially
that
converts
each
the
to L2
input
spaces
input
if
there
U\\e U{ to
\\
exists
another
and
11.2
well defined
is also
size of the
in the
polynomial
L{ if
U2
u2e
input
the
in
polynomial
input
spaces U x
size of u x.
input
the
in
and
x.
and
L2. The
if u2e
only
We
that the
assume
so,
\302\243/2,
343
Reductions
Polynomial-Time
is
algorithm
of
notion
size
is
the size of u2
in particular,
The
mentioned
algorithm
for L2,
an
11.1
\342\226\241
Theorem
reducible
If L\\ is polynomially
algorithm for L2, then
comes from
an
cases likely,
the
for
inputs
the definition
of L2 involved in the
Thus, if L \\ is polynomially
If
L\\
and
can
We
essence
u3
in
reducible
relation
or
of
equivalent,
simply
nontrivial
all
particular,
polynomial-time
The
(we
algorithms
leave
reducibility\"
\"polynomial
if
tractable
the
is
of the
an efficient
(This
to L2 and L2
to L 3.
the two
still
conversion
is one
a polynomial
of
the
method we present in
algorithm
problems.
reducible
is polynomially
cannot
to
reducible
reasons
this
be found.
whether
we try to find
cannot solve efficiently,
to be hard. The class of NP-complete
known
equivalent
equivalent,
In
to
algorithms
form
functions
conversionalgorithm.
when
theorem.
compose
to L3.
to an input
two polynomial
The
next
the
in
have
exercise).
L \\ is polynomially
from
then
all
polynomially
the other.
as an
is polynomially
Proof:
algorithm
to
11.2
\342\226\241
Theorem
L 3, then
are
L2
because
fact
is shown
as
and
reducible
polynomially
transitive,
It is
versa.
problem.
Two languages Lx
is
each
of L { can be
possible, and in many
reduction
are only a small
fraction
of all
reducible to L2, then we regard L2 to
inputs
L2.
that L{ is polynomially
to L \\. This asymmetry
requires
vice
not
but
x.
\342\226\
reducible
polynomially
of reducibility
of L2,
input
for
fact
the
symmetric;
is
a polynomial-time
discussion.
preceding
not
that L2
imply
equivalent
that
harder
not
that
fact
the
to
converted
be the
does
to L2
to L2 and there
is
polynomial time algorithm
the
is
of reducibility
notion
The
reducible
possible
The
Proof:
is a
there
function,
we chose
the
result
an
is a
a conversion
input
u2 in
a composition
L2
of
polynomial-time
to use polynomials.)
\342\226\2
chapter
NP-Completeness
344
state
we
Before
started with
NP-completeness
the theorem, we must
explain
of
theory
The
Theorem
and Cooks
Nondeterminism
11.3
We
notions.
several
of Cook
theorem
remarkable
discussion
and
intuitive
is
is Garey
an algorithm
the
of
part
of
scope
terms what
for describing
important
reasonable
use
[1971].
will
all
by
definition of an
accesses).
memory
have
used decisiontrees to prove lower bounds in Chapter
bounds.
lower
(We
6,
proving
The most fundamental
model
of computation
restricted
is a
but this is a very
model.)
used model is that of a random
access machine.
commonly
Turing machine. Another
for our purposes,
these
and other reasonable models are equivalent
because
Fortunately,
an algorithm
from one model to another
without
the
we can transform
changing
running
A precise
time
the
more
by
We first
which
mystery.
One
and
leads
realistic goal.
the
deterministic
name
has,
algorithm
non-
rather
to all
addition
in
of the
development
the
of a
not
theory
the
for using
techniques
algorithm,
suggests,
one.
This notion is
important to the
class
than it is to
existence of this
nondeterministic
a nondeterministic
is more
to think
Nondeterminism
that require
details
any
nondeterminism.
of
notion
people
many
of the
explanation
theory.
the
think of
should
go into
not
will
to discuss
need
intuitive,
as a
it is
but
model
specific
a polynomial
than
Turing machines,
here, since we
use of
but
it does
so
in
an
This primitive is associated with a fixed number of choices, such that, for
each
the algorithm
follows
a different
We
can assume,
computation path.
without loss of generality,
that the number of choices is always
two.
Let L be a language
unusual
that
way.
choice,
deterministic
regular
end,
to recognize.
want
we
it decides
We say
following
during
outcome
an
in the
nondeterministic
of the
The key
difference
a language.
recognizes a language
recognize
they
way
algorithm performs
primitive, and, at the
and
between
deterministic
nd-choice
algorithm
if
the
is satisfied:
input
the
uses
with
nondeterministic
the
that
condition
Given
lies
algorithms
jc,
input
accept x.
not to
or
whether
an
interleaved
steps
nondeterministic
Given
x,
it is
execution
of the
algorithm
of
be to
will
must
into
algorithm
provide
outcome,
an accept
and
accept
jc,
at least
it
must
outcome.
nd-choiceencountered
a real
if and
choice such
only
if
x e
one possibleway
not provide
Notice
the
any
asymmetry
that
the
L.
for
way
inputs
for
in the
belonging
not
definition.
inputs
345
input
xe
has
execution sequence
nondeterministic
to L
belonging
to
leads
that
an
of
time
to worst-case
refers
algorithm
L (inputs
not
are ignored).
Let's see an
of a nondeterministic
of
example
algorithm. Consider the problem
whether
a
has
a
The
is
a
G=(V,
E)
deciding
given
perfect matching.
following
graph
We maintain
nondeterministic algorithm
for this problem.
a set M of edges, which
is
We examine all the edges
of G, one edge e at a time,
and use an
initially
empty.
nd-choice
to whether or not we include
e in M. When we are done
corresponding
all the edges, we check to see whether
M is a perfect matching. The checking
examining
in linear time, since we have to determine only whether
M contains
can
be done
exactly
whether
and
vertex
is
incident
to
one
from
M. The output
V
each
| 112 edges
exactly
edge
is yes if M is a perfect matching,
no otherwise.
of the algorithm
and
This is a correct
nondeterministic
then
there
for perfect
algorithm
is a
sequence of choicesthat
nondeterministic
the
of size exactly
efficiently
is to
problem
in the
algorithms
are very
algorithms
(1) if
a perfect
exists,
matching
outputs
put it in M; and (2) the algorithm
was proved (because of the checking).
will
matching
perfect
supposethat
matching because
but
powerful,
section.
next
their
power
the
whether
is
not
unlimited.
by a nondeterministic
determine
yes
We
maximum
matching
in
given
Definition:
NP
k.
problem
is polynomially
Definition:
belongs to NP,
We can
X is called an
reducible to X.
problem
and
called
NP-hard.
X is
(2) X is
NP-hard
an
problem
NP-complete
if every
problem
problem in
if
(1) X
NP-Completeness
346
of NP-hardness
definition
The
then
to P,
belong
that
proof
Cook [1971]proved
exhibited
NP-complete
implies
that
easier.Given
a new
NP-complete
problem,
problem
is
it is
Y,
if
that,
problem is ever
NP-hard
any
would
we
other
problems
sufficient
to prove that
to
reducible
polynomially
he
problems; in particular,
Once
we
have
found
an
shortly.
are also NP-complete becomes
describe
will
proved to
Y.
Cook's
or
problem,
other
any
follows from
This
next
the
lemma.
11.3
Lemma
\342\226\241
A problem
Proof:
is
to
reducible
is a
reducibility
the
definition
Y.
But
2 in
condition
By
polynomially
belongsto
problem if(J)X
NP-complete
reducible to X,for someproblem
is an
is polynomially
transitive
of
since
and (2')
NP,
is NP-complete.
Y that
every problem
NP-completeness,
Y is
reducible to
is polynomially
well.
as
\342\226\
much easier to
It is
prove that
news. As
more choicesfor proving
more good
[1972] found
hundreds of
same
24
problems
problems(maybe
even
have been
thousands,
it is
the
depending
discovered to be
NP-complete.
on
became
In
known,
Karp
time,
variations of
Since
count
the
to prove
there is
we have
NP-complete
we
how
And
that
next section, we
problems
NP-completeness
proof. We
The
without
most
difficult
part of
NP-complete
problems
proof.
not
2
to
condition
(but
(or 2').
always)
verify
mention
the problem that Cook proved to be NP-complete,
and
The problem is known
as satisfiability
(SAT). Let 5 be a Boolean
of
normal form (CNF). That
is, S is the product
(and)
conjunctive
present
examples
also list several other
such proofsis usually
We now describe
the idea of the proof.
in
problems
condition
important
problem)
five
expression
two
2 directly.
condition
the
NP
in
X and
to
reducible
polynomially
NP
of NP-complete
with
their
Boolean
expression
a truth
assignment.
NP because we can
is in
11.4 Examples
state
accepting
becomes
for
the
quite large
number of
described by
steps the
an
instance
complicated,
machine
Turing
of a
347
This
input.
given
and
of NP-Completeness
Proofs
SAT problem.
theorem:
\342\226\241
Cook's
In
of
Examples
this
we prove
section,
following
dominating
more
they
we must
problem,
always!) easy,
first prove
then
reduce
Proofs
NP-Completeness
the
that
\342\226\
that
a known
the
clique.
section.
problem
The reduction
11.1.
To
rather
than
NP-completeness
to NP, which
is usually
prove
belongs
time.
are typical,
NP-completeness
in
this
Fig.
11.1
in
order
Figure
11.1
proofs
in the
text.
(but not
in
Fig.
of difficulty
by the numbers
edges.
and
new
polynomial
is illustrated
section
the proofs
of a
of
the
NP-Completeness
348
G =(V,
Let
an
undirected
incident
G is
every edge in
The Problem
k, determine
at
Given
one of
an
undirected
a vertex
cover of G is a set of
vertices
such
that
these vertices.
least
G has
whether
A vertex
graph.
to
graph G
=(V, E)
cover containing
< k
and
an
integer
vertices.
11.4
Theorem
\342\226\241
vertex-cover
The
check
it easily
in polynomial
time. To prove
since we
that
can
guess
the vertex-cover
a cover
of
problem is
11.4.2
Dominating
Let G =(V, E) be
that
every
the
clique
vertex-cover
corresponding
vertex
an
problem
problem
Set
undirected
of G is either
graph.
in
D or
A dominating
is adjacent to
at
set D is a
least
one
11.4 Examplesof
Given
The Problem
k, determine whether
\342\226\241
Theorem
an
G has
undirected
graph
a dominating
Proofs
NP-Completeness
G = (V,
E)
set containing
and
< k
an
349
integer
vertices.
11.5
<k and
check
vertex-cover
The
that
dominating-set
problem
it is a dominating
set
problem to
the
belongs to NP since we
can
guess
in polynomial time.
easily
Given an arbitrary
problem.
dominating-set
We
a set
reduce
of size
the
instance (G, k)
G' that has a
the
vertex-cover
problem,
vw
ft
VZ
uw
zw
Figure
11.2
350 NP-Completeness
11.4.3 3SAT
3SAT
The
problem
is a Boolean
expression
is a simplification
in which each
Given a
Problem
The
contains
clause
Boolean
exactly
expression
3SAT
that each
such
CNF
in
variables, determine
three
An instance of
variables.
is
it
whether
satisfiable.
11.6
Theorem
\342\226\241
3SAT is N?-complete.
Proof: This
the additional
can be
3SAT
guess a
is
can
truth
arbitrary
instance
of
has
each
clauses,
and
assignment
Let E be an
which
seems
problem
variables.
three
exactly
each
clauses,
satisfiability.
into C such
=
C'
We use new
with
several
\342\200\242
\342\200\242
Let C =(x\\ +jc2+ \342\200\242
be an
+xk)
in its \"positive\" form (i.e.,
We now show how to replace C with
idea is to
variables
different)
of E
variable
a 3SAT
into
clause
the
(and
The
variables.
three
with
that transform
y\\iy2>~->yk-s
each
clause
each
time.
in polynomial
expression
to
introduce
new
without
formulation
for each
clause. C is
variables
affecting
its
transformed
that
\"'
(x[+x2+y[)-(x3+yl+y2)'(x4+y2+y3)
(**-i+**+)'*-3)-
x,s
that C
claim
the *,s
clauses
be
must
be
must
set to
_L
all
if
Indeed,
xfs
are 0,
then
the
expression
becomes
CVi)*CVi+>;2),CV2+>;3)
Cy*-3).
expression is clearly unsatisfiable.
this
variables
three
that has more than
we
can replace any clause
reduction,
Using
with several clauses, each with
clauses
It remains to transform
three variables.
exactly
with one or two variables.
If C has only
two variables,
namely, C=(x{ +Jt2), then
\"\342\226\240\342\200\242
This
C' = (xx
where z is a new
C' =
where both
+jc2 + z\,
+x2+z)'(xx
(x{ +y
y and
variable.
Finally,
+ z)-(x{ +y +
z are
new
z)-(jr1
variables.
if C
=jc ,, then
+>>+z)-(jt1
+y + z),
11.4
is satisfiable if
instance
done in
polynomial
11.4.4
reduced
have
we
Thus,
one
that
of NP-Completeness
Examples
of 3SAT such
can clearly be
an instance
into
is.
The
351
Proofs
reduction
time.
\342\226\2
Clique
definedin
Section
when we
11.4.1,
discussed
the
E)
integer
vertex-cover
problem.
Given
Problem
The
an
undirected
G contains
whether
ky determine
graph
a cliqueof
size
G =(V,
> k.
and
an
11.7
\342\226\241
Theorem
The
and
check
vertices
problem.
Consider
Let
the
E be
clause
an
E,- = (x+y+z
We associate
also
appear in other
they
purposes).
arbitrary
\"column\"
Boolean
+ w)
(we
of
four
expression
use four
vertices
in
CNF,
variables
with
the
E =EXE2
only
variables
of >k
clique
''
Em.
illustration
for
in
even
\302\243,
if
That
NP-Completeness
352
Figure11.3 An
each other, which
assumethat
from
of the
cannot
happen
clique reduction
u+)>+z)-(i+y+z)-o>+z).
a clique
G contains
column
each
course
of
example
(since
two
in
of size >m.
vertices
from
a consistent
The
the expression
must
clique
the same
for
connected).
We
11.4.5
3-Coloring
Let G =(V, E)
to the vertices
be an
have the
color.
same
The
whether
such
coloring of G is an
each vertex is assigned one color and no
undirected
that
graph.
Given an
Problem
G can be coloredwith
\342\226\241
Theorem
A valid
undirected
three
graph
G=(V,
assignment
two
adjacent
of colors
vertices
determine
\302\243),
colors.
11.8
3-coloring is N?-complete.
Proof: The
3-coloring
problem
belongs to NP since we can guess a 3-coloring and
valid coloring easily in polynomial
We reduce 3SAT to the 3time.
This is a more complicatedproof
for
two reasons.
First, the two
coloring
problem.
deal with
different
versus
problems
(Boolean
objects
graphs). Second,we
expressions
cannot
one
with
another
vertex,
just replace
object (e.g.,
(e.g., clause); we have to
edge)
deal with the whole structure. The idea is to use building
and then to tie them
blocks
instance
of 3SAT. We have to construct
G such
a graph
together. Let E be an arbitrary
if and only if G can be 3-colored. First,
that
E is satisfiable
we build the main triangle M.
check
that
it
is a
11.4 Examplesof
a triangle,
Since M is
F (for
true),
(for
used only
colors
the
with
another
build
vertex
M.
in
assignment
We
We
do
with the
that
six new
introduce
11.5.
(since
vertices
vertex
the
that
condition
the \"colors\" T
These colors are
connect
and
for each
outer
xy
vertices
yy or
J, and
x is
if
them
is
the
same
Assume
to the
z.) Let's
one vertex
call the
by
as shown
vertices,
existing
is only
there
A, where
one
triangles, all sharing
colored with the color 7, then x
to A), and vice versa. This
is
k +1
at least
construct.
following
x,
have
connected
both
consistent,so that
is that,
idea
they are
consistent
value 1.
with
M with
in
Mx
triangle
So, if there
be colored
must
label
We
the bottom
(see
triangle
Fig. 11.4).
these
they are not part of the graph. We will later associate
of truth values to the variables
of E. For each variable x, we
and
proof;
vertex A (see
common
We
false),
the
for
colors.
at least three
it requires
353
Proofs
NP-Completeness
three
in
new
O in the
whole
the
vertices
figure),
and
in
Fig
graph
connected
the
three
the inner
11.6.
is
We have to prove two sides:
the proof.
(1) if \302\243
complete
can be coloredwith three colors; and (2) if G can be colored with three
then E is satisfiable. If E is satisfiablethen there is a satisfiable truth
colors,
assignment.
the variables
We color the vertices associatedwith
(7
according to this truth assignment
M is colored with 7, F, and A as indicated. Eachclause must
if x = 1, and F otherwise).
outer
at least one variable whosevalue
have
is 1. Hence, we can color the corresponding
vertex with F, the rest of the outer vertices with A, and the inner triangle
accordingly.
three
if G can
be colored with
three
colors.
Thus, G can be colored with
Conversely,
We can
satisfiable,
Figure
now
then G
11.4
The
first
part
of the
construction
in
the
reduction
of 3SAT to
3-coloring.
354 NP-Completeness
Figure11.5 The
to the
corresponding
subgraphs
clauses
in
the
of 3SAT to
reduction
3-
coloring.
y
Figure
colors,
three
we name
colors).
to a consistent
variable in
polynomial
11.6 The
the
colors
according
corresponding
to the
to (;t
+>>
+F)
coloring of M (which
+z).
\342\200\242(*+>>
must
be colored
with
of
graph
the
in Fig.
Examples of
11.4
NP-CompletenessProofs
355
to
look very
which
It is
briefly
first
The
always).
or similar
that
a problem
some general methods for proving
Q is NP\342\200\224
\342\200\224
that
to
NP
is
Q belongs
showing
usually
easy (but
that seems related
Then, we have to select a known
NP-complete
problem
to define
this \"similarity\"
since sometimes
the problems
Q. It is hard
goal,
here
discuss
We
different
to reduce
condition
a good idea to
try
several
reductions
and
SAT).
several
with
the right
Finding
can be
which
learned
until a
problems
problem
by
only
from
experience.
successful one is
found.
We
stress
The
done
from
an
instance
arbitrary
of
the
known
NP-
the
complete
proofs is to perform
reduction
to
remember
the
order
to
that
the
is
ensure
NPway
right
a black-box
complete
problem can be solved
by
algorithm for Q. This is a little
The natural thing to do when
counterintuitive.
a problem Q is to try solve
it. Here,
given
however, we try to show that we can solve another
(the NP-complete
problem
problem)
the solution
of Q. We are not trying to solve Q\\
using
are several degrees of freedom that
There
can
be used in the reduction.
For
in
if
a
can
be
convenient
includes
then
its
value
set
Q
parameter,
any
way. (In
example,
in the problem that is reduced
to Q, which
cannot
be fixed!)
contrast with the parameter
therefore, we can use it in any
Again, Q is just a tool to solve the NP-complete
problem;
to special
its
cases in other ways, besides fixing
way we wish. Q can be restricted
a
of
For
we
want
to
use
certain
may
only
types
input
(e.g.,
regular
parameter.
example,
we have is the fact that
flexibility
graphs, biconnected graphs) for Q. Another
important
the efficiency of the reduction
is unimportant,
as long as the reduction can be done in
time.
We can ignore not only constants
and, for example, double the size of
polynomial
the problem, but we can also square the size of the problem! We can introduce
in a graph by a new large
vertex
polynomially
many new variables, we can replaceeach
the
on.
There
to
be
efficient
bounds of a polynomial),
and
so
is
no
need
(within
graph,
problem to Q.
backward.
One
most
error
common
since
the reduction
is not meant to be converted into
found to be equal to NP, if ever).
There
are
some
common techniques used in
(again, Garey
and
Johnson
[1979]
provides
problem is a
many
in such
an
the
(at least
algorithm
examples).
until
P is
of the reductions
construction
special caseof
not
that
an NP-complete
Q.
If it
NP-Completeness
356
Considerthe
cover
vertex-cover
independent
and
problem,
be solved
can
it
extra requirements
found
vertex-cover
the
an
independent
In
constraint
that
the vertex
we are looking for a small
set at the same time. (An
7.115).
(Exercise
sets to such
this problem is
however,
fact,
time
polynomial
candidate
the
limit
in
words,
are
that
other
In
to worry
have
of vertices
than either
more difficult
seems
we
a set
set is
vertices.
that forms
of vertices
set
that we add
Suppose
problem.
two adjacent
include
not
must
that
extent
an
It
that the
out
turns
easier
an
can be
the minimum
easily.
Another
object
in
done
is mapped
problem
in a local
involves local
easy technique
relatively
one
manner,
an
into
of
independently
reductions. In
this
an
case,
is
proof
11.4.7
The
list contains
following
reductions
other
for
and
Johnson
half
the work.
[ 1979J.
Hamiltonian
cycle: A Hamiltonian
once. The problem
Hamiltonian circuit. The problem
is
(Reduction from vertex cover.)
graphs.
each
vertex exactly
Traveling salesman:Let
salesman
tour
is
Hamiltonian
path.
The
G contains
integer
reduction from clique.)
3-dimensionalmatching:
of
triples
(x, y,
z) such
that
problem
problem
X,ye
is
graph
a simple
open
is to determine whether
is NP-complete for both
directed
that
path
a given
graph
and
undirected
cover.)
set in an
undirected
graph
The problem is to
set
an independent
Let
xe
in a
path
NP-complete
A travelinga weighted
graph.
complete
G and a
to
The
is
determine, given
cycle.
problem
of its
exists a traveling-salesman tour such that the total length
reduction
from Hamiltonian cycle.)
directed
ky
contains
contains
E) be
G =(V,
vertices
graph
determine
a Hamiltonian
number W, whether
there
<
is
W.
edges
(Straightforward
contains a
in a
cycle
is to
A\\ K, and
Yy
and
>
k vertices.
E) is
given
a set of
and
an
(Straightforward
Z be
z e
with
G =(V,
determine,
set
11.5
a subset of
there exists
two-dimensional
with
problem
357
Problems
NP-Complete
each element
is the
regular
contains
that
matching
(Reduction from
For Dealing
Techniques
corresponding
problem.
matching
bipartite
3SAT.)
Partition:
element
each
xe
an associated
has
size
it is possible
to partition the set into two
total size. (Reduction from 3-dimensional
Matching.)
(Notice that this problem, as well as the next problem, can be solved
efficiently
by
However, since the
algorithm Knapsack (Section 5.10) if the sizes are all small integers.
of bits required to represent that
of the input is the number
size
such algorithms,
input,
are
in
the
which are calledpseudopolynomial
size
of the input.)
algorithms,
exponential
is
a
setX
such
that
each
element
xe
X
has
The
an
size
associated
Knapsack:
input
there
is a subset B^X whose
s(x) and value v(x). The problem is to determine whether
total value is > v. (Reduction
from partition.)
total
size is < s and
whose
is a sequence
of numbers a\\,a2, ...,a\342\200\236,
Bin packing: The input
and
two other
the set can be partitioned
numbers b and k. The problem is to determine whether
into k
subsets such that the sum of numbers in each subset is < b. (Reduction from
partition.)
subsets
with
11.5
the same
exactly
for
Techniques
with NP-Complete
Dealing
Problems
notion
The
is a basis for
no polynomial algorithm
of NP-completeness
for which
problems
an
elegant
is likely
problem is NP-complete does not make the problem go away! We still need to solve it.
different
from the
The
for solving NP-complete problems are sometimes
techniques
NPseen.
cannot
solve
an
we
that
have
.We
(most probably)
techniques
previously
precisely
problem
complete
have to compromise.
robustness,
in
different
situations,
that
guarantee a
on
bound
in
degree
compromises
completenessof
the
different
compromises.
lead to
Of particular
the
algorithm. So, we
concern the
optimality,
solution.
There are other
a polynomial-time
sacrifice
not
may
algorithm.
approximation
which
resulting
An algorithm
common
most
The
of
all
with
completely
efficiency, or
guaranteed
alternatives as well,
and
the
(or
optimal
are
interest
of imprecision.
We
may
be used
result is called
precise)
approximation
algorithms
see three examples
will
an
that can
of such
later.
algorithms
In Section
6.II,
to be
primality-testing
such
in
P,
but
we discussed
that
testing,
believed to be NP-complete
is not
algorithms,
probabilisticalgorithms
are the
algorithms
approximation
can be used as
algorithms.
We
either.
will not
describe
It is
they requires knowledge of number
theory.
cannot
be
a
solved
problems
by
polynomial-time
mistakes
with low probability
for all inputs.
Therefore,
because
part
of
other
strategies
that
are not
not
\342\200\224
for
known
common.
example,
to be in P
Probabilistic
as part of
NP-Completeness
358
Another compromise
can
We
times.
The problem
exclude
try
with
for
inputs
from
vertices)
well on
the
example, the
the
Another
definition.
We describeseveral
We start
section.
with
These
branch-and-bound.
an
of
two
approximation
We
describe
involves
is an
example
the
for
in
abstract a
which
techniques
an
only
the average
be useful if the
the right
distribution,
work
that
algorithms
completeness
involving
some
only
polynomial
from
problem
the
whose running
For
inputs.
for bipartite graphs
a real-life situation we
special
time
is included
input
times
in the abstract
are exponential,
but
they
may be sufficient.
algorithm or
Backtracking
which
on
isolated
very complicated.
of the algorithms; namely,
is usually
which
can
inputs
in designing
these
random
to
difficult
is
lower
may
inputs
difficulty
efficiently
can be solved
small inputs,
(e.g.,
with
major
is algorithms
example
it
a graph
Finding
them,
problem
example,
assumption.
also compromise
to work
algorithm
well for
reasonably
For
their
when we
Therefore,
that any extra condition
sure
make
follows
very difficult.
vertex-cover
7.110).
(Exercise
work
designed
time
polynomial
Such trivial
average.
for certain types of
can
we
we can allow
the
is analyzing
average
Finally,
should
in
of inputs
is usually
however,
in
is defining
average.
approach
which the particular problem is trivial
participating
distribution
problems
NP-complete
worst-case running
on the average.
for polynomial
requirement
this
Algorithms
significantly.
actual
the
involves
solve
to
them
with
examples
optimal
in
called backtracking
this
and
either
give
algorithms.
and
Branch-and-Bound
Consider
the 3-coloring problem,
techniques
through an example.
under
certain
to
n
vertices
of a graph. This
colors,
constraints,
assigning
of a problem that requires
finding optimal values (colors in this case) for n
these
call
11.5
not
which
important
is why
we can
we choose
colors
with
start
Techniques
For Dealing
with
coloring
the final
permute
of one.
The
359
Problems
NP-Complete
coloring),
which
of these
coloring
two
of children is likely
in coloring
add
we
and
and
two corresponding
continue
the rest
of the
this process.
vertices;
children to
a vertex
After
the
is
the number
therefore,
smaller
Induction hypothesis:We
that
has
know
how
to complete
< k
3-coloring
the
of a
3-coloring
colored, or to determine
graph
that
the
1B,2G
Figure 11.7
backtrack
tree.
An
example
of backtracking
for 3-coloring.
(a) The
graph
(b) The
NP-Completeness
360
Given a
possible colors
k-\\
uncolored
(one
(Gf var
in Fig.
is given
all
its
with
now have
(which
problems
find
and
used for
the vertex
color
we
remaining
The algorithm
by induction.
3-coloring
the
one of them
all colors
If
and solve
a time)
at
vertices)
Algorithm
we pick
colored,
yet
it.
colors
possible
not
to
be assigned
can
that
then the
neighbors,
the
that are
k vertices
with
graph
11.8.
U) ;
ifU =
then
\"coloring is
completed\";halt
else
a vertex
pick
3 do
if no neighbor
add
in U
not
forC := 1 to
to U
C then
color
with color
3-coloring(G,U)
end
Figure11.8Algorithm
It is
problem
that
not
common
is quite
to
hard
come
in a
results
in
\"good\"
order, we
does
not
specify
have
a degree
point
shortly.
up
tree
an
how
find
the solution
to pick
is a
Since
minimum
color
similar
than
just
a yes
3-coloring,
but
3-coloring
for problems
Consider
the minimum
involving
be
to this
the
finding
coloring
colors required
We
3-coloring.
we
next,
return
the general
number of
of branches may
number
will
can build
quite
large.
so far
described
we
be chosen
can
vertex
function.
or no answer for
the
The algorithm
any
of backtracking
variation
the
for
traversal
design heuristics. We
use to
of
of nodes (Exercise11.34).This
the tree in
Our hope is that, by traversing
number
early enough.
can
we
and an order
vertex.
next
the
of freedom that
Branch-and-bound
exponential
algorithms.
backtracking
will
a graph
with
with
3-coloring.
to
a tree
Each
its
can be colored either
by one of the colors already used (unless one
The 3-coloring
or by a new
color.
neighbors already uses that color),
algorithm is thus
modified
in two ways: (1) the constant
number of colors
3 is replaced by the maximal
be
used so far, and (2) the algorithm does not terminate
when
V = U, since there may
better ways to color the graph.
The problem
is that this algorithm
a leaf is reached (i.e.,
backtracks
only when
is
V = U), since a new color can always
be assigned
to the vertex. Thus, the algorithm
almost guaranteed to have poor performance
We can
the graph is very
(unless
dense).
new vertex
improve
the
of
performance
of this
algorithm
by
the
following
observation,
which is
the
11.5
For Dealing
Techniques
method.
Suppose
k colors.
with
with
that we
361
Problems
NP-Complete
traverse the
all the
tree
after
to a
way
backtracking
that
color
steps up the tree, we traverse another
requires
know a better solution.
number k +1. At this point, we can backtrack, since we already
At each node, we compute a lower
bound
on
Thus, k serves as a bound for backtracking.
the tree. If that lower bound
that
can be found farther
down
the best solution
is greater
we backtrack.
One key to making a branch-and-bound algorithm
than a known
solution,
if we want to maximize the
is computing
efficient
bounds,
good lower bounds (or upper
Another
is
a
traversal
order
so that good solutions
function).
good
objective
key
finding
case we can backtrack earlier.
are found fast, in which
of integer
We
illustrate
this idea through the
linear programming
problem
in Section
is also mentioned
The problem is similar to linear programming,
(which
10.3).
with
that
of the variables
but
the extra constraints
the values
are integers.
Let
be vectors
be the vector of variables', ax,a2, \342\200\242\342\200\242\342\200\242,ak
of real numbers,
! = (*!,jt2, ..,*\342\200\236)
and c i, c2,..., ck be real numbers.
The problem is to
each of size n\\ and b\\,b2,..;bk
maximize
the value of the linear objective function
several
the integrality
under
~d\\
* * *
+C2*2+
Z=CXXX
(11.1)
+C\342\200\236Xn
of J
constraints
and the
following
constraints
-X <bx
a2 'X
<b2
(11.2)
ak-x<bk.
(All
tf/S
b(s are
and
constants.)
is NP-hard.
It
can be
easily posed as
is in
integer
to NP
is
complicated.
quite
The following
(The problem here
is
an
is to
such
that
function
Z
= 1 if v, belongs
Xj
is
=X\\
JC2
implies
* * *
that we
to
for
all
n
the
+ *\342\200\236,
want
to
per vertex
0<^; < 1
programming
maximal
the
are
linear
integer
find
which
problems
problems
programming
integer
programming
NP-complete
Many
!</<\302\253,
select
as many vertices
as we can.
There
is one
constraint
NP-Completeness
362
and one
+
Xj
Xj
for
pair of
each
constraints guarantee
therefore,the
linear
Integer
(v,, v;) 4
v, such that
0 or
to either
that are
not
1.
of
set
second
The
cannot
adjacent
a clique.
be solved with
E.
both be
selected;
that
vertices
and
variables
the
vertices
two
that
vertices v,
restrict
of constraints
set
first
The
<
vertices
pair of nonadjacent
each
for
constraint
can
programming
branch-and-bound
the
using
by
of
kind
This
be
can
information
helpful
farther
As
tree.
the
down
in
regular
optimal
solution
since
impossible,
(e.g., set
the
values
program. If at some
where a is smaller than
Thus,
us to
allows
the
when
we
are
the
other
the
next
the
linear
done
and
with
program has a
maximal
program
backtrack,
prune
of the
whole
point
the
we
It
gives
bounds
\"most
likely
subproblem
to explore.
This
heuristic
depends
done by
that consists
breadth-first
of
the
search,
best
solution
depth-first
of z=tf,
on
will
when
thus
the efficiency
and
the particular
the
and
hope that,
be able to
We
\342\200\224
and
of pruning
The amount
subproblems
substantially.
\342\200\224
the
divide
to
on
the
heuristic
algorithm
problems
depends
approximation
can be
upper
hints at which
it also
value
maximal
can backtrack.
to choose
application.
all subproblems
and obtain
of the tree
algorithm
An
extreme
11.5
example of
example,
the
in
(the
idea
This
is a
solution
first
we can
algorithm,
with
color the
by
outcome
the
by a certain heuristic)
of the algorithm.
order of
in reverse
vertices
the color
fixing
363
Problems
NP-Complete
(chosen
path
leaf) as
at a
(usually
coloring
that we
being
the
is talcing
early
terminating
a feasible
to
leads
For Dealing
Techniques
of a small-degree
that
For
degree
vertex).
simple greedyalgorithm.
11.5.2
with Guaranteed
Algorithms
Approximation
Performance
In
this
section,
vertex cover,
for
three
NP-complete
salesman
is,
problem.
we can
problems:
All these
prove
that
the
Cover
Vertex
for finding
the minimum vertex cover of
algorithm
find
The
is
to
a
cover
that contains no more than
graph.
algorithm
guaranteed
twice
cover. Let G =(V, E) be a graph
the number of vertices containedin a minimum
in G. Since M is a matching,
and
let M be a maximal
its edges
have no vertex
matching
in common,
and since M is maximal, all other edges have
one vertex
in common with at
in M.
least one of the edges
We
with
start
a simple approximation
a given
11.9
D Theorem
set
The
of all
vertex cover
size
vertex
Therefore,
set of
is
at least
to
incident
more
the edges
than twice
the
of a
number
maximal
matching
of vertices
of a
M is
minimum-
a vertex
cover, because M is
belong to M forms
in particular,
the edges of M.
cover all the edges \342\200\224
cover
more
than one edge of M.
a matching, a vertex of M cannot
\342\226\
half of the vertices
of M must belong to every vertex cover.
vertex
Every
since
But,
no
cover.
Proof: The
maximal.
vertices
with
vertices
that
cover must
maximal matching
can
find a
by
simply
collecting
edges
until
all
in
One-Dimensional
Bin Packing
bin
packing
sizedbins using
as
364 NP-Completeness
The Problem
0 and
between
The one-dimensional
in
problems
blocks need
bin
put
the
the
for this
bin
has
that
The
than
twice
fact
that
sizes (in
be placedin
the sum
the
of
of bins
case
all items
which
a new
Bin
bin,
and
the
packing
for each
then,
bin if there is no
fit is
First
algorithm.
bound
20 PT
room in
not \"too
less than
half
any
bad\"
/, to
of
in
Therefore, the
items (rounded up). The theorem
best solution cannot be less than
the
the
full;
of all
in
is
OPT
where
bins,
first bin.
the
sizes
the
number
It turns
fit cannot
First
bin could
in the first
at most
requires
of bins.
number
Proof:
second
problem is to put x,
room for it, or to start
is called
the first fit
in the next theorem.
algorithm
fit
first
memory, and
11.10
\342\226\241
Theorem
minimum
memory-management
of available memory.
chunks
large
in
blocks of
different-sized
many
problem (Exercise11.8).
heuristic
Xj in the first
problem
packing
is at
subset
each
is an NP-complete
One
in
of real numbers
each
subsets as possible
most 1.
into as few
numbers
the
a set
be
Xy,x2,...,xn
of numbers
the sum
that
such
Let
1; partition
from
follows
the
by Theorem
given
the
of all the
sum
\342\226\
The
conservative.
quite
constant
of
in
the
can be
theorem
proof).
11.11
\342\226\241
Theorem
OPT is the
This constant
is
The
natural
strategies
fit
also
tight.
methods
we
First
leading
described
at most
\342\200\224
OPT
+ 4
bins, where
of bins.
number
minimum
requires
algorithm
and
fit
decreasing
to better
are typical
approaches corresponding to
what
first
constants.
one
are
fit
In
most
of heuristics
would
both
cases,
They
algorithms.
probably
do
simple heuristics.
the analysis is
by
hand.
present
We have
11.5
seen
large
it
Therefore,
inputs.
that straightforward
is very important
however,
times,
many
to
the
analyze
365
Problems
NP-Complete
for
poorly
quite
of these
performance
algorithms.
Euclidean
Salesman
Traveling
The traveling
The Problem
Let
corresponding to the
iltonian cycle (traveling
The problem is
still
designing
somewhat
by assuming only
the direct
distance
the
tree
but
we
cities;
algorithm for
that
the
between
of
in
points
the
more
the Euclidean
(We
problem.
two
satisfy
the
the
length
of the
This
this
the
than
which
route
any
tree
claim
(here,
that
through
cost =
the cost
in
assumption
inequality,
triangle
is shorter
points
assumption helps
can relax
than
many
that
plane
Ham-
a minimum-distance
that
see
the
distances
any
a set
find
with
constraint
additional
among them.
tour)
will
problem
important
with the
starts
is a
which
is no
of n
salesman
states that
other points.)
The algorithm
be
...,C\342\200\236
C{,C2,
location
NP-hard,
an approximation
distance),
an
We discuss
applications.
of
is
it a
containing all vertices; therefore,removing
any edge
minimum-cost
tree.
whosecost
is
thus
at
least
that
of
the
spanning
spanning tree,
A spanning
to a TSP tour. We need to
does not correspond directly
tree,
however,
traversal
of the
search
it. First, consider the circuit
that consists of a depth-first
modify
in the opposite direction whenever the
from
tree (starting
any city), and includes an edge
a tree-shaped
search
backtracks.
(This circuit corresponds, for example, to traversing
to
the
with
on
exhibits
both
sides
of
hall,
going
right.)
Every
every
by always
gallery,
edge will be traversed
exactly twice, so the cost of this circuit is twice the cost of the
TSP
the cost of the minimum
twice
minimum-cost spanning
which is no more than
tree,
direct routes instead of
tour. We can now convert
this circuit into a TSP tour
by taking
the same
(see Fig. 11.9). That is, instead of backtracking
using
backtracking
always
to the first new vertex. The assumption
that
the distances
are
edge, we go directly
Euclidean is important,
because
it guarantees
that the direct route
between
any two cities
is always at least as good as the nondirect
route.
The length of the resulting
TSP tour is
of the minimum TSP tour,
thus
still no more than twice the length
it
is
often less
although
a cycle
than
that.
The running
Complexity
the minimum-cost
O (n
log
(see,
\302\253)
spanning
for example,
time of this
tree
algorithm,
Preparata
and
algorithm
which,
Shamos
is dominated
in
the
[1985]).
case
time
of
by the running
of Euclidean graphs, is
NP-Completeness
366
(b)
(a)
and
point,
(b) A
tree,
spanning
TSP
the tree
from
obtained
tour
by
starting
at the
right first.
going
Improvement
The
Another
tour.
the
tree,
from
have
we
algorithm
4'sloppiest\" part
way
to
by repeating
the
Eulerian
just
the
of
described can be
is the conversion
algorithm
look at this
each edge
effectively.
An Eulerian
odd-degree
nodes
in the
graph
then
can convert
We
circuit.
from
conversion is that
twice. We
must
include
be
in the following
the tree traversal
improved
it builds
obtain
the
tree
The
way.
into
a TSP
the TSP
into
an Eulerian
only even-degree
an even number
graph
nodes. Considerall
of them (otherwise,
more
the
the
total sum
be
odd,
which
We
Gabow
for general graphs
in 0(n3)
(see
perfect matching can be done
works
or
that
Lawler
to
due
is
a recent algorithm,
[1976]
[1988],
[1976]). There
Vaidya
for the special
is a
this
case of Euclidean distances in time
(Whether
0(n25
(log\302\253)4).
better algorithm
in practice
is not clear.) The final TSP tour is then obtained from the
Eulerian
tree plus the minimum(which includes the minimum-length
graph
spanning
shortcuts.
The
TSP tour obtained
length
matching) by taking
by this algorithm for the
11.11.
tree in Fig. 11.10 is given in Fig.
minimum-weight
11.5
Techniques
For Dealingwith
(b)
(a)
Figure 11.11The
tree
spanning
plus
minimum
Eulerian
circuit
the matching,
and
its
obtained
TSP
corresponding
(a) The
tour,
circuit.
the Eulerian
from
11.12
\342\226\241
Theorem
times the
367
Problems
NP-Complete
length
a TSP
produces
of the minimum
tour whose
length
is at
most 1.5
TSP tour.
the shortcuts (since there may not be any in the worst case),
The circuit consists of the tree
circuit.
length of the Eulerian
of the tree is at most the length
and the matching. We have already seen that
the length
TSP
of a minimum
to prove that the length of the matching
is
tour;
hence, it is sufficient
of a minimum
TSP tour.
Let Q be a minimum TSP tour.
is
a
at most half the length
Q
all vertices.
Let D be the set of odd-degree vertices in T. We can obtain
cycle containing
of D such that the sum of their lengths
two
is no more than that of Q in
disjoint
matchings
the following
way (see Fig. 11.12). We start with an arbitrary vertex v of D and match it
is its closest neighbor clockwise in Q. We then continue matching
in
to a vertex of D that
direction. If the matched
a clockwise
vertices
are not neighbors in Qy then the distance
between them is no more than the length
of the path connecting them in Q (by the
is obtained
This
us
one
processgives
triangle property).
matching. The second matching
of both
The sum of the lengths
counterclockwise.
by repeating the same
process
in Figure
11.12. But, since M was a
matchings is at most the length of (?, as is shown
of D, its length
is at most half the length
of Q.
\342\226\
minimum-weight
matching
Proof:
and
We
Finding
minimum-cost
whether
it
achieving
type
of
problem
will
a minimum-weight
but
tree,
spanning
is possible
this
ignore
on the
concentrate
will
\342\200\224
and
to improve
This
algorithm
We abstract an easier
constant.
algorithm:
either
perfect
it results
then
design the
heuristic
takes
matching
in
a better
much
bound.
It is
than
longer
finding
an
open problem
the
constant
of 1.5, or to find
a faster
algorithm
illustrates one of the main characteristics
of this
problem
accordingly.
\342\200\224
or relax
still
some parts
of
the
original
NP-Completeness
368
Figure 11.12
11.6
The
Two matchings
is at most
sum
whose
of the
that
TSP tour.
Summary
have generated
should
chapters
previous
somedeservedoptimism
our
about
ability
The
discovered
was
result
similar
[1972] presented
included
this
work,
was
of NP-completeness
notion
in
Section
11.4).
their Turing
and
the
in
24
of
list
Further
and
Notes
Bibliographic
Both Cook
the
and
Union independently
Karp
were
lectures
Several
of languages
acceptance
different
requirements
is PSPACE,
turns out that
which
polynomial
space
nondeterministic
in
are
Cook
by a nondeterministic
other
algorithm
words,
can
award
[1983],
not known to
a Boolean
from
NP
are
which
due to
in part
and
be
in
in
Karp
P have
of all problems
expression is always
of
is that the definition
[Savitch 1970].In
Karp
[1973]).
the complements
algorithm
for accepting
and
all problems
contains
nondeterminism
polynomial space
by
the Turing
published
Cook [1971](a
Levin
by
problems (some of
received
of problems
that
been studied. One such class is co-NP, which
contains
in
NP.
For example,
the problem of deciding
whether
false
to
co-NP. The reasonco-NPis different
belongs
[1986].
paper of
seminal
NP-complete
important
award
in
introduced
Soviet
Reading
algorithm
can
It
space.
polynomial
is limited to
be
polynomial
solved
in
space
Notes and
Bibliographic
is an example of a problem
where
(Even
[1976]),
Tarjan
completeness is defined
is
to
the
it
for
defined
the
class
that
the reductions
can use
NP, except
similarly
way
There
of
is
classes
P
a
between
and
PSPACE
called
the
polynomial space.
hierarchy
to note that at present there is no proof that
polynomial-time hierarchy. It is important
PI We do not know
of any problem
from
any of the classes mentioned aboveis different
to PSPACE and does not belong
that
to P. There are, however, problemsthat are
belongs
by
a deterministic
that
The generalized
algorithm.
for PSPACE
is complete
known
more
HEX
game
and
and
[1972]), or even
and Stockmeyer
(Meyer
space
(Fischer
There are
The
several
are the
that are
problems
ones
notable
most
to be
known
not
graph isomorphism
times for
in polynomial
either
and
in NP-complete.
P or
Graph
testing.
primality
(see, for
cases
special
many
in
example,
and Solovay and
Rabin
[1976]
problem is still open.
for
Monte Carlo probabilistic algorithms
primality
testing.
determine
that a number is prime with
little
error
(These algorithms
very
probability, and
make
no
when
number
is
not
error
determine
that
a
a
Goldwasser
and
they
they
prime.)
a
that
tests
Killian
Las
without
[1986]
present
Vegas probabilisticalgorithm
primality,
is polynomial.
Another seemingly simple problem
time
errors, whoseexpectedrunning
which is to determine whether
that
is still open is the
a given
even-cycle
problem,
directed graph contains a (simple) even-length
Manber
(see Klee, Ladner, and
cycle
A wealth of information
and
related
can be
about
[1984]).
subjects
NP-completeness
column by Johnson
in Garey
that
found
and Johnson [1979] and in an NP-completeness
A
since
natural
to
ask
is
has been appearing in the Journal
1981.
of Algorithms
question
in NP are either NP-completeor are in P. This question was
whether
all problems
answered
that, unless P = NP, there are infinitely
by Ladner [1975] who
proved
partially
in between.
classes
many
An
that runs in polynomial
time on the average for the
Hamiltonian
algorithm
one
for
is
is
in
and
Valiant
and
described
[1979],
satisfiability
cycle problem
Angluin
for
the
and Brown [1985b]. An approximation
described in Purdom
weighted
algorithm
Strassen
vertex-cover
Pitt
with
These
[1988] for a
precise
performanceof
for k
these
(see,
and
estimating
given
by
Horowitz
the
Pearl
is
and
are given
by
Brelaz
\"almost
for
for k = 5.)
for
it is
However,
Sahni
Heuristics for
and
graphs
average
Turner
Turner
size of
independent
the average
best
by
(see
than
the
of n.
good
to the
known
guaranteed
given by Wigderson [1983]. Backtrack
and Baumert
and
[1965] (see also Bitner
Reingold
Knuth [1975] describes a technique
for
[1978].)
coloring
Golomb
cover. Gusfield
[1979],
all\"
finds
algorithm
the minimum-weight
of this algorithm.
1 million
of
[1981]. The
are due
graph
fact,
k-co\\ohng
graph
algorithms
of the algorithms
for
bound
approximate
[1975]
In
= 3, and
strength
that
explanation
successful
proven
definition).
tree for
200
intuitive
(for fixed k)
k colors
heuristics are
backtrack
most twice
is at
Even
and
Bar-Yehuda
in
a more
present
coloringgraphs
simple
is given
whose weight
[1986]
[1988].
general
problem
vertex cover
and
the
present
[1977]
is
programs.
A general
discussion on heuristics is
NP-Completeness
370
The
an
The
traveling
in
problem
salesman
terms
problem is given
1.5 for the Euclidean
problem
An
and
problem,
and
including
very close to
[1982].
Karp
algorithm
approximation
in Section
given
studied
most
the
is due
algorithm,
Kan,
is probably
fit
that comes
heuristic
solutions.
of proposed
by Lin and
first
decreasing
Johnson
of the bin-packing
solution
optimal
the performance
of first fit and
et al. [1974]. For another
on
bounds
NP-complete
for the general
bound
the
achieving
of
Lenstra,
by Lawler,
most
of this
aspects
algorithms, special
Rinnooy
problem,
cases, and
applications.
Exercises
Drill
the NP-complete
only
problems discussed
the
in
text or
in
given
in
other
11.1
Prove
that
all problems
in
P are
polynomially
according
equivalent
to the
definition
Section 11.2.
11.2
Prove
that
not allow
11.3
11.4
of
the definition
one algorithm
Write
the
3SAT
expression
different
language
does
languages.
that
Draw
the
Section
Draw
the
graph
Section
11.4.5)
the knapsack
the reduction
reduction of SAT
to
3SAT
(given
in
from
obtained
for
of SAT to
the reduction
the expression
\342\200\242
(x +y
+ U+V)'(X+y).
the
clique
problem (given
in
+ z).
+ v+z)-(jc+y
that is
{x +y + z) \342\200\242
(x + v +F)
from the
+ W+U+V)'(X+y+'z+W
from
graph that is obtained
11.4.4) for the expression
(given in
is obtained
the expression
(x+y+^+W+U+V)'(x+y+Z
11.6
two
Section 11.4.3)for
11.5
algorithm recognizes a
a nondeterministic
how
to recognize
+z).
problem is NP-complete.
of 3SAT to
the
3-coloring
problem
11.8
11.9
Pose
You
integer
linear program.
Exercises
exercises using
chapter.
complete these
exercises from this
can
11.10 Prove
only
the NP-complete
problem is NP-complete:Given
contains a spanning
the following
that
k, determine whether G
an integer
problems discussed
an undirected
tree
T such
that
in
the
text or
in
< k.
degree
11.11
problem as an
the 3-coloring
Creative
other
problem is NP-complete.
that
Prove
Prove
vertex-cover
the
that
graph are
restricted to
have
if
all
the vertices
in
the
even degree.
in Chapter 5.
discussed
11.12 Consideragain the problem of finding large induced
subgraphs
has
Suppose that, instead of the requirement that each vertex in the induced subgraph
is the formulation of the problem
in
that its degree be < d. Here
degree > d, we require
an undirected
and two integer
terms of a decision problem. Given
graph G = (V, \302\243),
parameters d and k, determine whether G contains an induced subgraph H with at least k
vertices, such that the degree of each vertex in H is < d. Prove that this problem is NPcomplete.
11.13
the following
that
Prove
G=(V, E)
integer
problem is NP-complete:Given
contains a subset of
the following
that
an undirected
problem is NP-complete:Given
k, determine whether G contains a clique
k, determine whether G
an integer
connected graph
size k and an
of
k.
set of size
independent
11.14 Prove
an
and
graph G = (V, E)
whose induced subgraph
an undirected
k vertices
and
is
acyclic.
11.15 Let E
be a CNF
expression
once as x. Either
are satisfiable
11.16Prove
as the one
assignment such that
variables
satisfying
assignment
Use
Exercise
11.18The input
looking
exactly once
whether
as x and
exactly
such expressions
of 3SAT,
The problem
with
is NP-complete: The input
is a Boolean expression in CNF
whether there exists a
clause, and the problem is to determine
such that in every clause exactly 2 of the 4 variables are true. (Hint:
per
is again
whose
for
x appears
11.16.)
there exists a
variables
in
2-in-4SAT
that
Prove
exactly 4
variable
that
each
algorithm to determine
that this problem is NP-complete.
or prove
the same
11.17
such that
a polynomial-time
find
CNF.
The problem is to determine
whether
that every clause contains
an odd number of
are 1. For example, if the input is a 3SAT input,
then
we are
such that, in every clause, either 1 or 3 variables have value
1.
a Boolean
satisfying
values
assignments
expression
assignment
such
in
372
NP-Completeness
(Another way
This
operations.)
11.17, but
in
may
is
The input
undirected
an
exact coverby
11.20 The
3-sets
(X3C)
and a collection of
elements
, S,,
is to
that each
such
Sin
the
regular
the clique
problem is
of
subsets
determine
the
The input
following.
element of S is contained
is a set S
eacn
\342\226\240$*\302\273
containing
S, S,,S2
with
same
3n
exactly three
of
subcollection
subsets
in exactly
is NP-complete.
X3C
11.21 Prove
following
problem is NP-complete:Given
determine
whether the vertices of G can
the
that
3a7
with
vertices,
contains
three elements,
whether
determine
graph G=(V, E)
into n groups, such
each other. In other words, the
undirected
an
be partitioned
each connected
to
n triangles.
into
U, E) be a bipartite graph such that V is the set of vertices on one side, U is the
V corresponds
vertices on the other side, and E is the set of edges connecting them.
to
is
and U to a set of parts. A machine
connected
to
a part u} if the
a set of machines
v,
machine is used to work on that part. Suppose that a room can accommodate
at most K
machines and unlimited
number of parts (for simplicity,
we assume that all machines
have
the same size). We assume
that we have as many
rooms
as needed, but we want
to
minimize
the movements of parts
from
one room to another.
Each
edge (vhpj) is
associatedwith a cost c (v,, py), which is the cost of moving part u} to machine v, if the part
and the machine are not in the same room. We define
the cost of a partition of machines
and parts into rooms as the sum of the costs of the edges
that
parts to machines
connecting
are not in the same room. Prove that the following
the
problem is NP-complete:Given
whether
it is possible to
graph G, the parameter K, and another parameter C, determine
the machines and parts into rooms
with cost <C. (In other words, the set of
partition
vertices should
be partitioned
into subsets, each with at most K vertices from V, such that
the sum of the costs associated with the edges that connect
subsets
vertices in two different
does not exceed C.)
set of
11.23 Let
four
of S each with
set, and C = IC,, C2,...,Ck} be a collection of subsets
Two subsets of S are said to be connected if they contain a common element. A
C is said to be a cycle if C, is connected to C/+1 for all /', 1 <i<k-\\,
and C] is
to Ck. An subcollection C'^C is called acyclic if it does not contain a cycle.
S be a
elements.
collection
connected
Prove that
element
in
11.24 Assume
that
the
Hamiltonian
the
are defined
that
NP-complete:Determine
such
that
(1) every
element of S is included
whether
two
subsets
in at
there
have
collection C
at
most
one
of C.
Prove
is NP-complete.
graphs
path problem for undirected
cycle problem for undirected
graphs is also NP-complete. (Both
in
Section
11.4.)
is an
a given
in C
Hamiltonian
Hamiltonian
input
Prove
common,
that
the
problems
11-25 The
problem is
subcollection C
following
an acyclic
contains
vertices v and w in G.
whether G contains a
373
Creative Exercises
11.26 Consider
whether
a graph G=(V, E) contains
the problem of determining
a
path with given end vertices v and vv (see Exercise 11.25). Let G' be the graph
obtained by adding two new vertices v and vv and two new edges (v, v) and (vv, vv). If G'
contains a Hamiltonian path, then its two end vertices must
be v and vv. Therefore,
such a
in
G
with
the
v
a
end
vertices
vv.
to
Hamiltonian
and
What
have
we
path corresponds
path
again
Hamiltonian
just proved?
11.27
that
Prove
graph
undirected
graph
^-coloring
G=(V, E)
is NP-complete. The
an
and
integer
is
problem
k, whether G can
to determine,
be colored with
given
an
at most
colors.
is a polynomial-time approximation
that can color any graph
algorithm
the minimal
number
of colors required to color that graph, then
if there
with less
than
times
4/3
P=NP.
11.29
Prove
of at
11.30
most
Given a
problem, called feedback edge set, is NP-complete:
E) and an integer parameter k, determine whether G contains a set F
edges such that every directed cycle in G contains at least one edge from F.
the
that
directed graph
k
following
G =(V,
vertices
whether
there
be an
undirected
Prove
exist k paths in G
of G.
graph,
w,),
(v2,
(vk, wk) be k
w2)
pairs of
problem is NP-complete:Determine
following
such that path / connects v, to vv,, and all paths are vertex
that
the
disjoint.
11.32
11.34
are
graph such that the edges incident to each vertex
nodes.
order. There is no relationship
between ordering at different
is to find an Eulerian tour covering
G that
Suppose further that G is Eulerian. The problem
v at an edge e,
If the tour enters a vertex
satisfies the following \"noncrossing\" property:
then
the next edge in the tour must be adjacent
to e in the cyclic order imposed on the edges
incident to v (from either side of e). Oneway to view this property is to look at a road map.
The goal is to travel through all the edges (road segments) such that an intersection (vertex)
is never crossed except from one edge to its neighbor. Prove that determining
whether such
an
tour exists for a given graph and cyclic orderings is NP-complete. (The
Eulerian
for planar graphs, but the proof is more difficult.)
remains
NP-complete
problem
E)
be an undirected
ordered
in a cyclic
Show an
example
3-coloring
in
is a
the simple
which
a graph results
in
exponential
backtracking algorithm
number
finding
described
in Section
11.5 for
of nodes.
a vertex
cover.
In
each
step of the
algorithm,
374
NP-Completeness
is added to the cover,
then it is
of highest degree (ties are broken
vertex
arbitrarily)
terminates
when
removed from the graph together with all its incident edges. The algorithm
an edge is removed only
after
a vertex incident to it is
Since
no more edges remain.
indeed finds a vertex cover. This is a greedy algorithm
included in the cover, the algorithm
of
since it always selects the vertex with the highest \"payoff.\" The worst-case behavior
of a graph and an order of execution of
this algorithm is not very good. Show an example
cover
with more than twice the number of vertices of
that leads to a vertex
this
algorithm
the
the
11.36
cover.
minimum
Show
of the
instance
an
11.37 You
are
a traveling
you
are
planning
bin
more
salesman of
your
flight
1980s.
the
algorithm
Your boss
You do not
itinerary.
first-fit
gives
receive
will
suggest
11.38
approximation
algorithms for
it.
certain university,
and let f,, t2,..., tn be
which the courses are offered. For
example, f, may be Tuesday from 10:00 to 11:00,t2 may be Tuesday from 10:30 to 12:00,
and so on. Your job is to assign classroomsto the courses.
The only requirement
is that no
two
courses
is to use the minimal
overlap at the same time at the same room. The goal
number
of classrooms that satisfies
the requirement.
a. Reduce this
b.
Discuss
problem
to a coloring
complete problem).
at a
during
problem, and
does not
imply
design
that P = NP
an efficient algorithm
(even though
coloring
to solveit.
is an NP-
12
CHAPTER
ALGORITHMS
PARALLEL
with one
person
a person
with
two
what time
is never sure.
knows
watch
watches
it
is;
Anon
small
first place, it is to be remarked that, however
the republic may be, the representatives must be raised to a
certain number, in order to guard against the cabals
of the
few; and that, however large it may be, they must be limited
to guard against the
to a certain
number, in order
confusion
of the multitude.
In the
James
1787
Introduction
12.1
The
Madison,
subject
science within
computer
science).
from 2 to
65,536 processors.
The
differences
to mainstream computer
the exotic
fast (even relative to other areas of
of parallelcomputers
in operation,
ranging
the various existing parallel
We can no longer adopt
are major.
between
user is concerned,
and hope that it adapts to all parallel computers.
computation
them
correct
is much more
them, and proving
Designingparallel
algorithms,
analyzing
than
the corresponding
difficult
steps for sequential algorithms.
to cover all (or even most) areas in parallel
We
cannot
chapter
hope in this short
of examples
and
We present a variety
using different models of computation
computing.
We try to give the flavor of parallelalgorithms
and
to explore the
different
techniques.
with
them. We start
characteristics. We then
some common
in designing
difficulties
of
models
follow
used in this chapter, and
the main
describe
parallel
computing
briefly
as
the naive
of
ins
Parallel
376
Algorithms
and techniques.
are running
time and
for sequential algorithms
in
as
These
utilization.
well, but we
parallel
algorithms
important
space
resource
is
the
number
other
resources.
One
about
of
also worry
must
important
and they
cannot
There are problems that are inherently
be
sequential,
processors.
number
of processors is available. Most other
even if an infinite
\"parallelized\"
we use \342\200\224
can be parallelized to a degree.The more processors
however,
up
problems,
\342\200\224
the
faster
the algorithm becomes. It is important
to study the
to a certain limit
to be able to characterize the
that have
of parallel algorithms, and
limitations
problems
is
Since
the
number
of
solutions.
fast
limited, however, it is also
processors
parallel
very
Another
use
the
to
important issue is communication
processors
effectively.
important
to exchange data between two
the processors.
longer
Generally, it takes
among
than it does to perform simple operationson the data.
some
Furthermore,
processors
be
\"close\"
to
one
whereas
other
be
farther
another,
processors
may
may
processors
it is important
to minimize
and to arrange it in an
communication,
Therefore,
apart.
issue
is
which
is a major problem
effective way. Yet another
important
synchronization,
that
run
on independent
machines connected by some
for parallel algorithms
network.
Such algorithms
are
called distributed
communication
usually
algorithms.
in this book.
For lack of space,we will not consider distributed algorithms
We discuss
of algorithms
examples
with
The main
that assume
models
only
instruction
SIMD
full
each
in
step.
example
execute a
different
program
machines.
Unless specified
prominent
12.2
We
to
only
We
p
is
case we say
be
taken
utilization
of the
model.
the MIMD
section,
the
parallel
algorithm
from
the best
of the
processors is
\302\243(\342\200\236,p)=am
achieves
known
The
ratio
efficiency
and
one type
S(p)
book.
of model,and
this
in
that
definitions
of algorithms for
/?), where
is most
a perfect
sequential
the
used
apply
a
includes
it.
by T(a?,
algorithm
algorithm
of this
is the
input size
speedup.
algorithm.
of a parallel
The
value
An important
algorithm,
which
of T (a?,
1) should
measure of
is defined
the
as
= j>o>_.
pT(nyp)
and examples
model
scope
that are
the
those
discussion
covers
sections
time of an
on
emphasis
some general
of processors.
algorithm.
that
can
processor
Multiple-Data)
(Multiple-Instruction
following
the running
the number
of the
speedup
machine is a
each
which
in
exact
the
are called
restriction
connection
The
assume
description of the
denote
MIMD
called
are
this
follow
machines.
otherwise, we will
to execute
all processors
that
Parallel computers
computers.
this
in
Each
models.
more detailed
and
a few
We include,
many
computers
of parallel
survey
comprehensive
chapter.
restrict
of Parallel Computation
Models
mention
computation
Parallel
Multiple-Data)
(Single-Instruction
of such
synchronization.
of parallel
models
Some
same
of complexity
measures
are
measures
ratio
of the
time used by
one
processor
12.2 Modelsof
and
time
total
the
used
by p
by the number
time that is not
multiplied
processors'
wasted,
Parallel
actual elapsed
is the
time
time
of
the percentage
indicates
compared
377
Computation
the
p)= 1,
If E(n,
then
done by
of work
amount
all
throughout
processors
efficiency.
we could
fix p, according to the number
we design a parallel algorithm
of
and
to
minimize
available
to
so
would
us,
T(a?, p). But, doing
try
processors
potentially
whenever
the number of processors changes. It is more
require a new
algorithm
that works for as many
values
of p as possible. We discuss
desirable to find an algorithm
next how to translate
that works for a certain value
an algorithm
of p to algorithms for
When
smaller values of p,
modify
an
factor of k
of
not
T(n,p)=X
In other words, we
more time.
to
steps in
significantly.
efficiency
an
with
algorithm
can use a
factor
the modified
construct
We
with k
algorithm
original
the
changing
with
algorithm
k>\\.
constant
without
k less
of
algorithm by
emulates
one processor
which
In
T{n, plk)~kX,
processors running
replacing
the
processors
emulate
principle,
for
any
for
step of
the
of one
step
For example, k may
of the
pattern
execution
processors.
divide
each
can
we
general,
called
processors
to
the parallelism
processors without
S(n) = n/\\og2n
Suppose
The
that
now
running
folding
the
changing
\342\200\224
a
very
time
of the
is possible),
if p
2 with
efficiency
speedup
impressive
is 7(1024,
algorithm
is a
which
-16,
\342\200\224
and
of processorsavailable
parallel
that
then
speedup
the
running
of
about
the
to
efficiency
us is
256)
25
over
time would
be 640, which
16 processors).
in the way
the processors
computation differ
mainly
will
and synchronize. We
consider
full
assume
only models that
and concentrate on different
communication
The sharedparadigms.
synchronization,
there is a random-access shared memory,
that
that any
such
memory models assume
with unit cost. This assumption
variable
of unit-cost
access
processor can access any
or
the
is
size
of
of
but
it is a
the
of
number
the
unrealistic,
memory
processors
regardless
communicate
models
of parallel
Parallel Algorithms
378
most difficult
are connected
model
Other
hardware.
in
network.
interconnection
an
through
way to
easiest
the
usually
to implement
model
the
but
communication,
assume that
An interconnection
models
handle
they
way
12.3.
several
discuss
memory is
Shared
models differ in
shared-memory
different alternatives in Section
The
the
the
it is
processors
be
can
network
and two
correspond to the processors
them.
vertices are connectedif the corresponding
processors have a direct link between
has a local memory that
it can access quickly. Communication is
Each processor usually
to arrive
at their
which
done through
may have to traverse several links
messages,
the
of
the
communication
on
the
distance
Therefore,
destinations.
speed
depends
different
have been studied as
the communicating processors. Many
between
graphs
will
several
networks.
We
mention
12.4.
interconnection
popular ones in Section
with
are
Parallel
message
computers based on interconnection networks
passing
sometimes calledmulticomputers.
that we discuss is that
of
A systolic
model
Another
systolic
computation.
line.
The data move through
the
in a
architecture resembles an assembly
processors
and
are
on
them.
Instead
of
fashion,
very simple operations
performed
rhythmic
having
a shared (or nonshared) memory,
the processors
receive their input
from
to access
their
are discussed
in Section
operate on it, and pass it on. Systolic algorithms
neighbors,
12.5.
A basic
theoretical
model that we will use only for illustration purposes is that of a
A circuit is a directedacyclic
circuit.
in which the vertices correspondto simple
graph
and
the
show
the
movement
of the operands. For example,a Boolean
operations
edges
circuit
is one in which
all indegrees
are at most 2 and all operations are Boolean
vertices
for input (with indegree of
(or,
and, and not). There are designated
operations
0), and for output (with outdegree of 0). The depth of a circuit is the longest path from
by a
represented
an input to
12.3
A
an
depth corresponds to
The
output.
fully
consists
of steps.
algorithms
each
synchronized
In
each
of several
consists
computer
shared-memory
step,
in
Exclusive-ReadExclusive-Write
Write
(CREW)
the
The
result of
alternatives
two
EREW
processors and
a shared
We assume
that
but
we assume
they
way
on
computation
the data
(In
handle
memory
We
memory.
the
it
practice,
global.)
The
conflicts.
same
time, but only
Concurrent-Write
(CRCW)
at
Machines
and
processors
to handle
model
(EREW)
location
same
the
time.
running
parallel
section.
this
to access
the
for Shared-Memory
Algorithms
use only
the vertices
that
such
graph,
one
at the
processor
model poses no
CREW
models
at the
write.
can
Finally,
the
Concurrent-Read
on memory conflicts.
defined, but it is not clear
restrictions
are well
what
is
the
for Shared-MemoryMachines
12.3 Algorithms
location
to
we
model
CRCW
values to
different
write
Surprisingly, as we
assume that
same location
the
to
write
another
We
a simple
with
start
problem
looks
that
thing.
at the
same
12.3.2,
If two
the
at
is
an
that
processor
arbitrary
the
to
write
processors
the
time,
is very
when
that,
feature
this
to
processors
same
same
attempt
halts.
algorithm
powerful.
Another
several
processors
processorsucceeds.Yet
succeeds.
Addition
Parallel
12.3.1
in Section
assumption
possible
several
the
location
same
the
see
will
is to
alternative
\342\200\224
allows
same
the
at
book
this
in
utilize
379
example of a
The Problem
Find
of two
sum
the
at first glance.
sequential
inherently
by induction, for
developed
algorithm,
parallel
AZ-bit
numbers.
binary
adds two
and
or
not be
may
at a
bits
of the
outcome
/th
step
a carry.
Nevertheless,
to az, since
this
go from n-\\
The
has
a
much
divide-and-conquerapproach
implies
for parallel
better potential
algorithms, since it may be possible to solve all smaller parts
in parallel.
into two subproblems of size nil (we
Suppose that we divide the problem
We can find the sums of the two pairs in
that n is a power of 2 for simplicity).
assume
If the sum of the least
But we still have the problem of the carry.
significant
parallel.
pair.
pair has a carry, we have to changethe sum of the most significant
have a
are only two possibilities \342\200\224
we either
The key observation here is that there
to
include
we
can
the
induction
or
we
do
not.
Therefore,
strengthen
hypothesis
carry
We
without
We
The
cases.
both
initial
an
pair
significant
we
or
exceptthat
no
with
sums
find
also
either
carry.
numbers: L,
get four
corresponding
problem is to find
Suppose that we now
modified
whether
initial
for the
it
generates
which
/?c,
the
carry,
a carry.
on whether
/?c, depending
L is replaced
by Lc.
of size
solve a problem
same
pair,
significant
The
L had
sum
the
of the
correspond
problem for
sum
with
respectively.
final
a carry.
sum
to
final
sum
initial
an
For each
5, without
The
the
sum
initial
and
with
numbers
two
this modified
solve
of
carry,
both
pairs.
the least
and
the
of these sums,
carry,
Sc is the
is L
and
same as 5,
We
number
Parallel
Algorithms
for example,
Ladner and
parallelization
of an algorithm.
380
in parallel,
quickly
very
12.3.2
Fischer
in an
of
example
an
easy
further improved.
maximum
the
Find
a good
that the
Algorithms
Maximum-Finding
The Problem
is
it
clear
becomes
it
can be
solution
the
but
[1980]),
Once
among
n distinct
numbers,
given
array.
We solve this
The algorithms
for two
problem
both
for
EREW Model
The
algorithm
sequential
straightforward
comparisons.
of a
think
can
We
of the
for
n -1
requires
the two numbers,
maximum
the
finding
comparison as a game
between
played
to
problem is thus equivalent
whole
set.
An
efficient
being
in parallel
is to use a tree. The players
are divided into pairs for
way to run a tournament
the
first round
of players),
(with possibly one player sitting
out, in case of an odd number
all the winners
are again divided into
and so on, until
the finals.
The number of
pairs,
rounds
is riog2Az].
We can obtain
a parallel
algorithm from the tournament
by assigning
a processor to every game (think
of the processor
as the referee
of the game). We have
to ensure, however,
that each processor
knows the two competing
This can be
numbers.
the winner of the game
in the larger indexed position of the two
arranged
by putting
That is, if the game is played between xf and Xj such that j > /, then the maximal
players.
of Xj and Xj is put in position
jc 2, -1 to x2j
round,
j. In the first
processor
Pf compares
the
with
larger
running a
(1 <i <nll\\
^i^n/4),
EREW
model
Let's try
algorithm
of the
of the
maximum
the
second
involved in
The running
time
of this simple
the number of processors.
we
presented
just
the
Since
T(n\\nl2\\)=\\\\og2n\\.
efficiency
the
them if necessary;
in
Since each number is
so on.
is sufficient.
maximum-finding
the winner
exchanges
and
to minimize
now
The
and
The
winning.
with
tournament
*4/ 0
two
parallel algorithm
requires
\\nll\\
is clearly
algorithm
and
processors,
to
jt4/_2
time, an
O (log n).
we
have
sequential
algorithm
(n, nil) = 1/log2fl. If
is E
round, P{ compares
one game at a
only
nil
the
are available
processors
that
is a part of another
algorithm
algorithm
modifications,
With
some
is
and
efficient.
algorithm
simple
however, we can achieve
a parallel
time of O (logn) with O (1) efficiency.
as
The total number
is n -1, the same
of comparisons
required for this algorithm
the sequential algorithm. The reason for the low
are
is
that
processors
many
efficiency
anyway
(e.g.,
requires
them),
idle
and
nl\\og2n
in
if the
then
rounds.
later
performing
load
processors.
per group,
and
assign
maximum-finding
this
We can
improve the
balancing
We divide the
in
a group
the
input
to each
by reducing
efficiency
following
into
way.
Suppose
the number of
that
we
about
A?/log2A? groups, with
the first phase, each
processor. In
processors
only about
log2A? elements
use
processor
finds
12.3
for Shared-Memory
Algorithms
381
Machines
the sequential
algorithm that takes about
group, using
log2fl
steps. It
the maximum among about
but
there
are
now
maximums,
nl\\og2n
to
the
tournament
The
time
of
use
this
enough
processors
algorithm.
running
algorithm
that n is a power of 2) is r(AZ,[Az/log2Az]) = 21og2AZ. The corresponding
(assuming
efficiencyis E (n)= lfr. Next, we formalize the idea we just used for saving processors.
We
call
a parallel algorithm static if the assignment
of processors
to actions is
and for each processor Pp
predefined. We know apriori, for each step / of the algorithm
the
the
in its
maximum
remains
find
to
now
is a static
example,
Pj uses at
the operands
and
operation
algorithm,
static
If there exists an EREW
total number of steps (over
all
static algorithm
with
Let a{ (i =
of the
processors
in step /
processors
to perform
with
stePs
|~0,7(5/01
p processorsin
of steps is now
the
1,2,...,/)
step /,
do not
we
and
in which
total
the
be
available
the
^+1
problem,
at=s.
\302\243
sit processors
the
of steps
number
If #,<$//,
have to change
(following
algorithm
of the
complexity
then
of 0(1).)
We have
algorithm.
original
folding
it.
performed by
all
Otherwise,
emulate
the
principle).
steps
taken
step
by the
= 2l
*+--i>-
/=1
(slt)\\
Hence,
O (t).
modified
Proof:
processors)
sit) =
T(n,
algorithm
if s
that,
(Notice
for
algorithm,
12.1
\342\226\241
Lemma
the
The maximum-finding
/.
step
all the
because
time of the
the running
modified
is still
algorithm
O (t).
\342\226\2
after
Brent [1973] (which contains a proof of
as Brent's lemma
the same
lemma
shows that, in some
cases,
spirit of a more complicatedcase). Brent's
the total
the efficiency of a parallel
depends
mainly on the ratio between
algorithm
and the running
time of the sequential
number of operationsperformed
by all processors
is known
12.1
Lemma
algorithm.
know
which
We need the restriction
to static algorithms because we must
to
emulate.
Lemma
12.1is
valid
for
nonstatic
as
well, provided
processors
algorithms
that
the emulation
can be done quickly.
An example
of a case where Lemma 12.1is not
valid is as follows. Supposethat
there
are n processors and
n elements.
After the first
decide
on
some
of
the
the
results
of
the
first
to withdraw.
(based
step,
processors
step)
after the second step, the third step, and so on. This algorithm
The same thing happens
is
similar
to the
processors
step, we
computation
tournament
withdraw.
need to
time
know
algorithm,
If we
try
which
to find that
out.
to
of
which
that, in this case, we do not know
except
emulate
the remaining processors, say, after
the first
them
are
still
active.
But,
it may
require some
Parallel Algorithms
382
Model
CRCW
It may seem
first
at
a parallel algorithm
are used. But
that
if
the two-step
algorithm
We can improve
cannot
this
is not
algorithm.
to
the
the
one
for
be
can
group
the
efficiency
the EREW
squared
in
each
round,
leading
to
an
algorithm
that
requires
slower
that the two-step algorithm
(O (log logn)
versus 0(\\/n) of the
better. It is 0(l/loglogAz),
This technique has been called divide
since we divide
and
two-step
crush,
algorithm.
the
into groups of size small
with lots of
them
\"crush\"
we
can
that
input
enough
is not limited to the CRCW
model.
processors. This technique
this algorithm
Although
versus 0(1)),
12.3.3
The
the
its
efficiency
The Parallel-Prefix
problem
parallel-prefix
design
operation
of numerous
\342\200\224
namely,
it
slightly
is much
Problem
12.3
for
Algorithms
can
product. For example, \342\200\242
represent
addition,
383
Machines
Shared-Memory
or
multiplication,
two
of
maximum
numbers.
for
denote
all
the prefixes in
we use is divide
compute
method
Induction
The caseof
is trivial.
element
one
half
values
by
associativity
Complexity
we have n
are
is divided
input
in
as
not
\342\200\224
we
a power
n is
simply
The
to solve.
easy
of 2.
problem
parallel-prefix
be
parallel,
for
is given
two
sets
in each
disjoint
in parallel
solved
of size az,
12.1.
then
read
x[Middle]
Although several processorsmust
distinct
so a CRCW model is not required.
locations,
(since the sequential
E(n
yn) = 0(\\/\\ogn)
algorithm
we
cannot improve the efficiency
Unfortunately,
recursive call of
the
of them
one-half
and
steps,
same
can be allocated
to
they can also be
clearly
runs
use x[Middle].
all
time,
they
T(n, n)
Overall,
in
If
model.
EREW
becausethey
at the
nil)
the
the
under
used
we use
that
(notice
in Fig.
and
PR(1,
computing
by
induction
by
first
the
in half,
input
of PR(\\,k)
half can be
values
can be obtained
known
into
values
the
by dividing
the
obtain
n/2
The combining step requires
but an CREW model is required
subproblem.
performed
The
The
we
Thus,
\\<k<nl2.
algorithm.
the
proceeds
algorithm
for n/2<m<n
terms
Both
of the
k%
PR(\\,m),
\342\200\242
PR(n/2+\\ym).
The
induction.
for all
+ k\\
PR(n/2+\\,n/2
directly. The
how to solve
We know
compute PR(\\,k)
is trivial
problem
prefix
parallel-prefix
problem is
As usual, we assume that
conquer.
is to
goal
elements.
solving each
each
and
\\<k<n.
that
\342\200\242
\342\200\242
\342\200\242
\342\200\242
Our
\342\200\242xj.
version of the
..., *\342\200\236,
compute
x \\, jc2,
numbers
k, such
jc/*jc/+i
The
order.
hypothesis:
for nil
and
* * *
\342\200\242
for all
\342\200\242**,
x \\ mx2
the products
We
Given a sequenceof
Problem
The
O(n)
all
write
= 0(\\ogn),
to
and
steps).
of this
the Efficiency of
Improving
The
trick
is to
use
the
set
of all
Jt,s
with
that
a?
is a
i even.
but to divide
in a different
the
input
are n processors. Let E denote
If we find
the parallel
in \302\243,
then
prefixes of all elements
for
(those with odd indices) is easy: If PR (1, 2/) is known
we
need
to compute one
for each odd prefix PR (1, 2/ +
induction
same
the
Parallel Prefix
hypothesis,
power of 2
and
that
there
1),
Parallel Algorithms
384
Input: x (an
{
(x, n) ;
range 1 to n).
Parallel_Preftx_l
Algorithm
a powerof
that n is
assume
we
x (the
Output:
in the
array
contains
element
/th
the
/th
prefix).
begin
PPJ(Ln);
end
Right)
begin
if
Right
= 1
Left
then
:= x[Left]
x[Right]
%x[Right]
else
:=
Middle
(Left +
Right
binary operation }
is an associative
{ \342\200\242
- l)/2 ;
in parallel
do
{ assignedto
+
1,Right); {assigned
PPJ(Middle
+ 1 to Right do in parallel
for i := Middle
Middle);
PP_l(Leftf
x[i]
P,
to Pnl2
to Pn/2+\\
to Pn }
:=x[Middle]mx[i]
end
more
Parallel
prefix
J.
phases.
result in
that
this
First,
2/)
(f7?(l,
we
algorithm
\342\200\242
compute
requires
Jt2/+1).
We
can
find
the
prefixes
(in parallel)
only
the
EREW
model.
The
algorithm
is given
in Fig 12.2
Complexity
in parallel
Parallel_Prefix_2 can be performed
the
half
of
to
a
is
call
The
recursive
processors.
problem
applied
size, so the running
of steps S(n)
time
of the algorithm is O(logAz). The total
number
satisfies the recurrence
that
relation
implies
S(n) = S(n/2) + n-1, 5(2)=1, which
this
that
to
S(n) = 0(n). But,
we
now
use
Brent's
lemma
can
implies
improve the
we
can
Brent's
with
in
time
the
run
to
lemma,
efficiency.
By
0(logA?)
modify
algorithm
in O (1)
time
with
Both
loops
in algorithm
nil
to an 0(\\)
only 0(n/\\ogn) processors,leading
efficiency. The key to this improvement
is using
call
one
recursive
of
(instead
while
still being able to perform the
only
two)
merge
step in parallel.
12.3
Input: x (an
that n is
assume
we
x (the
Output:
in the
array
a powerof
2 )
contains
the
element
/th
385
Machines
Shared-Memory
(x, n) ;
range 1 to n).
Parallel_Prefix_2
Algorithm
for
Algorithms
/th
prefix).
begin
PPJ(I);
end
= nil
if Inc
then
:= x[n/2]mx[n]
x[n]
else
for i := I to
x[2
is an
{\342\200\242
\342\200\242
n/(2
do in
Inc)
\342\200\242
\342\200\242
/ Inc ]
associative
parallel
\342\200\242
\342\200\242
:= x[2
binary operation}
Inc-Inc
\342\226\240
\342\226\240
\342\200\242
/ Inc ] ;
jc[2
PP_2(2Inc)
do in parallel
for i := J to n/(2 Inc)-\\
\342\226\240
\342\226\240
\342\200\242
\342\200\242
\342\200\242
/ Inc -\\-Inc ] := x [2 i Inc ] \342\200\242*[2
/ Inc -\\-Inc
x[2 \342\200\242
end
12.3.4
environment
than
only access to
one
element
more
is much
it
Generally,
with
arrays.
link
list is
the
at a
elements of the
time
with
J'refixj.
in Linked Lists
Ranks
Finding
Parallel
to
difficult
through
no
with
deal
lists, for
Linked
the
possibility
head
linked representations
of the
list,
we
then
of parallelism.
In
sequential.
have
many
in
to traverse
cases,
parallel
If the
the
however,
list
the
in a contiguous array;
the
to them) are actually
stored
In such cases,
of the array.
order
imposed on the elements
by the list is independent
where parallelaccessto the list is possible, there is hope for fast parallelalgorithms.
of the
The
rank
of an element in a linked
list is defined here as the distance
element has rank
element from the end of the list (thus,
the head has rank az, the second
n- 1, and soon).
list
(or pointers
The Problem
storedin an array
We can solve
the
Given
A [\\..n ],
sequential
linked
list of
problem
by simply
all of
elements,
traversing
its rank
the
list.
which are
in the list.
The method we
will
Parallel
386
Algorithms
for
use for designing a parallel algorithm
element.
to
each
each
Initially,
processor
element in the list. In the first step, each
the first step, each processor knows
After
problem is
knows
processor
this
called doubling.
We
assign
of its
only
right
neighbor
finds the neighbor
of its neighbor.
at distance 2 from
its element.
the element
If,
k
from
its
the element
at distance
then in one
element,
at step /, each processorknows
until
the
step each processorcan find the element at distance 2k. This processcontinues
to the right
of / in the list that
element
Let N [i ] be the farthest
end of the list is reached.
is /'s right neighbor (except for the last
is known to Pt at a given moment. Initially,
N[i]
whose right neighbor is nil). Basically,in each step, Pt updates N[i] to N[N[i]]
element
is reached.
Let R[i] be the rank of /. Initially, R [i] is set to 0,
the end of the list
until
the
last
element
in the
for
list, for which it is set to 1 (this element is detectedby
except
with a nonzero rank
When a processorencounters
a neighbor
its nil pointer).
/?, this
of rank 1 knows its
Initially, only the element
processor can determine its own rank.
of rank 2 finds
After the first step, the element
that its right neighbor has rank
rank.
1, so
the second
rank
and
rank
4
are
3
it knows that its own rank is 2. After
determined,
step,
If f\\ finds that N[i] points
to a \"ranked\"
so on.
element of rank R after d doubling
and
is given
in Fig 12.3. This
steps, then /'s rank is 2d~l +/?. The precisealgorithm
can be easily adapted to the EREW
model (Exercise
12.4).
algorithm
processor
(N);
List_Rank
Algorithm
the
N (an array
N (the rank
Output:
Input:
range 1 to
element
in the
of each
of
in
indices).
the
array).
begin
D:=l;
{
each
do in
can have
processor
we use only
local
}
P( is activeuntil
{ processor
parallel
own
its
D variable;
D variable
one
R [i
] becomes
nonzero }
R[i]:=0;
ifN[i] =
while
R[i]
then
nil
= l
R[i]
=0do
ifR[N[i]]*0then
R[i]:=D+R[N[i]]
else
N[i]:=N[N[i]];
D := 2D
end
12.3 AlgorithmListRank.
Figure
Complexity
The doubling
of
most
the
E(n,
list
in at
n) = 0(\\llogn).
[~log2\"l
Improving
process
guarantees
steps.
Therefore,
the
efficiency
that each
processor
will
reach
the end
efficiency
T(n, n) = 0 (logn). The
requires making a major modification
is
to
12.3
the
the total
since
algorithm,
of
amount
Machines 387
for Shared-Memory
Algorithms
is 0(n
work
discussion on recentresults.)
rank
allows us to convert a
computation
Bibliography
for a
section
The
time
an array in O(logAz)
ranks are computed,
into
list
linked
than
the
(even though
perfect efficiency).
locations in the array, and the rest
can be copied into the appropriate
on the array, which is much easier.
can then be performed directly
computation
less
with
elements
the
We
can
parallelize
many types
the
tree
in parallel
(e.g.,
time of
an
such
as
need
az-1,
in
instrumental
another
the
the
the
find
is not
tree
number
balanced,
Euler's
The
approach.
the tree.
is the
a whole level of
maximum).
of
height
(where
algorithms on trees,
parallel
designing
to
algorithm
However,when
is quite good.
and we
approach
high
by operating on
trees
on
algorithms
is proportional to
is O(logAz)
height
its
and
of
tournament
the
algorithm
balanced,
reasonably
this
of the
12.3.5
as
all
After
and
The running
If the tree is
of nodes), then
may
be
tour technique
is
the
height
unbalanced
on
especially
trees.
Let T be a tree.
representation,
of the list of
/ is
since
edges incident
to
that include
records
contains
additional
known),
and
the
a pointer
T is
that
assume
We
one
with
represented
the
by
adjacency-list
regular
As
is a
there
usual,
copies, one for (/, j) and one for (/, /). The
by two (directed)
extra pointer for each edge that points to the other copy of that
when
need that extra pointer to find the edge (/, /) quickly
edge (/, j) is given.
feature
additional
We
edge.
is an
This representation
is illustrated
in Fig.
12.4 (the
pointers
connecting
the copies
are
not
shown).
The
tree
that
key idea
forms
behind
an Euler's
the
Euler's
tour of
list
the
is built,
the
of
which
can
edges
of the
edge
be performed
every
E
(1,2)\342\200\224>
I\342\200\224\342\226\272
(1,5)
\342\226\272
(2,3)
\342\226\272
(3,2)
\342\226\272
(4,2)
\342\226\272
(5,1)
Figure
tree
list
tree.
\342\226\272
(2,1)
\342\226\272
(2,4)
Parallel
388
always traverse
parallel.
construct
will see
two
perform
us to
examples
list
was linear.
together
operations
of
such operations.
But
algorithm
sequential
with
first
the
the
we
traversal.
can
This
tree
efficiently in
discuss how to
tour.
Euler's
the
allows
tree
the
the
though
and
tree
the
\"linearization\" of
We
as
list almost
the
on
directly
Algorithms
tour of T (with
an
Euler's
It is easy to find
edge
appearing
twice)
every
the opposite
the tree using depth-first search, taking
order
sequentially. We can traverse
in parallel.
the search backtracks. We will do a similar thing
Let
of an edge whenever
in
the edge following
the
tour.
We
claim
that
NextTour
NextTour (/, j) denote
(/,
j)
edge
which can be easily computed in parallel:
is defined by the following
rule,
i)
if Next
(j\\
i) is
not
nil
otherwise.
\302\243(/)
{Next(j\\
words,
(/, 0 is
/ is
/s
list
the
of edges
If/(/, j) is the
number for j is n
the tree).
of forward
edges following
of r,
(The preorder number
which is the only
to it, is 1.) We can now use a
forward
edge
pointing
variation
of the doubling
(/', y), the value of
edge
algorithm to compute, for each forward
We leave the exact implementation of this
as an exercise.
/ 0\\ 7 )\342\200\242
algorithm
doubling
The secondexample
involves
for each vertex y\", the number of vertices
computing,
below j (descendants)
in the tree.
Let (/, j) be the (unique)
forward
edge leading to j.
Considerthe edges following the edge (/', j) in the list. The number of verticesbelow j in
of forward edges that are below j in the tree. We already
the tree is equal to the number
(namely,
(/, j)
in the
list,
parent
in
the preorder
vertex
with no
then
number
-f
(/', j).
12.4
know
how
edges
that
to compute
are after (/,
edges
/(/,
/) following
of j is equal
performed
in
We can
list.
the
list.
y),
compute
similarly
It is
is
which
the
of forward
of forward
of descendants
the
number
the
number
number
be modeled
can
to
the
nodes and
two
by
graphs
nodes
389
The
be
Networks
for Interconnection
networks
processorscorrespond
(almost
are connected
the corresponding
between
access, through
can be
the
shared,
and
processor
the
(/, /)
Algorithms
Interconnection
(if
j)
the
value of/(/,
the
parallel
in
Networks
Interconnection
algorithms
12.4
in
for
Algorithms
variable
Processors
variable
that
is located
is a
graph
message
Numerous
Parallel
390
Algorithms
12.5
Figure
The hypercube
any
two
a rich
provides
(e.g., we
processors
combine the
with
hypercube
A four-dimensional
hypercube.
are many
can changethe
bits
appropriate
by,
for
of the
faces
12.4.1
We
Sorting
with
start
on an
We
order).
embedding
example,
include
hypercube. Other suggestednetworks
connectedcycles,quad and octal trees, mesh of trees, butterfly,
the
any
between
routes
different
in
can also
meshes in
shuffle, cube-
the perfect
more.
and
Array
on
of processors.
an array
There
one
smallest
the
such
that
the
goal
input among
processors
we may want to assign
input is in P x, the second smallest is in P2, and so on. In general,
more than one input to each processor. We will see that the same algorithm
be
can
to
Each
this
case
as well. The processors are connected
in a linear fashion.
adapted
processorP{ is connected to Pi+l, 1 < / < n.
are
input.
processors
is to
The
Since each
comparisons
the
array.
and
distribute
processor
possible
Each
...,*\342\200\236.
processor
holds
the
can
exchanges
communicate
can be
only
done
with
must
algorithm
its
with
neighbors,
that
elements
allow
the
only
are consecutive
in
- 1 steps, which
is
The
array
of
is
number
to
the
as
its
number
follows.
Each
algorithm
basically
processorcompares
one of its neighbors, exchanges the numbers if they are in the wrong order, and then does
the
same
with the second neighbor. (We must
because otherwise the
alternate
neighbors,
same
numbers
all numbers
will be compared again.) The same process
continues until
are in the correct order. The steps are divided
into odd steps and even steps. In the odd
with those of their
numbers
their
right
steps, the odd-numbered processors compare
in the even steps, the even-numbered
neighbors;
compare their numbers with
processors
those
of their right neighbors (see Fig. 12.6). This
are synchronized
all processors
way,
and
a comparison
If a processor does not have
always involved the correct
processors.
the
time
one
end
of the
to
the
other.
12.4
X2
Xt,
X$
X$
X(y
X-j
;*8
corresponding
this step.
through
in Fig.
12.7.
sort
be
very
algorithm
Odd-even
12.6
neighbor (e.g.,
the
This algorithm
is called
numeric
first
of the
example
sort.
transposition
in the
processor
the odd-even
is
algorithm
in the
the
Networks 391
X i
Figure
the
for Interconnection
Algorithms
steps.
in
can
the
seems
of
natural
and clear, but
its
Algorithm
proof
Sorting_on_an_Array
correctness is far from obvious.
For one thing,
an element
from
its final
may move away
destination. For example,in Fig. 12.8, 5 moves to the left for two steps before it starts
to the right,
and 3 moves to the
leftmost
and stays there for three steps
moving
position
before it moves back to the right.
is
the
correctness
of parallel algorithms
Proving
of
because
the
the
of
different
actions
the
difficult,
interdependencies
among
processors.
Algorithm
Input:
Output:
(x, n)
Sorting_on_an_Array
x (an array
x (the
the
in
range
az,
that xt
such
order, such
in sorted
array
1 to
the
that
resides
at
Pt).
/th smallest
element is
begin
do in parallel
\\
n/2\\
times
/>2/-i and
elements
and exchange them
such that \\<2i<n}
their elements and exchange them
P2i comparetheir
if necessary ; {for all i,
P2j and
P2t+\\
if necessary
compare
; (for
step
is done
< 2/
only
<n}
|_a?/2J
end
Sorting_on_an_Array.
time }
in
Pt).
Parallel
392
Algorithms
L
5
J
7
J
6
L
2
j
8
J
5
L
2
J
7
L
4
/
i
l_
j
5
J
5
L
6
/
i
12
Figure
The behavior
on
An
example
of odd-even
transposition sort.
one
only
processors
12.8
L_
one
hard to
focus
to consider all
together.
12.2
\342\226\241
Theorem
When
Proof: The
terminates,
Sorting_on_an_Array
algorithm
the numbers
are sorted.
of processors
induction on the number
(or elements). If
then
one
sorts
the
two
We assume
numbers.
processors,
comparison
that
the theorem
Let's
is true for n processors
and consider the case of n + 1 processors.
focus our attention
on the maximum
element and assume that it is xm (e.g., x5 in Fig.
In the first step, xm will be compared to either xm_\\ or to ;cm+1, depending on
12.8).
whether
m is even or odd. If m is even,
then no exchange will take place because xm is
than
this
is
the
same
at Pm-\\
But
as the case of xm residing initially
greater
xm_x.
exactly
an
can
loss
of
we
without
(and
Therefore,
assume,
exchange taking
generality,
place).
that m is odd. In this case, xm is compared to ;cm+1, exchanged,
and moved step by step
to the right
at xn+\\
it arrives
it is greater than
all other
(since
numbers) until
(diagonally)
and stays there.
This is its correct
so the sort works correctly for the maximum
position,
proof
is by
two
element.
too. There are
the elements are sorted
correctly
induction. To do that, we have to map the
execution
of the n processors in the array of size n +1 to a possible execution of n -1
is done as follows. Consider the diagonal
processors in an array of size n. This mapping
formed
the
movement
of the maximum
element
by
(see Fig. 12.9). The comparisons
the
maximum
element (i.e., those that are on the diagonal) are ignored. We
involving
divide
the other comparisons into
and the one
two
the one below the diagonal
groups,
above it. We then \"move\" the triangle
one step down. In other
the diagonal
above
in
the
for
the
For
called
words,
comparisons
upper
step /+1.
triangle, step / is now
1
consider
the
12.8.
versus
and
4
of
8
first
versus 2 in the
comparisons
example,
Fig.
step
other
elements,
and we
that
would
the
rest of
like
to use
12.4
Figure 12.9
first comparison is on
so we consider it
The
the
step involving
side of
5, 6
7 versus
versus 1, and
in the
so it
diagonal,
of step
(from
We can
n elements.
only
step
be part
to
diagonal,
consistsof
induction
The
now
proof of theorem
is ignored;
step
of
1) 4
versus 2.
ignore
simply
12.2.
step
is above
one
second
the
(instead
393
Networks
Interconnection
for
Algorithms
the
step 1 (i.e.,
1) on
row
the left
(the last
involving
from
of
that
can
result
a
column), and the rest
running
sequence comparisons
is
the sort on n elements
for n elements. By the induction
the
hypothesis,
algorithm
\342\226\
correct; therefore, this sort is correct too, and it requires
only one more step.
the
and
diagonal
all comparisons
the
element
maximum
is exactly
So
each
we have
far,
assume
the
that
at P
reside
the
of one input
case
and
consider
redistribute
the
first the
elements
reside at
better
so
we
cannot
do
moved,
the largest
be
the sort is to
to achieve
inputs
is to
goal
x, and
must
elements
discussed
holds k
processor
k elements
repeat the
following
step
P2>
than
clear
worst case
in the
that
2k element
movements.
One
completed: P x
aH
way
its
sends
terminates
largest element to P2 and P2 sends its smallest element to Px. The process
when the largest element in P x is not greater than the smallest elementin P2. This step
is called merge-split. If we use this step as the basic step in the odd-even transposition
we can
sort,
possibly
an
extend
and
of a comparison
elements per processor. Instead
is
done.
a
elements, merge-splitoperation
for an array, its
in this section is optimal
presented
algorithm
total
for n steps; therefore, the
each running
n processors
to many
of neighboring
exchange
Although
sort
the
the
efficiency is low.
sorting
We
have
of steps is
number
algorithm
12.4.2
When
number
as
Sorting
we
design
of steps.
independent
as
sorting
support
that
are
Networks
we are concerned
an efficient sequential algorithm,
of a parallel algorithm,
we must also
In a design
Consider mergesort (Section 6.4.3). The
possible.
only
try
to
two
with
the
total
the steps
calls
recursive
make
Parallel
394
Algorithms
completely
odd
independent,
the
of
a^a3y
subsequences
order of
the
x2n=en.
The
even
rest of
the
theorem
and
...,fl\342\200\236_i
bub3
merge
let ex,e2,
and
...ybn_Xy
subsequences
b2,bA...,bn.
as
can
be the
...,\302\243\342\200\236
be seen
the
by
and
following
12.10).
12.3
\342\226\241
Theorem
above,
for
all i such
=max(oi+i,
that
we
\\<i<n-\\,
order of
both
-+ *2
a2
nil
aA
merge
-\342\200\242
-*4
network
<*8
-* *10
b2
b3
b4
b5
nil
-\342\200\242
*ll
merge
-\342\200\242
x\\2
network
-\342\200\242
*13
be
-\342\200\242
*14
bi
-\342\200\242
*15
bs
-\342\200\242
*16
Figure
12.10
have
et).
Proof:
sequences,
merged
Clearly, x{=o{
merge.
the
even
sequences.
12.4
even element,
Algorithms
for
Interconnection
Networks
395
(since
we started
with
two
makes
we
can
add one
sorted
only
e2 ando3
fitjt4
andjt5,and
soon.
\342\226
important
one
The recurrence
Complexity
x\\6
Figure
12.11
mergesort.
Ts(2n)
= 2Ts(n)
circuit
contains
+ 0(n\\ogn)y
n processors
there are
of
computation
12.4.3
We
There are
a sequence
x \\,
inputs
(e.g.,
(see,for
2h
pixels
example,
- 1
processors,
such
that
*\342\200\236,
Jt2,...,
suggestedmainly
and its
depth
The
Ts(n) = 0(n\\og2n).
is 0(log2Az),
so overall
for
network
applications,
in a
picture) and
the
[Uhr
1987]).
In this
example
a parallel
the
and
algorithm,
is a
each associated
xt resides initially
image-processing
on a Tree
Element
Ath-Smallest
the
assume
now
that
implies
in the
processors
Finding
leaves.
which
\"column\"
each
0(n log2a?)
can be used for all
comparatorswith
75(2)=1,
in
algorithms
complete
with
at
a node
leaf
where
/. Tree
the
leaves
pipelining
(which
is described
to the
correspond
use of
is
in
are hierarchical
of finding
the
algorithm
to
more
detail
in
Section 12.5).
for
the &th-smallest element
finding
sequential algorithm
describedin Section 6.5. We assume, for simplicity, that the elements are distinct. The
one. In each step, a random element x is chosenas the pivot.
is a probabilistic
algorithm
The rank of x is computed by comparing
x to all other elements and, according to
whether
the rank is smaller or greater
than
that are less than x or greater
ky the elements
than x are eliminated. The algorithm
when the rank of the pivot
is k. The
terminates
number
of
iterations
is
and
the
number
of
is
expected
O(logAz),
expected
comparisons
in each iteration
of the algorithm:
0(n). There are three different
(1) choosing a
phases
random
its rank, and (3) eliminating.
We first describe
efficient
element,
(2) computing
even further.
the parallelization
parallel implementations of each phase,then improve
a random element can be achieved
Choosing
by a tournament
arranged on the tree.
*
Each leaf sends its number
to its parent where the number
with the number
'competes\"
of its sibling leaf by flipping a coin. The winning
number
is then promoted up again,
and
the same process continues up the tree until the root chooses the overall
winner.
(This
works only
in the first iteration; we discuss later how
it work after
some
to make
elements
are eliminated.) The winning
so
number
the tree,
is then \"broadcast\" down
that all leaves can compareit with their number.
at
of the pivot is known
If the identity
all the leaves, they
can
all compare
their numbers to the pivot in one step. They then
send
a 1 (if their number is smaller than
to their
or equal
to the pivot) or 0 (otherwise)
of the pivot is the
number
of Is that are sent up. Summing n
parent node. The rank
numbers
can then broadcast the rank down the tree,
root
up the tree is easy to do. The
and
each
leaf can determine whether
be eliminated.
or not its number should
Overall,
there are four \"waves\"
tree to choose a pivot;
of communication
the
(1) up
per iteration:
the tree to broadcast it; (3) up the tree to compute its rank; and (4) down the
(2) down
tree to broadcast
the rank.
First,
we
recall
the
12.4
The problem is
for
Algorithms
397
Networks
Interconnection
In
Complexity
times
way
the
as
expected
A Sketch
number
The
the height
that,
extreme
the
of
the
sequential
tree.
some
of (parallel)
Since
algorithm,
of an
after
elements
one-half
in
steps involved
the
expected
in
each
phase
eliminates elementsin
this algorithm
number
of phases is
still
is equal to
exactly
four
the same
O(logAz).
The
O (log2Az).
Improved Algorithm
of the
root
of the
computation.
Parallel
398
Algorithms
12.1
D Example
An
of this
example
number.
smallest
so we
(In
shown.
The
numbers
outside
numbers
first
the
numbers.)
inside
the
nodes
one
second
is 5
(which
proceeds from
8,
we
and
is equal
number
For each
are
(Fig. 12.12e),and
from
obtained
4, 3,
then
1, and
fourth
element
is broadcast
case,
one
pivot.
of
nodes
the
this
we do
(which
need to run
by
not
process
&th-smallest
The expected
overall.
O(loglogAz)
second phase,
12.12(j),
Fig.
any
in
should
be
\342\226\
to 0{
is reduced
time
running
graph
ProcessorP [/, j ] is
P[U
J;
+ 1],
P[i +
to P[0,
addition
the
of indices
and
with
elegant
without
the
The
and
Given
Problem
0] =
right
A
\302\243 L0,
two
j] resideat
matrix
regular
n-\\
C[0,
has
).
and some
results,
[1987].
the
column
same
two-dimensional
y,
1]. We
it
In
other
we
all additions
words,
is more
present
within
and
symmetric
a constant)
around.
wrap
the
that
and
is a
and
j] and B[i,
resides atP[i,j].
such
at row /
j-
this assumption;
A[i,
We use the
have
now
assume that
for example, P[0, 0] is connected
P[i,
simulation
log
the Mesh
P[\\,
0] (see Fig. 12.13).
are done modulo n. The algorithm
1]
subtractions
we consider
algorithm,
and Manber
in Greenberg
processor
l,y],
so that,
around,
wrapped
in
of the
details
the
Matrix Multiplication on
interconnection
The
In
element);
elements
log
12.4.4
is 3
12.12(g),
terminates.
the algorithm
show),
another
pivot
at Fig.
root
the
the
Complexity
factor
to
there was no
general, however,
eliminated until the
is discovered
element
smallest
value,
and
In Fig.
4.
fourth-
nodes is
below,
chosen
first
top
right
of all
contents
the
to
left
down. The
are broadcast
that
step,
ones
the
are
on they
and from then
start the
(3) arrives at the leaves,
the rank. The fact that 3 has rank 3 is discovered in
pivot
and the
with
of each
and
are those
1 to
is computing
which
this
from
the rank
case,
12.12
Fig.
ranks
the nodes
(Fig. 12.12d),the
in
numbers
the
this
show both
not
do
process is given
elementsare
down). The
numbers
nxn
P[i,
matrices
y], compute
A and
B, such
C=A-B,
such
that
that
multiplication
algorithm. The problem is
are at the right
at the right
place
k ] \342\200\242
B [*, 0], which is
the
inner
product
of the
first
row
initially
C[/,
j]
and
the
first
\342\226\240>o
Ui
*\302\251\342\200\2240\342\200\224G-
(EX-XE)
o
-U
-U
GX0O
G>0XE)
-U
4t
-uo-u
-uu^-u
U,
L*
L*
(00HE)
GHM)
G-XE
*k.
(EXEXE)
(EXEXE)
l*
<-*
-U
<-*
-uo-u
-UO-U
GXBXD
GXEXE)
-0\342\200\2240\342\200\224\342\202\254)-
(EX-XE)
GW3
*k.
L*
0M^)
&\342\202\254XE)
L*
(EXEXE)
-0\342\200\224GM\302\251\"-
EX\302\260XE)
U,
\342\226\240U
(EX*XE)
V*j
<-\302\273\302\273
l*
GXEXE)
GXXEX^E
(EX\302\260X3
\342\226\240uu^-u
GKEKE)
Uj
Uj
<-\302\273\302\273
GXXEXXE)
GKEKE)
Ui
Uj
GHHMH)
GK5KH)
<-*
GH00)
GX00)
(EX\302\260XE)
Ui
-0\342\200\2240\342\200\2240*
\302\253-.
\342\226\240U
00H3
Ui
GXEH3
00HE)
00X3
<-*
&\342\202\254X3
-U
O
-0\342\200\2240\342\
(EX3-Q
-U
Parallel
400
Algorithms
Ck
0-
Q^
Ck
-0-
6-
-O-
6^-
-<y-
Figure 12.13 A
column of
the
final
B.
We
and, at the
time,
would
This can
result).
like
-6'
-o^
two-dimensional
to be computed
C[0,0]
be done by
by
first column
of B
of A
up
left one
the
to
one
step
have
step at a
a time. At the
also must
(which
P[0,0]
first row
the
shifting
the
mesh.
wrap-around
at
step, P[0,0]
step, it gets A[0y
and
partial result,
The initial
Complexity
the
when
number
shifts,
for
row
of shifts is
each
so
\302\243[0,0],
shifts
more
far.
processor.
take nil
than
nll)\\
Then,
All
it computes
and
these
the
there are
steps
steps,
can be
true
for
involving
done
in
shift
the column
to the
right
shifts. No
both computations
The overall
parallel.
to
running time is thus O (n). The efficiency is O (1) if we compare the parallel algorithm
we
if
the regular 0(n3)
less
sequential algorithm. The efficiency is (asymptotically)
to asymptotically faster algorithms (e.g., Strassen's
compare the parallel
algorithm
algorithm).
12.4
Matrix
Algorithm
begin
for all rows
matrices).
of A and B).
(nxn
product
in parallel
do
shift row i of A
{ namely,
401
(A, B)
.Multiplication
Input: A and B
Output: C (the
Networks
for Interconnection
Algorithms
A [/,
perform
left i steps
to the
n ]
i times
in parallel
do
j steps
shift column j ofB upwards
{ namely, perform B [/, j]:=B [(/+
{thedata is now in the proper starting
for all i and j do in parallel
j times }
ny j]
\\)mod
positions
C[iJ]:=A[i,j]B[iJ];
for k := 1 to n-\\
for all i and j
do
do in
parallel
A[i,j]:=A[i,
(j+\\)modn]
B[iJ]:=B[(i
+ \\)modnJ] ;
C[iJ]:=C[iJ]+A[iJ]B[iJ]
end
12.14
Figure
012
013
0 14
b\\i
^13
b\\4
a2\\
a 22
0 23
b2\\
b22
^23
031
0 32
&31
0 13
0 14
011
b32
^43
bXA
24
021
022
b24
&31
bA2
^13
^24
0 33
0 34
0 34
031
0 32
033
b32
&33
bM
b4[
b\\2
^23
bM
<*41
042
043
044
041
042
043
044
bAX
b42
^43
^44
bn
^22
&33
bu
Figure 12.15The
The
we
examples
heavily
approach
powerful
an
have
algorithm
24
data
initial
in a
Routing
algorithm to
so
MatrixMultiplication.
012
^21
0 23
011
12.4.5
Algorithm
placement
and
the
result
of the
Hypercube
seen so
that
far
illustrate
runs on
an
the
interconnection
difficulty
of
the
architecture,
even
adapting
network.
shifts.
initial
If algorithms
difficult.
is
to
then
a simple
depend
Another
define some
to design the
Parallel
402
algorithms
these
using
translate
to
network
Algorithms
We have to
primitives.
all algorithms
implement only
these
use
that
the
on another
primitives
The problem
primitives.
this
with
a general
routing
networks as though
discuss
briefly
interconnection
We assumea hypercube
In an EREW
topologies).
variable
the
scheme
shared
that
memory
connection
(similar
xh
algorithm
to design
for
algorithms
is available.
schemes
shared-memory algorithm,
different
the
read
any
or write
conflicts, a step in
way to look at
Another
variables.
the
is as
step
buffer
us
allows
access
step
the
of
definition
permutation
requirements.
space
The best
generally
not
routing
possible
permutation is distributed
that works well on the
[1982].
The key
first phase,
(independently
describe
the
to analyze
among
average.
on the
depends
obviously
following
the best
Therefore, we are
the processors.
The
find
The
routing
consists
for
a scheme
suggestedby
of
two
the
since
routing,
looking
it is
However,
permutation.
particular
to
permutation
phases.
Valiant
In the
how
to select
this
random
processor
and how to
route
the
message
to it.)
In
will
the
phase,
12.4
403
Networks
for Interconnection
Algorithms
000
Figure 12.16An
decide
other words, we
the third bit. In
lines)
through
passes
next choice
much faster
without
than
it.
or
whether
not to
If we
decide
for
waiting
the
in
a 3-cube.
to 010.
message
the next
we
round;
When the
passing.
message
send
routing
the random
12.16,
Fig.
of random
example
choice
consider
we
Finally,
route
(marked
in
is
the
is done.
routing
more
are
edge
available.
becomes
is not
It
of
probability
chosen
the
bold
make
immediately
difficult
being
processor
to see that
as
chosen
number
every
the
random
k (in
the
destination.
r to j is done deterministically.
'''
The
Suppose
range)
appropriate
has the
the
in
same
random
t bits
with
and
the indices k{ <k2<
<kt.
messageis sent by changing bit A: i, then bit /:2\302\273
will be sent first from Oil to 111 (since
so on. In the example of Fig. 12.16,
the
message
the
destination
110 has a 1 in the first bit), and then
to
110 (the second bit is not
changed).
This routing
shortest routes (in
scheme
the
example
is simple
above,
to implement.
the route took
The routes
four
edges
are
versus
not
the
necessarily
two edges of the
networks
route).
automatically.
however,
is not
Parallel Algorithms
404
12.5
Computation
Systolic
called
an assembly line. The processors,usually
resembles
but
not
as
in
one
a
are
(usually,
always,
very regular way
arranged
processing
in
a
them
fashion.
Each
move
and
the
data
dimensional
rhythmic
or two
through
arrays),
beat,
simple
operations on the data it received in the previous
very
processorperforms
contains very limited (if
Each
to the next \"station.\"
and moves the result(s)
processor
one variable at a time, rather than being
is \"pushed\"
Most of the input
memory.
any)
of systolic
architecture
is
loaded all at once to some memory locations. The advantage
and
is
and
in
terms
of
the
hardware
both
(which
simple),
speed
specialized
efficiency,
As in assembly lines, the key is to
(the number of memory access cyclesis minimized).
that
is
avoid the need to fetch tools, material, and so on, during the work. Everything
the
The
drawback
is
the
arrives
on
line.
big
inflexibility
required to perform an operation
are efficient
of such schemes. Systolicarchitectures
only for certain algorithms. We will
A
architecture
systolic
elements,
examples of systolicalgorithms.
see three
Matrix-Vector
12.5.1
We
with
start
Multiplication
a straightforward
example,
use
then
develop more
it to
complicated
algorithms.
Find the
Problem
column vector b of sizen.
The
xn
of an m
x-Ab
product
A with
matrix
to the partial
(stations), such that P,- is responsible for adding
br The data movements and the actions of each processor for
the
case
of n-m-A
b resides
in the
are illustrated
in Fig.
12.17. We assume that
The results are accumulated
into
them regularly).
appropriate processors (or is pumped
as they move from left to right through the processors. The x{s are initially
0. In the first
P \\
with
move
closer.
P
move
into
and
all
the
other
step, x! (= 0) together
a,,
\\,
inputs
\342\200\242
receives
and
moves
the
to
its
second
result
In
the
computes
jc, +0,,
fc,,
step,P2
right.
There
n processors
are
product
the
term
x\\=a\\\\'b\\from
moves
the
result
involving
the
left,
to its
together
right,
and
with
so
an
on.
from above;
In each
/-1
result,
which is equal to
above,
or
it
from
clearly
below.
has the
\302\243
ajkbk,
element of b from
P{ computes
right value.
the left,
from
x+ajjb,
The whole
and
passes
product
the
it
X\\+a\\2'bi,
computes
either
it to
local
its
memory
right.
is computed
of the
entry
appropriate
in
(as
it
matrix
is in
When
jt, leaves
m +n
steps.
and
partial
from
the figure)
the array,
Each
is the data movement.
designing
systolic
algorithms
data element must be at the right place at the right time. The only trick in this example
was to introduce
so that column / of the matrix arrives at P, in step /. This
delays
element
of A was used only
is
since
each
When the same value is
once.
example simple,
in
12.5 SystolicComputation
405
'44
-X\"i
0 34
<*42
0 33
0 24
<*41
0 32
0 23
0 14
031
a 22
013
021
012
.
*2
043
\302\253
J b3
J b2
*i
\342\226\240
W
.
J b4
>
OUTPUT
-+ *out =
as is
movement,
12.5.2
Let
times
several
jc
usually
in the
The Convolution
=jc i,
jc2,
vector
and
\342\200\224,xn
Problem
The
The
(as is
illustrated
is
called
w =
+ ab
Figure
used
xin
the
it is much
case),
more complicatedto
design
its
next example.
Problem
w,, w>2....,wk
Compute y\\,yi,
the convolution
two
be
sequences
sucn
...,yn+\\-k*
of x
and
w.
We
of real numbers
tnat
J/ =
can reduce
with
w\\xt
the
k <n.
convolution
406
Parallel
Algorithms
-V,
X2
*v3
X2
A'3
*3
' '
X4
' '
x5
x4
*k
Xk
y\\
+ \\
xk+2
w,
yi
w2
^3
w3
(12.1)
Wk_
xn + \\-k
xn+2-k
' '
xn+l-k
yn
+ \\-k
xn
the matrix
the systolic algorithm
for this problem
in
by simply substituting
in Fig. 12.18.
Notice that each jt, is
(12.1) with the matrix of Fig. 12.17. This is shown
across
the array (except for the first k-\\
needed at the same time
jt,s which are not
we show
another solution
needed
Thus, a broadcast line is needed. Next,
everywhere).
with
no
broadcast.
to the convolution
problem
The processorsin Fig. 12.18 receive several inputs, but send only one output.
We
that
from
two
send
to
use
receive
directions
and
two
now
processors
output
inputs
directions. The idea is to move the vector x from left to right and the vector w from right
to left. The result
in the
We have to design this movement
y is accumulated
processors.
such that the appropriate values of w and x meet.
The problem with moving
the two
vectors in opposite
directions
is that they move twice as fast toward
each other.
As a
each
element
of
will
x
miss
half
of
the
elements
of
and
vice
versa.
The
solution
result,
w,
is to move
the vectors at half the speed. The left input will be **jc 1, nothing,
jt2,
in Fig. 12.19 (the
is illustrated
nothing,\" and so on, and the same for w. This solution
We
bullets
can obtain
correspond
to empty
slots).
x^
x^
x^
*3
*3
x3
X2
X2
x^
x\\
Wi
w->i.
'
1f
W-t
Wa
OUTPUT
Figure
12.18
Convolution
using
broadcast.
12.5
407
Computation
Systolic
w\\
\342\200\242
y*
^3
*2
1\342\200\224i\342\200\224' 1
I
I
.-*-_
\342\200\242
1
I
I
,-*-_ '
yi
y\\
X\\
'
I
I
I
I
,-*-- '
,_*__'
OUTPUT
y=y
Win
Wout
y
out
X'm
We leave
leaves Ph
data
12.5.3
the reader
to
value
of
is, of course,that
A =a\\
a2
an
the
and
The
of
right
the
in
Fig.
it
P,-
collects
can
move
array.
the value
out of
the
of
array
When
wk
through
the
yh
drawback to moving
as long.
The main
12.19.
takes twice
'''
bm
distance betweenA
proceeds
algorithm
and
computation
B\342\200\224b\\
b2
each
that
verify
a bidirectional
\"in
data
at
Comparisons
Sequence
'''
to
y,- is computed,
the array
below
illustrated
path
half speed
Let
it
final
the
~ *in
*out
Wout
Figure
win-xin
table.
was
be
and
presented
parallelize
two
strings
B (see
this algorithm,
then
present
the
solution
depends
a systolic
array
appears at the
on three entries
it,
the (/-
and
/(/-
408
Parallel
Algorithms
Figure 12.20Sequence
0
/
by a
comparisons
0
a
c
2
a
7
a
a
>
a
2
Figure12.21Sequence
needed to process these
2
a
a
3
3
3
comparisons
by a
2
1
3
a
one-dimensional
array.
This is achieved by
to the processor.
one
both
at
half
to
left,
right,
speed, and having
right
is
which
cell
used
use
half
for
We
the
connection.
memory
speed
per processor
diagonal
not only to ensure that the corresponding charactersmeet,
costs
but also to allow the
slots between two adjacent characters. For example,
so far to use the empty
let
computed
A =caa and B =aba (seealso Section
The first row (and column)
in the matrix is
10.2.3).
moving
A from
left to
are
characters
B
from
known
123 (corresponding
the right side will
of all
are
in the
all
from
that
and
to 0.
characters
two
When
the two
numbers
to compute
needed
information
the
3,
beat,
next
of inserting
costs
the
a \\b2a
are initialized
processors
compared;
has
to
be
409
Reading
the
a{ and
following
bf
entry
corresponding
meet
meet
them
cost
minimal
12.6
Summary
Since parallel
algorithms,
is a powerful
Pk-\\,
to B\\\\..j]
/\\[1../-1]
matching
A [1../] to
are
algorithms
we have to make
building
block.
even
more
good use of
It was
complicated to design
building
blocks.
than
sequential
The parallel-prefix
machine
paradigm
operation
(for
Bibliographic
Notes
and Further
Reading
of
as is evidenced by the number
rapidly,
computing is expanding
in the last few years. Books describing
books that have appeared
parallel
computing
and Parberry
include Akl [1985], which
is devoted
to parallel sorting;
[1986]
Hong
include
models of parallel computation
and their relationships;
theoretical
[1987), which
and Rytter [1988], which
are
books on parallel
Quinn [1987] and Gibbons
general
and
and
Malek
and Almasi and
[1984],
[1987],
Briggs
algorithms; Hwang
Lipovski
which
different
describe
Gottlieb
and
architectures
[1989],
parallel
parallel software;
with
which
deals
Ullman
and
VLSI;
Siewiorek,
[1984],
[ 1987],
mainly
Gehringer,
Segall
Cm
which describes experience with
describes
; Fox et al. [1987], which
experiments
which includes models, architectures,
with the hypercube; Reed and Fujimoto
[1987],
Bertsekas
and
which deals mainly
Tsitsiklis
several
detailed
and
[1989],
applications;
and
Misra
and optimization methods; and others.
with
numerical
[ 1988] present
Chandy
A
for
on
a general
(and
programs.
writing
parallel
methodology
sequential)
survey
The
area
of parallel
Parallel Algorithms
410
parallel graph
algorithms
by Moitra
algorithms
on parallel sorting.
Instruction
is given
and Iyengar
The
[1975] is
Valiant
from
these
for
algorithms
Borodin
an
machines
bibliography
to Single/Multiplealgorithms by circuits
Modeling
[1966].
Flynn
general parallel
a large
according
material
appear
in
Section
Shiloach
and
Vishkin
[1981],
ranking appear
in
and
merging,
finding,
Other parallel
is taken.
12.3.1.
in
for list
algorithms
Hopcroft [1985]. Additional
Snir [1985] (where parallel prefix
maximum
on parallel
work
early
problems
and on
[1984],
[1977].
of the
some
which
to
is due
Deo
and
of
classification
Single/Multiple-Data
sorting,
by Quinn
and
in Borodin
and
Rudolph,
and
Kruskal,
in Vishkin
used),
algorithms.
for
Algorithms
technique,
Vishkin
parallel
an
depth O(logAz).
(even though
Reif and
Ajtai,
Valiant
parallel
including
applications,
Steele
and
is
tree
multiplication
networks
multiplication
[1986].
on a
Nassimi,
by
are
[1985], and
given
Fox
Batcher
by
with
networks
exist
Section
for
it
3.2).
are
the fan-smallest
The
running
Experimental
et al. [1988].
other
in
time is
matrix
interconnection
results
The
given
element
appeared
algorithm
whose
algorithm
on a hypercube
[1981]).
algorithms
finding
[1988].
such
Cook
O(logAz) sort.
(the regular
[1987]
in
various
sorting
[1987].
from
networks
\342\200\224
see
O0(\\ogn)
for
algorithm
to run
that
a randomized
with
Frederickson
Sahni
and
[1983] proved
impractical
A deterministic
be adapted
a hypercube
an
Manber
and
[1983]).
case is given
mesh can
(Dekel,
on
Szemeredi
by Hillis
The improved
Rodeh
and
worst
the
long
there exist
for
conquer
Stout
whether
time
was definitely
exhibit a network
to Greenberg
due
Francez,
0(log3Az)in
Komlos,
and
and
in
the odd-even
is described
machine
Shrira,
and
and Snir
Rudolph,
problem
[1987]
and
Horowitz
for a
network
and
Rudolph,
in
including
open
their
connection
in Hillis
Kruskal,
networks,
Sorting
[1968].
on a
problem
and lower
[1980]. Fich [1983] providesupper
of gates in parallel-prefix
circuits.
Beame, Cook, and Hoover
describe parallel algorithms for division and related
The Euler's tour
problems.
as well as the algorithms
in
Section
are
from
and
12.3.5,
presented
Tarjan
addition, appeared in
bounds for the number
[1986]
the parallel-prefix
and Fischer
Ladner
routing
for
matrix
scheme
in Section
12.4.5 is
from
Valiant
[1982]
and Valiant
networks
on
more
For
[1982].
Upfal
and
Karlin
in
appear
[1987].
Schemes
for emulating shared memory
in
and
and
Ranade
[1986],
[1987],
Upfal
Wigderson
Upfal
networks see, for example, Feng
interconnection
[1981],
Siegel
[1982] and
in Aleliunas
appear
More on
[1980],
and Kung
algorithm
(Section
array
The
difficult
more
significantly
Nevertheless, Megiddo
Cole
by
12.1
Draw
12.2
Design an
designing
parallel
technique to design
An
algorithms.
corresponding to the
to add k binary
k) using nk processors.
Draw
the circuit
12.4
Prove
that
algorithm
12.5
Prove
that
the rules
12.6
Complete
in
Figure
that corresponds to
the details
a tree
parallel
numbers,
algorithm
n log
12.3
some types
of
of this technique
improvement
algorithm
for constructing
of the
addition
each with
Parallel
12.3)requires
an Euler's
doubling algorithm
discussed
only
Section
the EREW
in
12.3.1.
Prefix 2 of Section
tour given
to find
bits.
in
Section
should be
12.3.3.
model.
12.3.5 are
correct.
of all
vertices
(Section 12.3.5).
is an example of a graph
an interconnection network
representing
in this chapter. Identify
and label the nodes appropriately.
the network,
12.22
discussed
that
we
uses
the proof of correctnessof the odd-even
sort. The proof
transposition
and shows that, when an additional processor (and input) is added, the number of
= 2, uses
steps is incremented
by 1. The base case, n
exactly one comparison. Hence,the
total number of steps seems to be n - 1 and not n. Show an example
that requires n steps,
and discuss why the proof does not show that n - 1 steps are sufficient.
Consider
induction,
12.9 a.
Draw
the full
b.
Draw
the full
c. Draw the
12.10
usually
algorithms.
sequential
corresponding
an ingenious
found
algorithms is
parallel
designing
the
[1984].
the circuit
O(log
12.8
devoted to systolicarrays.
Exercises
Drill
12.7
[1983]has
Section
in
and
difficult,
than
by using
algorithms
sequential
is
in parallel\"
\"Thinking
was given
comparisons
Jagadish
of Computer is
issue
1987
July
in
Kung
matrix
for
are presented
9.5.1)
in
found
for sequence
algorithm
[1985].
can be
computing
systolic
Complete
fuli
circuit
Fig. 12.21 by
two
of size
8.
16 inputs.
for sorting
showing
sequences
10 inputs.
all the
steps until
the two
412
Parallel Algorithms
12.11 Design
multiplication
algorithm for matrix
O (n3 / log n) processors.
an CREW
12.12Modify
[\\..n
array
an
array
an
b.
an EREW
12.14 Parallelize
EREW
We want
of values.
to copy
n)
and
with T(n, n) = 0
algorithm
algorithm
with
efficiency
(log
0(1)
Horner's
O (log
time
n by n matrices
the
A [1]
value
to
all
other
locations
in
broadcast).
(i.e.,
a. Design
Design
two
multiply
the maximum
(Section 12.3.2) to work in the case
algorithm for finding
are not distinct. The parallel
time should still be O (1).
running
numbers
] be
that can
using
CRCW
the
the
where
12.13 Let A
n)
O (log
time
the
network.
interconnection
Exercises
Creative
in
An example of an
12.22
Figure
efficiency
n) for
and
this problem.
running
of degree n
time
under
O (log
n).
the EREW
model
in
0(1).
12.15 The
is the or function
of all of them.
of n Boolean variables. The output
is 1. It is
the output is 1 if and only if at least one of the variables
the or of n variables in flog2\302\253l steps
straightforward to compute
by an EREW algorithm
with
But
n processors.
It seems fairly
obvious
that doing better than |\"log2\302\253lis impossible.
this is not the case. Design an EREW algorithm whose running
time is less than
flog2\302\253l\342\20
You can assume an unlimited
is not
number of processors and space. (This algorithm
are not always so obvious.)
practical, but it shows that simple \"obvious\" lower bounds
In
12.16Solve
input
other
is an array
words,
Exercise
model.
12.15 (computing
or of n
Boolean
variables)
in time
O(l)
under
the
CRCW
413
Creative Exercises
12.17Draw
with 0(n
a circuit
O (log A7),
that the
such
12.19
Let
in time O
sequence
12.20
(log n).
assume
can
You
to
be a
x\342\200\236
X\\,x2
the
depth
than 2 and
parallel-prefix
assume
12.22 Let
for
an EREW algorithm
with
numbers.
distinct
can
You
design
A and B into
to merge two sorted arrays
an unlimited number of processorsand space.
0(1).
12.21
sequence of
be a
x\342\200\236
, x2
jcj
in exactly
circuit
of Exercise 12.17
with T(n, n) = 0 (log n).
problem
parallel-prefix problem
processing element is no more
one operation.
of every
fan-out
and
vertices
\\ogn)
fan-in
one
sorted
Design a CRCWalgorithm
distinct) integers in
sequence of (not necessarily
in the sequence with
to find the maximum
T(n,
array
in time
to find
the
range 1 to n.
n) = 0(\\).
the
real
let S be a subset
of {1, 2,..., n\\, and let* be
numbers,
The set S partitions
the sequence
on real numbers.
into
contains jcj, jc2
element in S,
xh where / is the smallest
groups, such
the second group contains ,v/+1
smallest element of S, and so on
jc;, where j is the second
the rest of the elements in case n does not belong to S). The group
(the last group contains
parallel-prefix problem is to compute parallel prefixes separately for all groups. Show that
the group parallel-prefix problem can be solved with the same resourcesas are used
for the
parallel-prefix
regular
problem.
an
be a
2, ...,*\342\200\236
associative
if i
that,
sequence of
binary operation
that the first group
words, C is a row
and B [j].) Design
A[0..n-\\]
two arrays
of
= k-m+r%
of an n x
representation
and
B[0..m-
0<r<m-l,
0<k<n-\\,
matrix
then
C[i]
whose
ij
an EREW
can
1] is an array C[0..nm1]
= A [k] -B[r]. (In other
product of A [i ]
processors and time
you can assume that n, m, or
is the
entry
C with p
12.25 The
a contiguous
12.26
area.) The
order
an EREW algorithm
Design
among
for this
the records
problem
that
with T(n,
are
moved
n) = 0 (log
should not be
changed.
n).
The
input
array.
of the
sufficient
is a
Design
original
efficient
reverse the
allocated to hold
list (i.e.,
space is
that
parallel
all of
414
12.28
Parallel Algorithms
an EREW algorithm
obtained by the Euler's tour
with
Design
the number of
the text, and it
n) = 0(\\ogn)
T(n,
method.
the forward
rank
to
edges
the
in
list
find
words, for each forward edge you should
is
in
the
list.
number
denoted
that
it
follow
forward
(This
by / (/, j) in
edges
associated with the Euler tour;
see Section
is needed to compute the preorder
other
In
12.3.5.)
without
the extra
12.29 Suppose that a tree T is given in the regular adjacency-list representation
a fast and efficient
EREW
the two copies of each edge. Design
pointers connecting
can
assume
that
have
as
much
work
these
You
to
construct
you
pointers.
spaceas
algorithm
want.
you
12.30
tree
by adjacency lists in the same way as described in Section
represented
EREW
algorithm to find, for each vertex /, the number
Design a T(n, n) = 0(\\ogn)
of / in the tree (i.e., the length
of the path to the root).
of ancestors
Let T be a
12.3.5.
12.31
a counter
Construct
representation
binary
of n binary
consists
Is in
the
numbers,
the
and
is the
output
input.
we
to spare, the
a step
have
terminates
algorithm
on time.
Discusswhy
this
proof
is not a
valid.
12.33
the
Consider
sum
the
have
computed
12.34
12.35
xx
in one
tree
binary
parallel-prefix problem
number
\342\200\242
\342\200\242
\342\200\242
(where
\342\200\242jc2\302\253
\342\200\242*,
\342\200\242
is an
time
associative
of the algorithm
operation
binary
should be O(log
a sorting
Design
algorithm
12.4.3.
Section
in
smallest element to
for processorsconnected
Each leaf contains a
Show
the
that
through
number,
running
and
time of
Solve
that
tree
described
algorithm
in
to
modifications
a complete binary
the goal is to get
your
the
should
can be
n).
described
xt.
each leaf /
under
tree
as
the kih
is optimal
to
a constant.
within
matrix
matrix
vector
product x-Ab
that
such
jc,
is stored
in P{ at the
end.
(Xi
^aik-bk.)
Design an
*=i
O (n)
12-37
Solve Exercise
matrix
12.38 The
(and
problem.
12.36,exceptthat
transpose ofanxn
initially
compute
initially
each
processor
Px
holds
the
\302\253th
column
of the
b{).
stored
in an n xn
the transpose
matrix
mesh
A is
the
such
that
matrix
P[z, j]
AT
that A
such that a]j = ajh
Suppose
an
O(n)
algorithm
Design
aVj.
holds
is
to
of A.
12.39 Suppose that you are given a mesh of nxn processors,each holding
a pixel of an nxn
In other words, each
black-and-white picture.
processorholds one binary number where 1
Creative Exercises
415
to white.
We would like to find the connected
to black and 0 corresponds
Two black pixels belong
to the same component if there
is a
components of the picture.
or
black path
them
are
not
considered
(horizontally
vertically;
diagonals
connecting
should
be labeled with a unique component
connected). All pixels in the same component
label. Initially,
the (//)th
that
pixel is labeled i-n+j. Considerthe following
algorithm
labels each component
in it. In each step, each processor
with the smallest label of a pixel
If any of them is
a black pixel looks at the current labels of its black
holding
neighbors.
smaller than its own label, then it updates it own. Prove that this algorithm will eventually
corresponds
that
label
12.40 Let A
be
an n x
in the
second row.
to row c(i)
12.41 Consider
element.
that o
such
/?),
in the
mesh
is stored
to n
j should
algorithm to permute
with the rows).
(starting
move to
usual
some input
column
way.
be two
is stored
to o and
Let o and n
in the first
an O(n)
Design
column
and
row
other
In
words,
row
move
should
n(j).
hypercube
topology and assume that each processor holds one
(n) parallel algorithm to find the maximum of the 2\" elements.
A7-dimensional
the
an O
Design
associated with
12.43
steps for
in an n x
stored
matrix
of A according
the columns
12.42
of (1,
permutations
\342\200\242
require c n2 parallel
it may
Let C = (V,
its index.
E)
be a directed
to compute
algorithm
number of vertices of
weighted
the costs of
G.
You
can
with
graph
shortest
all
paths
use as many
positive
weights. Design a
in G in time 0(log2/7)
where
processors as you
want,
CREW
n
is the
graph representation.
12.44
parallel
Design
Use as many
a. Design
for the
algorithms
as you can
processors
an algorithm
problem
following
while
preserving
to determine whether
a given
when
an adjacency
an efficiency
matrix
is given.
of O (1).
as
a subgraph.
b.
12.45
to determine whether
an algorithm
Design
a subgraph (namely, a cycle of length
4).
square
as
A common
Xj
= a
Design
when
12.46
a given
the seed
is given. The
running
time
of the algorithm
a.
b.
Prove
reasonable
efficiency).
416
Parallel Algorithms
12.47 Show
to embed a
how
two-dimensional
grid
of
size
2k
by
2m in the
(fc
+ m)-dimensional
hypercube.
12.48
12.49
is a parallel gossip problem. Supposethat there are n =2k persons, each with
a certain
can communicate
with another person
In each step, each person
and
item of information.
he or she knows (including
information
learned in previous steps).
share all the information
than one person in any step.
more
cannot
communicate with
A
Design a
person
communication (gossip)pattern such that after \\og2n steps, everyone knows everything.
This
12.50 Design
a two-dimensional
same as a
arrives
matrix
systolic
mesh.
two-dimensional
through
the
first
array
for matrix
column.
The
multiplication.
arrives from
result moves from
the
right
The
connections
are
the
last column.
12.51
there
are only four different
applications of sequence comparisons,
to
four
of
In
that
case, each character can be
types nucleotides).
(corresponding
Show
that
it
is
two
bits.
to
encode
the
associated with the
numbers
represented
by
possible
the characters
in the systolic implementation
costs that move following
in Section
given
12.5.3 with only two bits as well. (This short encoding is important,
since
it reduces the
In
some
biology
characters
required bandwidth
between
the processors.)
SOLUTIONS
OF
SKETCHES
EXERCISES
SELECTED
TO
You
know
my methods.
Sir Arthur
them.
Chapter
1.13
Apply
Conan Doyle
states
have 269 electoral votes overall:
possible ties. The following
Wisconsin,
Illinois, New Jersey, Massachusetts,
Maryland,
Rhode
Island, Washington DC, Ohio,
Connecticut,
Minnesota, Washington, Iowa,
North
Carolina, Oregon, and West Virginia.
Michigan,
well before the 1988 election.)
This is written
(Note:
are
There
a few
quite
New
California,
York,
Chapter 2
2.1
The
by
base
x-y
xn-yn
The
case of n = 1
for all
2.6
induction
x and
=x(xn~x
-yn-x)
+ y{xn-x
-y'^^
= x(xn~l
-y\"-l)
+ y(xn-1
-yn-l) + xy(xn-2-yn-2).
jc,
+ x-y\"^
xn~x -yn~x
x\"-yn
is divisible
in terms
of
-yx\"-]
two
first
the
terms
term
third
(_1)<*-ih
The
numbers
hypothesis;
principle by
numbers
is trivial.
natural
for k- 1.
that the claim is true
and indeed l2 = 1(1 + l)/2. Assume
the sum for k -1,
which by the induction
assumption is
_
This sum is
is (-1)*\"1*2.
(\302\243
i)\302\243/2,
plus the *th element, which
= 1,
is
(k -\\)k/2
(_1)*-i*2+(-l )(-!)*-\342\226\240
= (-\\)k-i(k2-k(k-\\)/2)
= (-\\)k-lk(k
+ \\)/2.
A\\n
418
Sketches of Solutionsto
2.8
n = 1
for
as we
can
side.
right
(a+b)2n-\\an + bn)
=
+ bn+l)
2n(an+l
The left
other
term
words,
The 2n~l
is what we want,
we need to prove
to make it
the expression
as close
ban + a bn
-bn+l).
that the
to prove
sufficient
ways to prove
are several
We say
is the
remainder
that
we
n is
divide
if
if
a < b,
a > b.
We present a proof of
congruent to k modulo 3, written
by 3. (k can be any one of 0,
theorem.
this
an integer
when
In
+ bn(a-b) = (b-a)(an-bn).
= an(b-a)
theorem.
most 0.
expression becomes
it the
without
a = b,
There
is at
term
right
that
ignored, and
-bn+l
-an+l
is
so it
+ abn)
+ban
bn+l
bn+l) < 0.
an+l
can be
factor
= 2n~l(an+l
+ 2n~\\ban+abn-an+l
2n~\\b an+abn-
2.9
is true
the desired
to
If
Assume
(a + b)n+l=(a
by the
is trivial.
+ bn+l)>(a + b)n+l.
2n(an+l
We
Exercises
Selected
is
the
1, or
2.)
We prove the
A.l
Theorem
is congruent
checked for
to k modulo 3 if
and
3.
modulo
n -
2.15
1, and
The subtle
consider
hypothesis
is that,
point
of the number
in
be easily
can
difference between
n. The
we subtract
when
question
we
\302\253,
on n-at
cannot be used
But the requirement was
vertices
for
n -
a single
is
1
vertices,
sure
numbers.
decimal
an element,
say
ah
to reduce the
size
must make
representation.
small
the
that
n -a%
in which
case both degrees are
and consider the case of n vertices.
edge,
1.
Assume
Since
all
Sketches of
Solutionsto
Selected
Exercises
419
and their sum is In -2, at least one of them, say dh must be equal
to 1
degrees are positive
and at least one of them,
1. We can now remove d( from the
say djy must be greater than
sequence and decrement dj by 1. There are now n -1 numbers whose sum is In - 4;
exists
a tree with these degrees.
We can add
therefore,
by the induction hypothesis, there
one additional
vertex to this tree and connect it to the vertex whose degreeis dj - 1 to obtain
the desired tree.
2.29
the
two
If
is true for
principle
balls
+1
true for n - 2. Assume that it is true for n and consider n + 2. Take any
The graph without them is Kn, and by the induction hypothesis it can
be partitioned into nil spanning
trees.
We need to extend
those spanning
trees to include
v
tree.
The edges included in Kn+2 that are not covered
and w, and to add one more spanning
are connected
to v and u. We partition the set of n vertices
into two
by Kn are those that
the edge (v, u) and all edges
equal-sized groups X and Y. The new spanning tree contains
trees
from v to vertices in X and from u to vertices in Y. Each of the previous spanning
we can extend
each of these spanning trees to span
includes all vertices in Kn. Therefore,
a vertex of X to u and an edge from a vertex of Y to v. We
Kn+2 by adding an edge from
will
different vertices of X and Y for each spanning tree.
choose
two
2.38
It
is easy
to remove the
2.
We
value of c.
We
+ Acn \\og2n
Both inequalities
clearly
that T(n)<cn
guess
and
c2(log22)2>4,
(1)
n/2
/i(/i + 1)/4+1/2.
+/i/2
power of
2cn (\\og2n)2
= T(n-2) + (n-\\)/2+
\342\200\242\342\200\242\342\200\242
=
+ 3/2+
Let n be a
3.5.1) that
for some
can
simplify
+ 2cn
hold
that
for
(2)
(\\og2n)2. We
have
>2cn
c(2n)(\\og2(2n))2
to verify
(\\og2n)2
(see Section
+4n
log2(2\302\253),
(2) to
+ 4n.
+ 4\302\253
\\og2n
c >2.
We have
\302\243/Mog2i
3.15
recursion:
= T(n-\\) + n/2
1+2/2
3.14
w.
T(n)
3.8
v and
We
Chapter
3.6
is clearly
vertices
<
The functions f
xk\\o%2xdx =
(n) = 3n,
g(n)
jf]-[
= 2n, and
\302\260gX~
s(n) = r(n) = n
\\\\
form
=0(nk+l\\ogn).
a counterexample
to the
claim.
SelectedExercises
of Solutions to
420
Sketches
3.17
We
can
and
the other
the two
construct
one grow
functions
by
one of
letting
odd numbers.
for the
fast
We
them
/<1) = S(0=1.
f
- 1) +
(/?
grow
define
odd
when
n is
when
a?
when
n is
even
when
n is
odd
is even
and
g (n
f
g(<n)=\\
all odd
3.19
+ (f(n-\\))2
g(n-\\)
/(n) is about
for
- 1) +
all even
m and
\302\261
O (g
c are constants.
The
result
the
square
of
/ (n)
is
S(n) = o[nlogmUmlogmi
3.21
is T(n) = 0(n).
The solution
to show
have
To
>
dn
it, we
prove
guess that
constant d.
for some
T(n)<dn
We
that
k
+ en
a-,(dnlbt)
\302\243
i=\\
=dn^
a\\lb\\
+ en.
/=!
3.25
Since
/=
If we
replace
> 0, the
a,lbt
\302\243
i = \\
above
inequality
the
term
T{\\yn\\ ) with T(n/2), then we get the typical
divide-and-conquer
is no
relation, whose solution,
3.4, is 0(n log a). Thus, since
by Theorem
than n/2 for n >2, T(n) = 0(n logn). However,
it is possible that
is smaller
greater
T(n)
than
that.
Let's try T(n) = en. The basecase implies
that e>\\. Consider 2n. We have to
prove that
recurrence
2cn > en
3.29
a.
is clearly
which
If
+ n,
+ Vo?
power of 2,
we can substitute
n is a
case,
the
then
n >2.
and
recurrence
the
T(k) = T(2k).
r(*) = r(*-i)+i,
The
of this
solution
b. Consider now
replaced
by
n = 2k' - 1,
numbers
recurrence relation
will
and
always
will
T(k) = T(2k -
always
1),and obtain
r(k) = 2T(k-\\), r(i) =
The solution
of
this
be used.
recurrence
In
relation
this
for
is clearly
T(k) = k + 1.
If we
n=2k-\\.
then we
always
no)=i.
7((a7-1)/2),
involved
will
following
be of the form 2J -1
the recurrence relation
for
some
j.
We can
now
define
i.
recurrence
relation
is
clearly
T(k)
= 2k~].
The
recurrence
relation
of Solutions to
Sketches
421
Selected Exercises
vastly
Chapter
4.5
16 is
Since
4.12
use an
will
We
occupied, 8 (16's
easy.
must
number
minimal
of size
array
is more
removal
parent)
Thus, 5 is the
be occupied.
must
location
specified
and
16.
/. Therefore,
insertions are
and we do not know which
locations are occupied. We handle removals by linking all the occupied locations in a
list. An insertion of / thus involves marking the /th location and inserting / to the list.
A removal deletesthe first element from the list and unmarks that element.
linked
4.13
We
first
and
rearrange
remove
roots of
the
appropriate
correct place.
4.15
4.17
We
use AVL trees so that
insertions
and deletions can be done in 0(\\ogn)
To
steps.
for jc, except
that we branch right
we use the regular search algorithm
perform Find_Next(x)
if we meet x and x is not a leaf. If x is not a leaf, then the search leads us to the desired
element. If x is a leaf, then the element for which we are looking is the last node on the path
from the root to x where
we branch to the left (if we always branch to the right, then x is the
We can, during the search, store a pointer to the last node from
in
element
the
tree).
largest
which
we branch left and use it if x is found to be a leaf.
We
AVL trees
new field
w.Left.Rank = w.Rank
- 1.
w.Left.Right.D
(If
any
of
Find_Next(x,
the
is unsuccessful).
according
introduced
children
k) we
first
We
does
find
not
v such
exist,
that
as it is according
a new field, we must
to ranks
show
the
v.Key
node
to keys.
how to
value
is
taken
as
0.)
To
perform
search
search
422
Sketchesof
take
Here is
4.23
in
another question:
did we
Why
search for
a key
along
(delete)
add a descendant
that,
(Notice
path.
if the
changes.)
field
we increment
element
an
the search
instead
of a rank field,
which
easier?
much
the smaller
elements) be /*, and the
(i.e., the tree with
Assume
that
other
case
is similar). First, delete
(the
hl>h2
right
height
the maximal element from the left tree. Denote this element
by r. Then, use r as a new root
for the right
tree, and insert this root in the appropriate
place on the right side of the left
More precisely, traverse the left tree, taking only right branches, for h, -h2 steps. Let
tree.
concatenated
tree will have r in place
node at that place be v, and its parent p. The new
the
of v as the right child of p, v as the left child of r and the root of the right tree as the right
of r (the right tree remains below its root on the right side of r). It is easy to verify
child
search
tree. This insertion may invalidate
the AVL property,'
that this is a consistent binary
case we can use the usual remedy of a rotation.
in which
Let
height
of the
of the
tree
the
tree
\"left\"
be h2.
Let
be the
Th
a tree
clearly
N(h +
\\)
+ N(h-\\)+\\.
= N(h)
is the
This
number of nodes
on
bound
4.27
nodes
all
is unsuccessful,
deletion
we insert
When
follows.
fields
descendant
would
4.20
is done as
This
place.
(decrement) the
insertion or the
Exercises
to Selected
Solutions
Its
lower
n
nodes
leads to
solution
the
be used
in the theorem.
can then
bound
given
on the
the upper
bound
lower
to prove
sums
of numbers in a special way. Let's
by S, to store
from 1 to
k. We store in S [2k~x ] the sum of all numbers
into two subproblems of half the size \342\200\22
so, we have divided the problem
the left side of the array (from 1 to 2*\"' -1), and the right
side of the array
maintaining
The first subproblem
(from 2*\"' + 1 to 2k - 1). Each subproblem can be solved
by induction.
We
use
the extra
array,
n = 2k -
solved independently,
in S
appropriate change
denoted
1 for
some
the
is performed,
Add operation
can also be solved
The second subproblem
sum.
that we add the value of S [2*_l] to the returned value of Partial
independently,
except
If we look beyond
the recursion,
what happens is that S [i ] holds the sum of the numbers
from A[j] to A [/], where j is the closest
index to / that is divided by a higher degree of 2
than
i. For example, S [ 12]holds
the
the sum A [9] +A [ 10] +A [ 11 ] +A [ 12], and S [6] holds
value of A [5] +A [6]. We leave the exact details
to the reader.
can be
4.29
We can
except
[2*_l
must
be made.
by
implement
using
the exercise,
Add_all(y),
we associate
all,
in a straightforward
one more
global
variable,
Scale
to the
of Solutions to
Sketches
Selected Exercises
423
Chapter 5
is the
loop invariant
The
5.4
Only one
that
following:
person
is i
person
next
the first
among
= j,
if next
orj
persons
can be the
candidate,
and
= i.
if next
The
5.12
5.14
is similar
solution
to
of the
that
induction
The
for a tree
be
left
is trivial
solution
solvethe
an
for a tree
problem
arbitrary
distance between w
Let v
with
and
one
or two
n vertices,
leaf in
a smaller
with
with
u.
5.15
be an
from the
Induction
hypothesis:
n vertices,
and
how
We know how
to find
the
maximal
to find
distance
of subtrees
a fixed root.
the diameter
from
<
with
Given
a tree with n vertices, we designate v as the root, and solve
for
all
the subtrees
rooted at the children of v. Notice that the
induction
problem
by
to find the
distances we find are those to the children of v, and not those to v. However,
distances to v, we need only to add 1 to all the distances. After doing that, we compare the
with the sum of the two maximum
maximum
diameter found among the subtrees
distances
from v. The larger of the two is the diameter.
the
5.21
without
input set
this
without
if
element;
the problem
has
been
Chapter
6.3
for the
We
been
cut.)
424
Sketches of
6.11
We
Selected
Exercises
Partition (Fig. 6.9) such that, if the pivot is not equal to X [Left ],
Let X[l] = 1, X[n]= 2, and let all other
with X[Left].
will
In
2.
this
the
than
be
elements greater
case,
clearly be X [n ]. Algorithm Partition
pivot
as
mentioned
then
with
will
first exchange
above,
X[n]
X[n] with X[2]
X[\\]
exchange
(sinceX [n ] is now < 2 and X [2] > 2). Since all other elements are greater than 2, no other
exchanges will take place. The result of the partition is to put 1 and 2 at their correct places
will then recursively sort the array from the third
of the array.
in the beginning
Quicksort
this pattern, we need to put the third smallest element in
to the last one. To continue
index
in the last position in the array after the partition
the third position, and, since the element
to put the fourth
in the second position, we need
smallest
element in the
was originally
if n = 8, then a bad order will be 1, 4, 3, 6, 5, 8, 7, 2.
For example,
second position.
first
algorithm
modify
the
then
6.14
Solutionsto
is
pivot
first
exchanged
sizes different
two equal-sized subsets
(or into two sets with
by 1 if the
and merge the results.
set is odd). Solveeach subproblem
recursively,
the maximal
and minimal
element.
recursive
solution produces two results,
Each
Only two
to
the
two
one
to
the
two
are
and
solutions;
maximals,
merge
compare
required
comparisons
The following recurrence relation
is obtained:
one to compare the two minimals.
Divide
the
set into
size of
the
original
T(n) = 2T(n/2) + 2,
7(2)= 1.
is easy
2T(n)+ 2
to
by
verify
induction
= 2(3/i/2-2)
suggested in
Section
6.5.1 is
better in
this
case.
6.19
6.20
We can
6.22
a. Sort the
set S\\
b. We
induction.
use the
use
simple
Induction
of <n
for each
then,
element z of S, perform
hypothesis:
We
know
how to
binary
solve the
search
problem
for the
for a
number x -
z.
sorted set
elements.
in increasing
and let
that S is sorted
order. Consider 5[1] and S[n],
S [n J. If y =*, then we are done. If y > jc, then clearly S [n ] cannot be part of
the solution because S[n] + S[i]>x for all /. Therefore,
from
we can eliminate
S[n]
consideration and solve the remaining
<
a
similar
If
induction.
then,
jc,
by
problem by
y
argument, we can eliminate S [ 1 ].
Assume
=S
[ 1J +
of Solutions to
Sketches
Selected Exercises
425
6.26
the largest,
the set, and pair the smallest element with
the second smallest
with the
second largest, and so on. This procedure is correct for the following reasons. Suppose that
there is another partition with
a better bound, in which x, and xn are not matched. Let x, be
matched
in this partition with
But then, we can change
the partition
so
jc, and xn with Xj.
that
jc i is matched
with xn and jc, is matched with xjy without
the
maximal
sum,
increasing
which
is a contradiction.
6.28
The
6.29
Sort
permutation
all the
put
Initially,
remove the
of the
elements
minimal
in the
element
minimal
heap
and
insert
sequences
the next
in
a heap.
Then,
element from
in
each
step,
the corresponding
sequence.
6.31
a.
each
Insert
the tree
a key
has
tree.
b. The
proof
6.33
permutations
to only
restrict
is at
does not
sorting
all
those
apply
to this
are possible
with 0(}ogn)
valid
different
values.
most
r(/o
To evaluate
7(7)=
= 3
+ i)//)l.
2xriog2((/i
T(n)v/e
log28
+ log24+riog28/3l
+ 2 + 2+1
+1+
+ 1.
There
are 4
assume
that
=2*.
It is
of size
not
difficult
it
6.36
T(n)
This
sum
be
most
at
Suppose
Zi2k-i.
was
that
comparisons
smaller
being
than
shown
an extra
equal to a
This
value
there
a series
can
of
or
that
the
is
outcome
of
the
consistent
with
implies
comparisons
Xj
z <Xj. But both cases lead to a different
kth
y >x, and to a value
is a contradiction.
is greater
426
Sketches of Solutionsto
6.38
The idea is to
consider
their median.
All
2k elements
elements
consisting of computing
greater
and do
next k elements
same.
the
the median
n + 1 possible
must have at
problem
process requires
of 2k
elements.
answers
least n
is discussed in
Exercises
Selected
+ 1 leaves.
This problem
6.46
6.51
b serves
Part
(and the
as a good
hint
for
each
tree that
solves
10.2.
6.44
Section
the
phases,
nlk
approximately
find
at
the solution
of part
T and
We find the
a.
hard
to do,
problem
minimal
edit
(see also
distance
minimal
sequence)
between
and to
since the
is known,
that
under
the reader.
The easiestway
6.54
other
that the
except
words,
to solve
initial
values
want.
The
6.60
(not
answer
surprising)
algorithm
Vegas
that runs
in
is
that
expected
it is
better to be correct
always
than
to be fast.
produces
Given a Las
the correct
result, we
6.61
transform
Each
candidates.
algorithm, except that we keep up to three different
is compared to the candidates,
the
then
and if it is equal to one of them,
from
is
is
different
if
new
element
incremented.
the
Otherwise,
corresponding multiplicity
the existing three candidates,
of all candidate (by the
then we can decrement the multiplicity
same argument as in the majority algorithm \342\200\224
from
we can eliminate four distinct values
If there
are less than three candidates,
we add the current element as a new
consideration).
candidate. At the end, all remaining
are compared to all the elements.
candidates
We
use
the majority
element
Chapter
7.4
Let T be
Let e
Sketches of
7.8
7.24
The conditions
and its
7.27
Perform
in
7.29
7.34
that
of the
traversal.
problem
(a BFS
satisfies the
a DFS
tree
will
imply that G
do just
is Eulerian.
as well), and
conditions of the
Solutionsto
Selected
C in
an Eulerian circuit
Consider
construct the
427
Exercises
DFS
tree.
Prove
that
any
G,
leaf
problem.
We
(preorder)
in
the
regard
maximal
that
That is, we
matches
the
define
suffix
v.match
to be
equal
to the
the root to
v.
size of
(This
of vertices in
7.35
Without the cycle, the graph would have been a tree for which the problem is easy (pick a
root
and direct all edges away
from it). We can discoverthe cycle
the first
by DFS when
is found.
to a previously visited vertex
The cycle can be directed
so
(and only) edge leading
1. All other edges should
that it becomes a directed
and all its vertices have indegree
cycle
another DFS from any
be directed
away from the cycle. This can be achieved by starting
all edges in the direction they are visited
the search.
vertex
on the cycle and directing
during
The single back edge will point to the root of the search which will have indegree 1 (as will
all other
7.37
the tree.
vertices).
Perform
7.40
7.42
known
paths
shortest
to y
when
428
Sketches of
7.43
The
no heap
time (since
Add to
compare
possible
small
be
7.48
V \\)
find a new
we
Instead,
time,
minimum
update for
each
is maintained).
each
the
that
O (|
takes
This
Exercises
Selected
to use a heap.
is not
idea
main
the vertices.
7.45
Solutionsto
that it
enough
taken
as m/( |
then
m -
1).
We
use
induction
does not
on k,
paths with
affect
V | + 1), where
find
and
different
the precision of
m is
we discuss
vertices. For simplicity,
of k = 1
themselves.The
case
paths
weight.
if they
(e.g.,
weights
are
integers,
path
only
finding
regular
the
is obvious.
7.50
well.
(x, y)
for a
(Hint
it is
if
consider
smaller
than
the
each edge(jc,
y) set
The
time
running
Acyclic
Improved
Algorithm
need to
w. We now
Whenever
proof:
y.SPk
is thus
edges as
it
from
the
7.52
The
must be Eulerian, since each cycle adds a degree of 2 to each of its vertices.
On
the edges of an Eulerian
can
a
set
of
be
into
always
partitioned
cycles.
graph
is enough
to note that there exists at least one cycle in an Eulerian graph, and that
graph
removing
a cycle from an
Eulerian
graph
gives
a set
of
(possibly
one)
connected
Eulerian
graphs.
7.55
7.60
Start with F
as
algorithm
Exercise
7.64
This
in
the
current
\"minimal\"
spanning tree
and
add
edges
to
it
according
to the
7.59.
For
the MCST, call it T. (The answer for all edges of T is obviously
T itself.)
v
v and w, find the maximum-cost
in
the
connecting
(unique)
path
edge
to w in T. This
is a preprocessing step. We may
for all pairs, but it
not need the information
First
find
each pair
the edge
tree containing
them. A minimum-cost spanning
the
the
maximum-cost
to
then
7,
edge
edge (jt, y)
by adding
removing
the unique cycle that is formed (or the second largest,
if (.v, y) is the maximum). This
is the maximum-cost edge on the path from x to y in 7\\ whose identity we obtained in
DFS on T for | V \\
step. The preprocessing
preprocessing
step can be done by using
is simpler
U, y) is
from
edge
the
of vertices
to compute
obtained
it
for all of
Sketches
7.67
time starting
each
times,
vertex.
all
order
of the
costs, it
also
does not
change the
result.
running of the
algorithm.
The
main
problem
vertices.
nil
7.71
Exercises 429
costs
the
change the
7.70
to Selected
Increasing
of
7.68
from another
of Solutions
Since every
E-F cannot
cycle
in G
an
must contain
We want
The largest
a cycle.
the size of E-F.
contain
set F, the
which
set of
is the
edges
same as
to
7.74
a spanning
Check
w.SP
the
length
tree.
430
7.79
connected in the
the other hand,
vertices according
7.84
order, and
a topological
Find
We proved
to their
in
that
path,
run
the biconnected
will
a slightly
prove
the claim
path.
stronger theorem
(and
leave it
There is
Proof:
is not
one
components
to show
reader
are
in
graph and to
to the same
it implies
that
A.2
that
to the
if they
the exercise).
in
D Theorem
from
gives
same biconnected
path
consecutive
in Section
the
We
place
vertices with
topological order
for all
check,
then
biconnected component
7.85
to SelectedExercises
of Solutions
Sketches
then
\302\243,
in
and to
component
can be connected
the existence of a
cycle
one simple
at least
obviously
contained
it has
come back
only
involving
more
path
v and
between
to leaveB and
through
through
component B, then
to come
another
(since
one articulation
than
back.
is
it
point).
one biconnected
But
simple
any
is
B
and two
in B.
If there
has
to leave
It
simple,
would
that
imply
which
component,
is
impossible.
7.87
Pick
articulation
point v. Remove v and its incident edges from the graph and
connected components of the resulting
graph. Then put back the removed
Consider
now each component separately,
and
edges into the appropriate components.
and so on.
perform the same procedure
by picking another articulation
point,
find
7.89
an
the
arbitrary
different
to
may be to try to modify the strongly connected component
algorithm
the
values
more
For
instead
of
DFS
we
numbers,
update
High
accurately.
example,
using
can use the currently
known High values. In other
words, when an edge (v, w) is
considered
for the purpose of computing
we use w.High instead of w.DFSin the
v.High,
However, w.High may be updated later (e.g., one of its other children may
computation.
have a path to a higher vertex). Updating
of
all the vertices that
used the previous value
A
first
w.High
Denote
same
approach
concerned. That
Hence,
it is
connected
DFS number
connected
strongly
is,
sufficient
component.
of
from v by v.RealHigh.
have the same characteristics
can reach a vertex w, then all
reachable
component
if any
approach.
them
for
v.RealHigh
compute
We
first use the strongly
to
All
in the
reachability is
them can reach w.
the
vertices
as far as
of
one representative
connected component
of
each
strongly
algorithm
to
of the root
connected
(this is the vertex with the highest DFS
component
number
in the component).
SG must be acyclic, so we have reduced the problem to that of
the
of this reduction is that it
The advantage
values of acyclic graphs.
finding
RealHigh
solves the problem with repeated
SGis acyclic, there
Since
we
before.
had
that
updating
are no back edges. Therefore, at the time the edge (v, w) (in SG) is used to update
the value of w.RealHigh
We leave it to the reader
is the correct final value.
v.RealHigh,
for example,
to prove this fact \342\200\224
induction
is
not
trivial).
by
(the proof
Each node
in
SG
is marked
with
the
DFS number
Sketches of
7.92
Solutionsto
431
Exercises
Selected
a. Supposethat
v belongs
indegree, then
b. Sinceno
exists
there
is on a
vertex
of indegree
a general graph
the SCC
shown,
7.95
now,
we
first
is acyclic.
tree with an
the tree as a rooted
both from
w.
Remove
parent
Consider
it to
graph
arbitrary
root.
7.9). As we have
component covers
one of the vertices
leaf
Take an arbitrary
*
from each
vertex)
and
match
become
(and
disconnected). Solve the resulting problem by induction. We need to prove that the edge
If M does
Let M be a maximum
(v, w) indeed belongs to a maximum
matching.
matching.
to w). If (w, w) is in the
not
contain
(v, w), then v is not matched (it is connected only
then simply replace it with (v, w). It is still a matching, and it has the
maximum
matching,
same cardinality.
7.99
its
a. A graph
must not
b.
include
we
vertex,
arbitrary
may
a vertex
tree
of degree0 to have
component that
up with a
end
the
all
other
'brothers\"
such a cover.
consists
that
When we remove an
of only
one vertex.
assume
We now
that
reduced
the
that
7.100 We use
binary
7.10.
Section
way
as
conquer.
as required
regular
procedure. The
10.18.
and
similarity
for
merge.
of
432 Sketchesof
7.106
Clearly,
to Selected
Solutions
at least
k colors
Exercises
all
edges
incident
differently.
We
will show
that
k colors
are also
such that
to the same
vertex
must
be
use divide
the subproblems
the problem
are the same as
but we have to divide
and conquer,
we have to ensure that
the
If we divide into two smaller subgraphs,
the
problem.
original
that
are
a
2.
in
are
and
of
This
is
each
vertices
of
all
they
power
subgraph
equal
degrees
a circuit exists, since all degrees
circuit of the graph (such
achieved by finding an Eulerian
the circuit and divide the edges into two groups by alternating
are even). Then, we traverse
between the two groups (i.e., the first edge goes to the first group, the second to the second
defines a subgraph with all the
group
group, the third to the first group, and so on). Each
of
the
The
and
one-half
vertices
original edges.
degree of each vertex in each
original
a
since
whenever
an
vertex is put in one group, the
is
fc/2,
entering
edge
exactly
subgraph
is leaving
the vertex) will be put in the second group.
next edge (which
The rest can be
We now have graphs with fewer edges and, more important,
smaller
done by induction.
but we use distinct
We color all the edges of the two subgraphs
colors
separately,
degrees.
The complexity
of
for each group. Therefore, we can just put the two colorings together.
the
for
Eulerian
which
takes
linear
this algorithm is dominated
circuits,
by
algorithm
finding
but which has to be performed
for each subgraph. The number
of stages of this
time
algorithm is log2fc (since k is halved at each stage); hence, the total running time is
is k can be
O (| E | log k). (It is interesting to note that every graph whose maximal
degree
edge-coloredwith at most k + 1 colors; see, for example, Chartrand and Lesniak [1986].)
colored
sufficient.
7.108 A
We
want to
common
from students'
a
examinations) is to consider
approach to this problem (judging
root of the tree (any vertex can serveas the root), to find minimum vertex coversfor all the
subtrees below the root, and then to include
the root in the vertex cover unless
all its
in the vertex coversof the subtrees.
This seems like a good
children are already
included
since the root must indeed
to the vertex cover if any of its children does
approach,
belong
not
vertex
covers (see Fig. A.l).
However, there may be many different minimum
belong.
For a given subtree, it is possible that there is a minimum vertex cover that includes its root
and one that does not include the root. Therefore, this approach
does not necessarily lead to
minimum
vertex covers (unless more precautions
are taken).
A different
can be
reduction
achieved by considering
a leaf first. The edge from a leaf to its parent must be covered, and
can be coveredby either the leaf or its parent. There is no advantage
the leaf,
to choosing
Figure A.l
tree
with two
minimum
vertex
covers.
of Solutions to
Sketches
and
the
incident
to have
guaranteed
More precisely,
else.
if we
Therefore,
parent.
its
all
433
there is a minimum
vertex
cover that
choose the parent (of any arbitrary
remove it
leaf),
edges, and then solve the remaining
problem
by induction, we are
a minimum vertex cover. It is not difficult to implement
this algorithm
it does
since
includes
SelectedExercises
in linear time.
7.109
It
We
cover.
weight
Solution 1: Perform
level number according
(even level
numbers
must be
that
show
will
of a vertex
to its
each
edge is covered by
exactly
one vertex,
so
the
vertex a
numbers
a vertex
has odd
cover
is
minimum.
trees with
the
Let the
tree.
problem
for
resulting
vertex
have
v and
be w,
again
is still valid;
include v in
otherwise,
cases the cover of G is minimal.
But, the
of G must be at least 1 more than the minimumextra
edge must be coveredby either v or w (and it costs 1
of w is increased). We achieve
this bound, so the
degree
to prove that
vertex
cover
since the
case, since the
cover is minimal.
of G',
in either
more
7.114
minimum-weight
cover
weight
leaf
vertex
adjacent to v
induction.
Consider
by
vertex cover of G', then
We
states that we
arbitrary
only
G'
minimum-weight
the cover for G.
size of a
hypothesis
Pick an
this
cover
in both
vertex
of G. Denote the set of vertices adjacent to v by N(v). If v
arbitrary
to the vertex cover,then none of the vertices in N(v) can belong to the vertex cover
all the vertices
Furthermore,
(since the vertices in the cover should be independent).
is
to
vertices
from
must
to
the
since
that
the
cover,
N(v)
only way to cover
adjacent
belong
if any two vertices of N(v) are adjacent,
then the edge between
those edges. In particular,
is continued until either it
be covered, and the procedure fails.
This procedure
them
cannot
is found. All the
case v cannot belong to the vertex cover, or a vertex
cover
fails, in which
the vertex cover that is
steps of the procedure are determined
by the choice of v; hence,
v that satisfies the conditions
of the problem. If this cover
found is the only one containing
v cannot belong to the vertex
is not too large, then we are done; otherwise,
cover. But, if we
vertex
all
must
determine that v cannot
to
the
then
its
vertices
cover,
belong
adjacent
belong
to the cover, and we can apply the same procedure.
Let v be an
belongs
7.116
represented by two numbers (/,, r7) (the left and right endpoints). Sort the
in the sorted order
to the r,s. Denote the intervals
(i.e.,
by /,, /2,..., /\342\200\236
is a maximum
the right endpoints). We claim that
there
r, is the minimum
among
To prove it, take any maximum independent set and
/,.
independent set that includes
the interval /, in it such that t'j is minimum among all other right endpoints in the
consider
with
But
that implies that /, does not intersect
set. Since r, is the global minimum,
r} >r,.
in the set (except possibly
the
with /,); hence, /, can replace /,, and
any other interval
follows
set. The algorithm
modified set is still maximum
independent
directly by induction,
because we know how to handle /1.
Each
interval
intervals
/, is
according
Sketchesof
434
7.119 a.
Let
v,,
matrix
Exercises
to Selected
Solutions
Is
to itself).
The rows corresponding to v, and v3 both contain
some
a
of
is
characterization
a
of
to
and
This
sense,
is,
v4.
v2
cycle
corresponding
is symmetric
since the graph is undirected.)
We can
4. (Notice that the matrix
length
look at all pairs of rows, and for each pair check its intersection. In other words, for each
There
pair of vertices, we check all the other vertices that are adjacent to both of them.
contains
at least two vertices. Any row
is a square if and only if any such intersection
a row and operation) can
be performed
in linear time.
intersection
(which is basically
There are O (| V \\2) pairs of vertices, so the overall running time is O (| V |3).
considered
b.
adjacent
representation. First sort all the edges in all the lists. Then
between two rows described in part a can be performed
in
number
of edges incident to the two corresponding
vertices.
Use an adjacency-list
the intersection
that
proportional to the
7.120 Compute
vertices,
squares
the
and
maximum
number
show
it may
Chapter
8.6
counterexample
The
that
is given
it
is on
the
of squares that
be more than
\342\200\242
(| V | | E |) time.
in
Fig.
A.2.
be contained
can
0(\\
The circled
\\
\342\200\242
point
\\E |).
will
in
show
time
with | V
a graph
Therefore, just
listing
be removed
Graham's
by
hull.
8.11
Sort the
8.16
Divide
the plane into vertical columns of width
d starting
among the points and ending with the largest x coordinate.
Figure A.2
counter
example
with the
Find
for Exercise
8.6.
x coordinate
minimal
the points
with
\\
all
the minimal
Sketches of Solutionsto
435
Exercises
Selected
and
y coordinates
maximal
since
column
the
can be
algorithm
observation
The necessary
points.
that
the
Thus,
here
requires
algorithm
comparisons to
additional
is that
to find the
comparisons
all
first
complexity.
other
vertex
remaining
of the
hull,
8.22
8.26
the
find
case is
We strengthen
induction
second,
of finding
that
angle
45-degree
hypothesis a
We know how to
Induction
other
hypothesis:
intervals among
endpoint among
a set of < n
intervals,
the intervals
all
mark
and how
to find
contained
the largest
in
right
them.
interval;
interval;
we mark
problem
endpoint
8.28
little.
The
for
the
according
first (leftmost)
to
-1
it.
intersection
constant
can
to the
or it
rectangle
in
axes can be computed
is a rectangle. Hence,we
after the other in any
order.
8.31
We give
slabs and
idea is
First,
basic
we sort
all
to
divide
the vertices
of Solutions to
Sketches
436
slabs. An
because each slab
into
intersections
slabs, as in shown
in
Fig.
a polygon
in
Since we know
A.3.
only to
compute intersections
two slabs can be computed
intersection
between
then
assemble
has at most four edges. We can
need
between
disjoint
in constant time,
the corresponding
time.
linear
8.34
Chapter
One example
The method
is w = 15.
(jc15 = (((x2-x)2)-x)2-x).
(jc15 =
9.10
into
sorted
pairs of
9.2
two polygons
lines, we
to divide the
lines
vertical
the
Selected Exercises
It
((jcjc)2jc)3). See
We use the
notation
the following
types of
of Section
which
introduced
presented
in
Section
to
compute
x]5
possible
p. 443]
1981,
[Knuth
definitions,
products
is
for a detailed
9.2 requires 6
with only 5
discussion of this
multiplications
multiplications
issue.
(B +C +D + F +G)X. We make
in the four
to
the
seven multiplications
correspond
directly
in Section 9.5.2: z, = b(e+f\\ z2 = c(g+h),
z3 =
9.5.2. We need to
compute
We
z4 = (a-h)e, z5 = (a -c)(g -e\\ z6 = (d-c)h, and z7 = (d-b)(f-h).
now
B contributes [z,, z,, 0, 0] (we write it
look at the contribution of each of the matrices.
as a row matrix instead of as a column matrix for convenience); C contributes [0, 0, z2, z2];
D contributes [0, z3, -z3, 0];F contributes
[0, z6 + z7,
[z4, 0, z4 +z5, 0]; and G contributes
= Z\\ +
we have p = Cu = z, +z4, r = C2J
0, z6]. So,overall,
z3+z6 + z1, s = C,i2 =
+ Z5, and t = C2,2 =z2 + z6. There
are 18 additions and 7 multiplications.
Z2+Z3+Z4
(c-b)(e+h),
part
it does
of this
problem is proving
by induction.
It
is easy
to
that
verify
Figure
A.3
Intersection
of two
convex polygons
and F(k +
1)<n.
By
log2(w -F(k))
by the slab
method.
1)> n).
the
We
induction
Fibonacci
Sketches of
numbers.
To
that
9.19
since
But,
the numbers
keeps
find
the
We denote
furthermore,
parts of the
four
the
437
Exercises
of n-F(k)
still
adding F(k) to the representation
- F (k)) + 1 <
it is easy to see that
\\og2(n
log2w.
all Fibonacci numbers until we reach one
we need, we compute
to n\\ we can then follow the proof given
here.
than or equal
second
the
Selected
n-F(k)<F(k),
distinct;
representation
is larger
Solutionsto
by a2,
polynomial
as shown in Fig.
first
by a,, b,, c j,
We can illustrate
polynomial
d2.
and
d,,
the
parts of
using a 4x4
problem
A.4. This table is similar to the 2x2 table that was given in Section
9.4. Each entry in the table corresponds to a product of two parts of the polynomials.
We
of each entry, only
do not have to compute the value
the sum of values in each
of the
in the figure. Each of these
to the coefficient of a
corresponds
diagonals shown
diagonals
in the product of the polynomials.
The following 9 products are sufficient
certain
to
degree
\342\200\242
\342\200\242
(1) a, \342\200\242
(a 2 + b2), (4)
a2, (2) b, b2, (3) (a j + b,)
compute the values of all the diagonals:
+ rf2), (7) (a 1\"+c1)-(tf2
+ 6-2), (8) (/>, +</,)\"(*2+</2),
c, -c2t (5) dx -d2, (6) (c,+rf,)-(c2
and (9) (a\\ +b\\ +c'i +d\\) -(a2+b2+c2+d2).
We leave it to the reader to verify that all
from these 9 products. The corresponding
recurrence
relation is
diagonals can be computed
T(n) = 9T(n/4)+ 0(n), which implies that T(n) = O(wlog9') = 0(wlog3), which is the same as
of the algorithm that divides
each polynomial into two parts.
Is that
the running
time
In other words, what
a good reason for it?
are the
equality coincidental, or is there
similarities between the two algorithms?
table
9.23
assumes
of multiplication
that
(i.e., it assumes
algorithm
commutativity
If
we
use
as
the
base
of
the
we
have
to
be
able
recursion,
x).
-y =y
Winograd's
algorithm
matrices for elements, but we cannot do that because
matrix multiplication is
substitute
Winograd's
x
to
not
9.29
commutative.
We solve the
the solution
The best way
problem
for
Stronger
by induction
n > 1,
induction
hypothesis:
We
know
the best
way
to compute
M[i.j]
<j<n.
ax
bx
cx
dx
ai
b2
Cl
d2
Figure
A.4
Finding
polynomials
parts.
438
Sketches of
to Selected
Solutions
Exercises
To extend
steps. The
of n-i
consists
loop
number
total
is
of steps
thus
which is
\302\243
XX#0),
k=\\ i=k j=i
0(n3).
10
Chapter
10.3 We show
the interval-containment
that
each
For
algorithm.
coordinate is -L,
interval is contained
maximal.
10.5
each
Split
and
vertices to
matching
all the
both
of those
bipartite
regular
solution
to this problem is by reduction.
We construct a new graph H with two
w of G. We call w{ the tail vertex of w, and w2 the
w{ and w2 for each vertex
head vertex of w. The edges of H are the following.
For each edge (w, w) in G, we add the
0
m
with
cost
to
H.
In
other
we
make
all edges go from head vertices to
words,
i)
edge (vv2,
tail
vertices.
In addition, we add an edge (u\\,u2)
with cost c(u) for each vertex u in G.
The problem
becomes
the regular single-source shortest-paths
from v2 to all tail
problem
vertices
vertices.
10.15 The
we
10.17
The
linear
solution
10.20 To compute
its
size.
changed
every organization to
easily,
donate
but
two
are x<0
smaller than 0.
constraints
number
it is much
easier
to
the
and
which
x*0.
that the
notice
This
best
its maximum.
matrices of half
10.24 We show
numbers
can be
program
is for
x.
variable
of matrices
not
squarings
half
of
size.
such an algorithm
to sort. Given a sequence x^x2
xn of distinct
sort, we associate a point pt with each jc, such that all points lie on
a fixed line and the distance of point
pt from a fixed origin is jc,. It is easy to see that the
minimum-cost
on the line \342\200\224
namely,
spanning tree must connect each point to its neighbors
the tree is a chain.
The tree has two leaves, which
elements
are the maximum and minimum
of the sequence. If we know the minimum-cost spanning
tree, then we can find the sorted
order of the points in linear time as follows.
the points,
First, we find the minimum
among
how
that
to use
we
wish to
Sketches of
Solutionsto
Selected
439
Exercises
say xh The chain defined by the tree gives us the sorted order, and we can follow it in linear
time. Thus, we have a lower bound of Q(n \\ogn) for this problem under the decision
tree
for sorting was proved).
model (which is the model under which the lower bound
11
Chapter
11.3 O
(nk)
(and
11.11
function in
is a polynomial in w, but it is an exponential
is
not
a
this
as
be
as
w),
polynomial-time algorithm.
may
high
k. Sincek
is part of
the
input
We use a
reduction
undirected
11.14
The problem
subgraph in
is obviously
in NP,
time.
polynomial
since
we
can
We use a reduction
guess
from
3SAT.
check
\\
C2
its induced
Cnbc
1. If
every
consistent truth
assignment.
exists a subset
S with In + 1 vertices
that induces an acyclic
graph, then we claim
include
r and exactly two vertices from every clause. Indeed, S cannot
more than two vertices from one clause, since such vertices are connected
(and thus form
to x
a triangle). Also, since S includes
r, it cannot include two vertices that correspond
for E.
and x for some variable x. Therefore, it is possible to obtain a truth assignment
2. If there
that
S includes
from
3SAT.
Let C =(x+y+z) be a clause in an arbitrary 3SAT
11.16 We use a reduction
We
with
the
three clauses (the o,s are all new variables):
C
problem.
following
replace
(x + a\\ + a2), Cy + 0 3+tf4), and (z+a5+a6). In the l-in-3-SAT
problem, exactly one of
We want to guarantee that
in each of the three clauses above must be satisfied.
the variables
that
We do that by adding
more clauses that guarantee
at least one of jc, y, or z is satisfied.
one of a2, tf 4, and a6 is
and no more than
no more than one of a \\, o3, and a5 is satisfied,
are (fl,+fl3+fl7), (a^-\\-a5-\\-a^),
satisfied.
The
clauses
(a2 + a4+a\\0),
(a5 + a\\+a9),
(a4+a6+au), and
problem
We ^eave tne
verification
to the
reader.
(a6+a2+an)'
440
let
We must
of G.
cliques
d be the
Let
Selected Exercises
of Solutions to
Sketches
maximal
add
vertices
degree
of G
it is
number of vertices of G.
and connect each of
vertices
be the
d-d(v)
to G
and edges
if
For
them
even or
each
the
vertex
to v.
no new
that
such
maximal
v of G
with
cliques are
plus
degree
degree
formed.
and
otherwise,
we add
d{y)<d,
is
vertices
additional
= dn
dn-^d(Vj)
2\\E
This number
|.
is even since
we
d to
chose
be even.
that
Notice
vertices now have the same degree d, and that no new cliques were added
to only one vertex). The only
is that the new
each new vertex is connected
problem
more
vertices have a degree of 1. We can change their degrees to d without
introducing
in
between
them
the
We
divide
set
of
new
the
following
way.
cliques by adding edges
vertices to two equal sets. We then connect each vertex of one set to exactly d - 1 vertices
thus induce a bipartite graph, which
does not contain
of the other set. The new vertices
this
cliques of size > 2. We leave it to the reader to verify that it is possibleto construct
all the original
(since
bipartite graph
11.25
If
we
then
of
in
way.
could
could
we
vertices.
Therefore, a
polynomial-time
in
Section
11.2, but
algorithm
polynomial-time
for
algorithm
algorithms
polynomial-time
defined
the desired
the
Hamiltonian
for all NP
it
problems.
path
Notice
for
with specified
path by just
this
problem,
that
trying
all pairs
problem leads
which
this is
end vertices,
in
not a pure
turn
to
leads to
reduction
as
is sufficient.
11.29 We use a
reduction
from the vertex-cover problem. Let G =(V, E) and K be an arbitrary
instance of the vertex-cover
We construct a directed
G' by replacing every
problem.
graph
vertex v in G with two vertices v, and v2 connected by a directed edge (v,, v2). We replace
each edge (v, w) of G with two directed
(w2, Vj) and (v2, h^). We now claim that
edges,
G' contains
a feedback-edge
set of size K if and only if G contains a vertex cover of sizeK.
11.33We
sketch
a reduction from SAT. Let v be a vertex in G and let the edges adjacent
to v be
the edges of v,
(in the cyclic order) e,, e2,..., ek. The Eulerian tour defines a pairing
among
such that the consecutive edges in the tour used to enter
and leave v are paired. By the
conditions
of the problem, an edge et can be paired
either
with e,-_j or with ei+] (additions
and
subtractions
are done modulo k). The main observation
is that, if e,- is paired with e/+1,
then
the pairings
for all other edgesof v are fixed (ei+2 must be paired
with e/+3, and so on).
The same is true if e,- is paired with e,-_i. In other words, there are only two ways of pairing
the edges of v. We will associate
such
a vertex with each variable in the SAT expression,
that the value
of this variable in the truth assignment will correspond
to the way the pairing
is done for this vertex in the tour. We have to make sure that the truth assignment is
consistent
(i.e., the values of x and J are complementary), and that every clause is satisfied.
We will have one vertex
connect
vv for every variable jc, and one vertex v^ for J, and we will
them such that a pairing for vt forces a certain
values
the
truth
associate
for
We
Vx.
pairing
three vertices
Then, for each clause (x +y + z), we connect the corresponding
accordingly.
such that at least one of the pairings that is associated with one the variables being true must
be taken. We leave the details
to the reader. The planar
(which are not straightforward)
case, which is more complicated, is proved
in Bent and Manber [1987].
Chapter 12
12.13
We
simply
use
copies
participates: P,
and
they
copy
of Solutions to
Sketches
the
12.16
array
This
' to A
]
A [1..2'
To improve the
+ 1..2'].
be solved
can
problem
[T
in
exactly
Fig.
A.5.
the
same
use Brent's
efficiency,
as
way
441
Selected Exercises
lemma.
problem of
the
the
finding
maximum.
=8 is given
forw
in
B [j], then
find
12.25 We
to B.
be applied
We
first
since there
12.27
If
we
are no
associate
index in the
a processor
appropriate
12.32 We
cannot
each
record,
successor.
of its
then
in
step each
one
Each processornow
processorsif
root,
and
for
If the
RL and
height of
it down
sends
height
binary
a particular
but a delay
step,
the numbers
We use induction.
root, which then
algorithm
complete
idle
It is true
number separately.
for one processorcan
do not arrive there fast enough.
on each
concentrate
in
with
field
processor can
knows
write
its predecessor
its
in
list.
the
12.33
conflicts.
h,
the
it
is 2, then the
can
be added
left
leaf
each
more
sends
incurs at
delays at other
number
its number x]
to jc2. Suppose
that
we
to
have
the
an
and consider
It is
easy to
root's left
subtree the sum of the
trees.
tree
where
that
cause
RR be the
Figure A.5
subtree.
parallel-prefix
A straightforward
circuit.
solution is
to
solve
the
442
Sketches of
Solutionsto
Selected
Exercises
12.37 Suppose
which
k = \\
(again,
all index
n).
12.40
12.42
of Solutions to
Sketches
neighbors
implies
in the
left
subtree.
The
The
recurrence
of
assignment
SelectedExercises
443
= T(d) + 2, which
be clear from this
description.
subject of gossip as
to computer networks;
see,
The exercise
[1988] for a survey.
presents a relatively
simple
gossip problem. It can be solved by the pattern shown in Fig.
is the parallel
which
has
most
notable
A.6, called butterfly,
many other uses (the
=
shows
the
for
n
A.6
solution
8.
of
the
FFT).
Figure
computation
for example
Figure
literature
on the
Hedetniemi,
A.6
and
Hedetniemi,
The solution
of the
gossip
it
is related
Liestman
problem
n =8.
BIBLIOGRAPHY
Add'son-Vel'skii G. M.,
Aho
A.
\"An
of
the organization
for
algorithm
A.
Aho
The
and E. Szemeredi,\"An
M., J. Komlos,
ACM Symposium on Theory of Computing,
Parallel
Aleliunas
Sorting
Almasi G.
S., and
RedwoodCity,
and
D.,
Boston
Academic
Algorithms,
sorting
logn)
0(n
Addison-
Algorithms,
New York,
Press,
A.
G. Valiant,
Journal
matchings,\"
Parallel
Highly
1-9.
1985.
ACM
on
Symposium
pp. 60-72.
1982),
Benjamin/Cummings,
Computing,
1989.
CA,
L.
and
Gottlieb,
First
Annual
15th
network,\"
1983), pp.
(April
\"Randomized
communication,\"
parallel
of Distributed Computing, Ottawa (August
R.,
Principles
Angluin
of Computer
Reading,
Wesley,
S. G.,
and Analysis
Design
MA, 1974.
Reading,
Addison-Wesley,
V., J.
D. Ullman,
Hopcroft, and J.
J. E.
V.,
Algorithms,
Akl
Dokl., 3
Aho
Ajtai
M. Landis,
Y.
and
Math.
Soviet
information,\"
for Hamiltonian
\"Fast probabilistic algorithms
and System Sciences, 18 (April
of Computer
circuits
1979),
pp.
155-193.
Apostolico A.,
Z.
and
1985.
York,
E.
V. L.,
Arlazarov
of
construction
1970),
(May
pp.
M. D.,
Atkinson
multiplication,\"
Attalah
Combinatorial
Galil,
M.
technique
J.,
A.
the
closure of
transitive
I.
and
directed
A.
\"On economical
Faradzev,
graph,\"
Soviet
Math
Dokl, 11
1209-1210.
and
N.
\"A
Santoro,
Processing
Information
R. Cole,
for
A. Kronrod,
M.
Dinic,
and
designing
M.
T.
parallel
practical
Letters,
Goodrich,
algorithms,\"
for
algorithm
29 (September
\"Cascading
28th
Boolean
matrix
divide-and-conquer:
Annual
Symposium
on
445
446
Bibliography
S.
L., and
Auslander
of Computer
Foundations
Mech.,10(May
V.
\"On
Parter,
1961),
Communications
\"Sums of
15
divisors,perfect
(November
Schwab,
\"A
of the ACM,
20 (May
B.
and
the
in
and
J. Math,
plane,\"
pp. 517-523.
J. Shallit,
and
Bach E., G. Miller,
SIAM Journal on Computing,
Baer J.-L.,
graphs
imbedding
151-160.
and
numbers
of tree-balancing
comparison
factoring,\"
1143-1154.
1986), pp.
algorithms,\"
pp. 322-330.
1977),
their applications,\"
and
networks
Bates
L.
R.
and
\"Proofs as
Constable,
BavelZ., Math
Virginia,
Beame P. W.,
S. A.
Beckenbach
E.,
Bellman,
House,
R. E., Dynamic
S.
Bent
R.
and
Random
Library,
Bellman
New
An Introduction
Princeton
the
\"Finding
on Theory of
Symposium
pp. 113-136.
for division
circuits
Reston,
Company,
Publishing
1986),
and
related
pp. 994-1003.
to Inequalities,
New
Mathematical
1961.
York,
Programming,
and J. John,
W.,
ACM
Science, Reston
on
Transactions
ACM
1985),
problems,\"
32nd
AFIPS
Proceedings
programs,\"
7 (January
Computer
for
Companion
1982.
in
307-314.
median
University
In comparisons,\"
requires
Computing,
RI
Providence,
17th
(May
Annual
1985), pp.
213-216.
Bent
Bentley
J. L.,
Eulerian
Reading,
circuits,\"
Discrete
1986.
MA,
F. P. Preparata, \"Approximation
and
Bentley J. L., M.G. Faust,
algorithms
hulls,\" Communication of the ACM, 25 (January 1982), pp. 64-68.
Bentley
J.
L.,
conquer
Bentley
J. L.,
1(1988),
D. Haken,
recurrences,\"
and B.
and
J.
B. Saxe,
SIGACT News,
W. Kernighan,
\"Tools
Applied
\"A general
method for
for printing
indexes,\" Electronic
solving
for convex
divide-and-
(Fall 1980),pp.36-44.
Publishing,
pp. 3-17.
for
Bentley J. L., and T. Ottmann,
\"Algorithms
intersections,\" IEEE Transactionson Computers,
reporting
C-28
and
(Sept.
counting
geometric
Bibliography 447
Berge C,
The
of Graphs
Theory
and
New
Sons,
York,
1962.
P., and
J.
Bitner
Prentice-Hall,
Methods,
R.,
M.
E.
and
Communications of the
Blelloch
Parallel
Processing,
parallel operations,\"1987International
R. W. Floyd,
selection,\"Journal
V. R.
Pratt, R.
L. Rivest,
and System
of Computer
B.,
Bollobas
Bondy J. A.,
S. R.
U.
and
on
Conference
and
An Introductory
Theory:
Bollobas
Graph
techniques,\"
pp. 355-362.
1987),
(August
Blum M.,
programming
1975), pp. 651-656.
18 (November
ACM,
Numerical
Computation,
1989.
\"Backtrack
Reingold,
as primitive
\"Scans
G.,
N. Tsitsiklis,
J.
1973.
London,
Holland,
Press,
1986.
Cambridge,
New York,
Elsevier,
Applications,
1979.
York,
1976.
time
and
Borodin A., \"On relating
6
1
977),
(December
Computing,
pp.
J. E.
A., and
Borodin
computation,\"
Borodin A.,
and
Problems,
Boyer R.
the
Brelaz
D.,
Burge
W.
System
Fourier
The Fast
the
Transform,
Combinatorics,
A., Introductory
Recursive
H.,
string
Complexity of Algebraic
New
York,
Communications
algorithm,\"
searching
Numeric
and
1975.
of
vertices
graph,\" Communications
of a
of the
22 (April
R.
Brualdi
to color
methods
\"New
\"A fast
30 (1985),
Sciences,
20 (October
E. O.,
Brigham
S. Moore,
J.
on
parallel models of
pp. 130-145.
and sorting on
merging
Computer and
I. Munro,
The Computational
Elsevier Computer ScienceLibrary,
S., and
ACM,
ACM,
of
Journal
SI AM
depth,\"
733-744.
\"Routing,
Hopcroft,
Journal
to size and
space
Programming
Prentice-Hall,
North
Holland,
Techniques,
Englewood
Cliffs, NJ,
New
1977.
York,
1974.
1975.
Bussey
24
W. H.,
(1917),
Capobianco M.,
J. L.,
and
mathematical
induction,\"
American Mathematical
Monthly,
199-207.
and
J.
C. Molluzzo,
New
North-Holland,
Carter
of
\"Origin
pp.
M.
N.
York,
Examples and
Counterexamplesin
Graph
Theory,
1978.
Wegman,
\"Universal classes
of
hash
functions,\"
Journal
of
Bibliography
448
ACM,
17
M., and
K.
Chandy
\"An
S. Kapur,
algorithm
(January 1970), pp. 78-86.
143-154.
for convex
and S.
D.
Chand
1979), pp.
18 (April
Sciences,
polytopes,\" Journal
A Foundation,
Design:
the
of
Addison-Wesley,
Chartrand
and
D.,
Computing,
Group,
minimum
R. E. Tarjan, \"Finding
5 (December 1976),pp.724-742.
S.T. Klein,
Fraenkel,
of
Proceedings
manipulation,\"
Montreal, Canada(1985),pp.
Christofides
&
Wadsworth
A.S.
Y.,
Choueka
International
Wadsworth
L. Lesniak,
and
G.,
Brooks/Cole,
Cheriton
Models,
CA, 1977.
Belmont,
Chartrand
Mathematical
as
Graphs
N., Graph
Y.
and
the
\"Huffman
Perl,
on
Journal
coding
bit-
without
ACM-S1G1R
Annual
Eighth
SIAM
trees,\"
spanning
Conference,
122-130.
Theory:
An
Press, London,
Academic
Approach,
Algorithmic
1975.
Christofides
\"Worst-case
N.,
problem,\" Technical
Cohen J.,
M.
and
down
\"Slowing
Annual
1984),
Foundations
on
Symposium
Industrial
Administration,
of Strassen's
fast
multiplication
6 (1976),pp.341-355.
networks
sorting
of
School
salesman
the traveling
PA, 1976.
the implementation
\"On
Roth,
Acta lnformatica,
algorithm,\"
ColeR.,
Pittsburgh,
for
heuristic
new
Graduate
Report,
University,
Carnegie-Mellon
of a
analysis
to
faster
obtain
of Computer
sorting
25th
algorithms,\"
(October
pp. 255-259.
merge sort,\"
SI AM
on Computing, 17
Journal
1988),
(August
pp.
770-785.
micro
Cook S. A.,
ACM
\"The
complexity
of theorem
Cook S. A.,
\"An
26 (June
Cook
overview
New
of computational
1983),pp.400-408.
on
the
Symposium
proving
York
proceudres,\"
(1971),
Annual
Third
ACM
pp. 151-158.
of the
complexity,\" Communications
time of parallel RAMs
on Theory of Computing,
to
compute
San
ACM,
simple
Francisco
Bibliography 449
CooleyJ. M.,and J. W.
Fourier series,\"Math.
Tuckey,
Comp.,
Annual
19th
of complex
calculation
progressions,\"
1987), pp.
(May
1-6.
Culberson
on Theory
Symposium
G.
Dantzig
Princeton, NJ,
Dekel
route
and
Programming
10
with
C. Pang,
14
M. E.
and
IT-22
and A.
S. E.,
New York,
J.,
Science,
Computer
and annotation,\"
Networks,
two
IEEE Transactions on
in cryptography,\"
directions
\"New
Boston, 1983.
Birkhauser,
problems
of Programming,
with
Prentice-Hall,
and
Engineering
Numerische
graphs,\"
E. W.,
Dreyfus
Englewood
Taxonomy
algorithms:
(June
\"A note on
W.,
Mathematik,
Dijkstra
SI AM
algorithms,\"
graph
1974.
of Programs,
Hellman,
Theory,
Information
E.
to
Applications
and
657-675.
Prentice-Hall,
\"Shortest-path
Dijkstra
Press,
University
pp. 275-323.
(1984),
Diffie W.,
matrix
1981), pp.
Science, 6
Management
Princeton
Extensions,
\"Parallel
Sahni,
(November
Programming,
Theory
Graph
S.
and
DenardoE. V.,Dynamic
Deo
ACM
Annual
a newtwork,\"
through
17th
trees,\"
RI (May
1963.
E., D. Nassimi,
Journal on Computing,
Deo N.,
search
binary
187-190.
B., Linear
G.
of
in
Providence,
Computing,
B.,
(1960),pp.
Dantzig
of updates
effects
\"The
J.,
B.
M. Law, The
1977.
Durian,
and
Art
the
Theory of
linear time
\"Unstable
\"Computing
Eulerian
mode,\"
trails,\"
Information
Cliffs, NJ,
Englewood
Theoretical
Computer
Dynamic Programming,
0(1) spacemerging,\"
Processing
Letters,
1976.
Science,
Academic
The
Computer
28 (June
1988),
pp. 93-97.
Edelsbrunner
Edmonds
network
H.,
J., and
flow
Algorithms
R. M. Karp,
problems,\"
in Combinatorial
\"Theoretical
Journal
of
the
Geometry, Springer-verlag,Berlin,
1987.
in algorithmic efficiency
improvements
19
ACM,
(1972), pp. 248-264.
for
450 Bibliography
Elliott
D.
F., and K. R.
Academic Press, New
J. L.,
Eppinger
\"An
A.
and
P.,
Szekers,
P., and
in geometry,\"
problem
J. Spencer,Probabilistic
in Combinatorics.
Methods
Even
S., Graph
Even
S.,
and
space,\"
R. E. Tarjan,
Journal of the
\"A
Press, Rockville,
Science
Computer
Algorithms,
combinatorial
in
polynomial
pp. 710-719.
23 (1976),
ACM,
MD, 1979.
is complete
which
problem
Commentarii
pertinentis,\"
128-140.
\342\200\224
a fast
J. Nievergelt,
R.,
Fagin
Compositio
463-470.
1974.
York,
Euler
search trees,\"
in binary
deletion
and
(September 1983),pp.663-669.
combinatorial
\"A
Mathematica, 2 (1935),pp.
Erdos
of insertion
study
ACM, 26
of the
Applications,
Analyses,
Algorithms
Transforms:
1982.
empirical
Communications
Erdos
Rao, Fast
York,
access
Systems,
of interconnection
\"A survey
T.,
networks,\"
14 (December
Computer,
1981), pp.
12-27.
Fich
F.
E., \"New
Theory of Computing,
Boston
FischerM.J., \"Efficiency
R. E. Miller and
Fischer M. J., and
of
J. W.
Fischer
Fischer M.
and
J.,
M. O.
in Complexity
arithmetic,\"
J.,
and
S.
L.
3 (1982),
Algorithms,
Rabin,
Floyd
R.
American
W.,
97:
Shortest
Mathematical
meanings
Society
and
majority
analysis of
August
transitive
Theory,
Ed.,
Computations,
153-168.
pp.
East
closure,\"
MI
Lansing,
of
complexity
R. M. Karp
a
1972,
York,
Automata
\"Super-exponential
Salzberg,
\"Finding
pp. 375-379.
and
Complexity
multiplication
and
of Computation,
'Assigning
matrix
Switching
in
algorithms,\"
\"Boolean
R. Meyer,
A.
ACM Symposium on
Annual
100-109.
1983), pp.
equivalence
Thatcher,
15th
circuits,\"
prefix
(April
Presburger
1974.
SIAM-AMS,
among
algorithms
Journal
n votes,\"
and
data
of
structures,\"
1987.
paths,\" Communications
of the
ACM,
5 (June
Mathematics,
Bibliography 451
the
M.
J.,
L.
systems,\" Proceedings of
the
54
IEEE,
1901-1909.
The
flow theory,\"
\"Network
R.,
Communication of
for selection,\"
bounds
computing
high-speed
\"Very
(1966), pp.
Ford
\"Expectedtime
165-172.
1975), pp.
18 (March
ACM,
Flynn
R. L. Rivest,
W., and
R.
Floyd
Rand
Monica, CA
Santa
P-293,
Corporation
(1956).
D. R.
R., and
L.
Ford
L.
Ford
Ford L. R.,
M.
S.
and
66 (1959),
Monthly,
J.
and
S.,
problem,\"
Franco J.,
Hopcroft,
the
\"On
problem,\"
Frederickson
Rabin's
on
note
\"A
nearest-neighbor
M.
Journal
optimization,\"
H.
on
Galil, T.
spanning
(1986),pp.
for selection in
algorithms
of
the
for
algorithms
Journal
sets,\"
34 (July
ACM,
in
searching
satisfiability
of Computer
and
uses
R.
undirected
in
network
E. Tarjan,
for
maximum
\"Efficient
directed
and
109-122.
their
and
and
Gabow
Galil Z.,
the
H. Spencer,and
trees
Walker,
and
Techniques
graphs,\"
N., Z.
H.
minimum
23 (August
D. W.
\"An efficient
N.,
matching
of
Letters,
E. Tarjan, \"Fibonacciheaps
and R.
L.,
performance
Processing
37 (December
Sciences,
Fredman
Gabow
probabilistic
\"Distributed
algorithm,\"
Letters,
Information
G.,
System
Gabow
Mathematical
American
pp. 387-389.
M. A.
C,
Press,
University
S. W. Otto, J. K. Salmon,
and
Johnson, G. A. Lyzenga,
1: General
Solving Problems on Concurrent Processors,Volume
Cliffs, NJ, 1988.
Regular Problems,Prentice-Hall,
Englewood
G.
Journal
Canadian
network,\"
Princeton
Networks,
\"A tournament
Johnson,
Processing
Information
Fox
in
Flows
Fulkerson,
through
NJ, 1962.
Princeton,
Fortune
D. R.
and
R.,
flow
\"Maximal
Fulkerson,
8 (1956),pp.399-404.
of Mathematics,
improving
algorithm,\"
the
worst
case
Communications
time
running
of
the
optimization
of
ACM,
22
problems,\"
the
Boyer-Moore string
1979), pp.
(September
505-508.
Galil
Z.,
\"Efficient
algorithms
Surveys, 18 (March
for
finding
1986),pp.23-38.
maximum
matching
in graphs,\" Computing
Bibliography
452
Galler
B. A., and
of the ACM,
\"An improved
J. Fishcer,
M.
7 (1964),pp.301-303.
equivalence algorithm,\"
D. P.
E. F.,
Gehringer
GibbonsA., Algorithmic
W. Rytter,
and
A.,
Gibbons
Parallel
Efficient
of
Theory
Parallel
The
Processing:
Cm
Cambridge
Theory,
Graph
Segall,
the
1979.
1987.
Digital
Experience,
Z.
and
Siewiorek,
to
A Guide
R., and
M.
Garey
Communications
University Press,
Cambridge
Algorithms,
Cambridge,1988.
A.
Goldberg
V.,
Journal of
the
and R.
E. Tarjan,
ACM,
35 (October
new
\"A
all
S., and J. Killian, \"Almost
ACM Symposium on Theory of Computing,
maximal-flow problem,\"
can be
primes
Berkeley
(May
18th Annual
certified,\"
quickly
pp. 316-329.
1986),
programming,\" Journal
\"Backtrack
L. Baumert,
to the
Goldwasser
GolombS.,and
approach
of the
ACM,
12 (1965),
pp. 516-524.
GolovinaL. I.,and
Golumbic
M.,
I. M.
Induction
Yaglom,
in
Geometry
(translated
from Russian),
D. C.
1963.
Boston,
Heath,
and Perfect
Theory
Graph
Algorithmic
1980.
York,
M., and M.
Gondran
Minoux,
Graphs
and
Algorithms,
Sons, New
York,
1984.
Gonnet
G. H., Handbookof
Graham
R. L.,
R.
Processign
L., D.
Reading, MA,
F., Pulse
Gray
tree machine,\"
D.
Boston,
E.
H., and
U.
Manber,
IEEE
planar
set,\"
D. E.
Patent
US
2632058
(March 1953).
Transactions
Mathematics
Knuth,
for
the Analysis
of Algorithms,
on the
359-362.
Birkhauser,
1982.
GrotschelM.,L. Lovasz,
in
of a
O. Patashnik,
and
Knuth,
hull
1989.
Code Communication,
GreenbergA., and
Greene
Addison-Wesley, Reading,
efficient
\"An
Information
Graham
Data Structures,
and
Algorithms
1984.
MA,
combinatorial
Programming,
and
Springer-Verlag,
A. Schrijver,
optimization,\"
New York,
1981.
its consequences
\"The ellipsoid method
and
1 (1981) pp. 169-197.
Combinatories,
453
Bibliography
L. J.,
Guibas
Gusfield
D.,
L.
and
Pitt,
D. Wood,
Combinatorial
M.,
10 (1935),pp.
Coding and
R. W.,
Hedetniemi
gossiping
and
Hillis
and
K., J.
sorting,\"
Connection
G.
L. Steele,
Practice
Hoffman C.
P.
broadcastingand
with
to
applications
pp. 13-28.
Cambridge, MA,
1985.
of the
Communications
Schorn,
ACM,
pair
problem
Letters,
for
algorithm
of the
ACM, 18
common
maximal
computing
(June 1975),pp.341-343.
results
\"Recent
S.,
in
Time
Wraps,
String
1983.
M., Group
Computer
Science,
Probabilistic
J.-W.,
Hong,
trees
1962),
linear-space
Prentice-Hall,
on the
of common-subsequence
complexity
and
Edits, and Macromolecules: The Theory
D.
Sankoff
and
J.
B.
AddisonKruskal,
eds.,
Comparison,
of Sequence
D.
problems,\"
Hofri M.,
Edition,
1170-1183.
Processing
\"A
S.,
9 (January
MIT Press,
Machine,
subsequences,\" Communications
Hirschberg
Society,
1969.
certain
of
properties
of the ACM,
Journal
Information
D.
Hirschberg
Hoare
Second
Nievergelt, and
elegantly,\"
1986.
York,
Mathematical
the London
of
New
1986.
29 (December1986),pp.
Hinrichs
by divide-and-conquer,\"
and Sons,
Wiley
Theory,
Information
searching
cover,\"
pp. 771-775.
Hibbard T.
node
26-30.
Englewood Cliffs,NJ,
Harary
John
Edition,
Journal
representatives of subsets,\"
Hamming
Second
Theory,
intersections
(July 1984),
C-33
Computers,
for
algorithms
pp. 291-294.
1986),
rectangles
\"Finding
IEEE Transactionson
Hall
Letters,22 (May
Boyer-Moore
pp. 672-682.
9 (1980),
approximation
\"Equivalent
Information Processing
of the
linearity
SI AM Journal
algorithm,\"
searching
string
proof of the
on Computing,
\"A new
M. Odlyzko,
A.
and
Theoretic
and Graph
Algorithms
Computation:
of Algorithms,
Computability,
(1962),
Journal,
York,
10-15.
Isomorphism, LectureNotes
and
New
Duality,
York,
1987.
Pitman, London,
1986.
Hopcroft
J. E.,
graphs,\"
algorithm
2 (December
in
1982.
Springer-Verlag,
Similarity
pp.
in
bipartite
454 Bibliography
J.
Hopcroft
SIAM
R. E. Tarjan,
\"Efficient
J.
J. D.
and
E.,
Ullman,
\"Divide
A. Zorat
and
E.,
Transactionson
C,
of
the
21
ACM,
of Computer
and
Programming
Integer
and
for
Network
Science
Computer
Algorithms,
conquer
(June 1983),
C-32
Computers,
Journal on
294-303.
T.
components,\"
SI AM
algorithms,\"
\"Set-merging
Horowitz
Hu
Journal
testing,\"
planarity
1973), pp.
2 (December
Computing,
Horowitz
into triconnected
graph
1973),pp. 135-158.
(October
Hopcroft
\"Dividing
2 (September
on Computing,
E., and
J.
Hopcroft
E. Tarjan,
and R.
E.,
Journal
IEEE
processing,\"
parallel
pp. 582-585.
Flows,
MA,
Reading,
Addison-Wesley,
1969.
and M. A. Langston, \"Practical
B. C,
Huang
in-place merging,\"
Huffman
\"A method
A.,
Proceedingsof
J.
Hunt
G.
and T.
W.,
for
\"A
Szymanski,
subsequences,\"
New
Incerpi J.,
A.,
R.
\"Practical
Sedgewick,
T.
and
Parallel
and
Processing,
minimum
1978), pp.
Kailath,
IEEE
multiplication,\"
\"Finding
7 (November
Jagadish H. V.,
algorithm
McGraw-
Processing
pp. 37-43.
M. Rodeh,
Computing,
fast
26 (1987)
and
codes,\"
redundancy
1984.
York,
and
Letters,
Itai
F. A.
and
K.,
Hill,
minimum
of the ACM,
Communications
Hwang
of
construction
the
IRE, 40 (September
the
of the
Communications
in
SI AM
graph,\"
Journal on
413-423.
\"A
efficient
new
of
family
on Computers,
Transactions
C-38
arrays
for
matrix
1989),
(January
pp.
149-155.
Johnson D.
S., A.
B., \"Efficient
D.
the
24 (January
ACM,
Jones D.
\"An
W.,
A.,
Akad.
and
Nauk
SSSR,
of
\"Multiplication
the
in
paths
of
comparison
empirical
Yu. Ofman,
in Sow
translated
Communications
implementations,\"
Karatsuba
case
SIAM
on Computing, 3 (1974),pp.299-325.
Journal
Johnson
J. D.
Demers,
bounds
performance
ACM,
of
networks,\"
sparse
priority-queue
and
Journal of
event-set
29 (April
multidigit
numbers
on
appeared
automata,\"
in Dokl.
Bibliography 455
\342\200\224
E. Upfal, \"Parallel hashing
An efficient
18th
Annual
ACM Symposium on Theory of
R., and
A.
Karlin
memory,\"
\"A
N.,
Combinatories,
Karmarkar
polynomial time
4 (1984),pp.373-395.
new
R. M. Karp,
and
N.,
Computer Science,(November
*
R. M.,
R.
Karp
M. Saks,
M.,
bound
Toronto
M.,
R.
L.
K.
N.,
B.
L. Minty,
linear
in
algorithm
\"An
14 (May
The
D. E.,
simplex
York (1972),
New
R. Manber,
Art of Computer
Art
of
\"Estimating
29
Computation,
Knuth
D. E.,
\"Big
(1975),
omicron
1976),pp. 18-24.
Knuth
Computer
MA,
Reading,
E.,
of
New
of
the
algorithms,\" IBM
programming,\"
pp.
algorithm
for
sink-finding,\"
109-111.
algorithm?\"
in Inequalities
111,O.
pp. 159-175.
and
Addison-Wesley,
Knuth D.
Press,
1987),pp.249-260.
optimal
1982),
Volume
Programming,
Complexity
Plenum
Communications
pattern-matching
31 (March
and
its Applications,
E., The
randomness,\"
randomized
Development,
Press,
and
Smith-Thomas,
Shisha,ed.,Academic
Knuth
in
19-28.
Processing Letters,
G.
of
191-194.
and
Information
Foundations
\"On a
Symposium
\"Efficient
and
\"A polynomial
G.,
problems,\"
W. Thatcher, eds.,
J.
and
complexity,
Research
20 (1979),pp.
King
pp.
one-
the
M. O. Rabin,
M., and
Journal of
Khachian
Annual
\"Combinatorics,
29 (February
ACM,
Karp
1986),
E. Miller
Wigderson,
27th
(October
Karp R.
A.
and
procedures,\"
on
Symposium
combinatorial
among
Computer Computations, R.
York (1972), pp. 85-103.
for
scheme
approximation
Annual
programming,\"
pp. 312-320.
1982),
deducibilities
linear
for
algorithm
efficient
\"An
problem,\" 23th
bin packing
dimensional
Karp
(May
Berkeley
160-168.
1986), pp.
Karmarkar
of shared
implementation
Computing,
the
Programming,
1973b.
efficiency
II Fundamental
Algorithms,
1973a.
Volume 31 Sorting
of backtrack
and Searching,
programs,\" Mathematics of
pp. 121-136.
and
big omega
and
big
theta,\"
Volume
D. E., The Art of Computer Programming,
Second edition, Addison-Wesley, Reading, MA, 1981.
S1GACT News,
21 Seminumerical
(April-June
Algorithms,
Bibliography
456
Knuth D. E.,
163-180.
Huffman
\"Dynamic
D. E., J. H. Morris,
Journal on Computing,
Kronrod, M. A.,
J.
Kruskal
\"An
Nauk.
(1985),
pp.
SIAM
in strings,\"
of
without a field
pp. 1256-1258.
graph
the
and
Mathematical
American
the
Algorithms,
matching
pattern
of a
subtree
spanning
Proceedings of
problem,\"
186 (1969),
SSSR,
of
323-350.
algorithm
ordering
optimal
B.,
1977), pp.
6 (June
Dok. Akad.
Russian),
Pratt, \"Fast
V. R.
and
Knuth
Journal
coding,\"
(in
operation,\"
salesman
traveling
71 (1956),
Society,
pp.
48-50.
C.
Kruskal
M.
L. Rudolph, and
on Computers, C-34
P.,
Transactions
C.
Kruskal
L. Rudolph,
P.,
1986
problems,\"
pp. 869-876.
M.
and
International
\"Efficient
Snir,
parallel algorithms
Processing,
for
on Parallel
Conference
H.
T.,
\"Let's
H.
Kung
T.,
and
U.
Ladner
time
of polynomial
Fischer,
\"Parallel
prefix
L.,
Combinatorial
New
York,
A., S.
Even, and
Theory of Graphs:
intersection
and
of the
Journal
reducibility,\"
ACM,
of the
Journal
ACM, 27
Optimization:
A.
H. G.
Wiley
and
Networks
International
Rinnooy Kan,
I. Cederbaum,
An
Mathematical
Cambridge
Discovery,
Matroids,
Holt,
Rinehart &
1976.
L., J. K. Lenstra,
SalesmanProblem,
John
Lawler E.
Lempel
for set
267-282.
831-838.
in
eds., Addison-Wesley,
Conway,
computation,\"
I., Proofs
Winston,
L.
and
Mead
processor arrays,\"
155-171.
pp.
1975),
E., and M. J.
(October1980),pp.
Lawler
for VLSI
\"Algorithms
Systems, C.
structure
the
Caltech
the
pp. 37-46.
1982),
Ladner R.
Lakatos
15 (January
Manber,
R. E., \"On
22 (January
of
Proceedings
systems,\"
sparse
pp. 271-292.
(1980),
its analysis,\"
of
pp. 65-90.
Leiserson,
to VLSI
Reading, MA,
Kurtz T.,
E.
C.
and
Introduction
VLSI
for
algorithms
T., \"Why
Kung H.
design
on VLSI, (1979),
Conference
graph
1986),
(August
IEEE
York,
and
D.
B. Shmoys,
The Traveling
1985.
testing
of graphs,\"
457
Bibliography
L. A.,
Levin
\"Universal
115-116.
English
Problems
in
Problemy
problems,\"
sorting
translation
of
pp.265-266.
Lin
B. W.
and
S.,
problem,\"
G. J.,
Lipovski
Wiley
Parallel
Malek,
New
the
salesman
traveling
498-516.
Theory and
Computing:
Comparisons,John
1987.
York,
Elements of Algebra
J. D.,
Lipson
M.
and
& Sons,
for
heuristic
effective
\"An
Kernighan,
and
Computing,
Algebraic
Park,CA, 1981.
R.
Lipton
Litwin
of the
\"Linear
W.,
Sixth
the
D. Lopresti,
and
J.,
Proceedings
\"A
hashing:
Conference
A new
on Very
on
L.,
colorings of
and
\"Coverings
Southeastern Conferenceon
Mathematica,
LovaszL.,Combinatorial
Lovasz
M. D.
and
L.,
Problems
and
S., \"Sometechniques
(December1980),pp. 419-436.
for
time,\"
Lynch
North
G.
Manacher
Manber
U.,
A. Plotkin,
Information
K.,
\"On
Letters,
dynamic
maintaining
15
U.,
and
(1988),
tree
spanning
as
Reinhold,
path-finding
is not
L. McVoy,
pp. 263-272.
\"Efficient
in a
information
1986), pp.
(November
to design
Manber U., \"Usinginduction
1988), pp. 1300-1313.
(November
18
Nostrand
optimal,\" Journal
of the
Journal on Computing,
Manber
polynomial
pp. 42-65.
Van
26 (January
algorithm
sorting
tested in
can be
Applications,
\"Minimum-cost
Processing
Computing Surveys, 12
25 (1982),
Sciences,
System
\"The Ford-Johnson
26 (July
ACM,
S.
and
M.,
problem,\"
valence
bounded
Maggs B.
Amsterdam, 1986.
Holland,
recurrences,\"
solving
of
1979.
1882.
Paris,
Luks
Exercises,
Theory,
Mathematiques,
Lueker G.
of the Fourth
and Computing, Utilitas
Theory,
Graph
pp. 3-12.
Plummer, Matching
LucasE.,Recreations
363-376.
Proceedings
hypergraphs,\"
Combinatorics,
(1973),
Winnipeg
comparison,\"
string
(\\9S5\\ pp.
Databases,
Large
212-223.
Lovasz
rapid
VLSI,
for file
tool
for
array
systolic
algorithms,\"
storage
concurrent
environment,\" SIAM
1130-1142.
Communications
of nonadaptive
of
the
ACM,
31
Bibliography
458
E.
\"Suffix
Myers,
arrays:
Z., Lectures on
Manna
Manna Z.,
R.
and
the
Reading,
MA
Sankoff
D.
N.,
tree construction
suffix
Journal
Mehlhorn
K.,
Menger
A.
Meyer
\"Zur
requires
Automata
Theory,
A.,
\"A
and
J.,
Moffat A.,
D. Gries,
and
T.
and
pp.
S. S.
all
Myers
\"Problems
ed.,
expression
regular
on
Switching
with
and
Marcel Dekker,New
E. W.,
related
\"An
O(ND)
subsequence
problem,\"
elements,\"
Science
of Computer
matrix
Computing
diagrams,\"
Technical Report
York,
algorithm
(June
Surveys,
in Computational
multiplication,\"
Press, New
difference
time
expected
path algorithm with
16 (December 1987),pp. 1023-1031.
shortest
parallel algorithms,\"
Science, Cornell University
of Computer
to
stuttering
of
\"Discussion
Iyengar,
Algorithmics
pair
on Computing,
Journal
for
71-75.
repeated
Moret
Rustin,
and
143-152.
\"An
Takaoka,
SIAM
longest
26 (1987),pp.
\"Finding
2 (1982),
TR-86-759,Department
Munro
of serial
Searching
Symposium
and Graphs,
Networks
for the
technique
halving
0(n2\\ogn)\"
A.,
design
852-865.
10 (1927),pp.95-115.
Fund. Math.,
for
Processing Letters,
Programming,
Moitra
the
in
of
125-129.
pp.
Algorithms
Optimization
Information
Misra
(1972),
Journal
algorithm,\"
1978.
York,
Mirzaian
in
1984.
R., and
of
quickly,\"
Multi-Dimensional
3:
Berlin,
Kurventheorie,\"
allgeminen
squaring
Minieka E.,
B.
J.
and
distances
string-edit
parallel
computation algorithms
the
30 (October1983),pp.
ACM,
of
\"Applying
algorithms,\"
Science
paradigm,\"
pp. 337-349.
(1983),
1980.
PA,
Macromolecules:
and
Edits,
String
CBMS-NSF Regional
search
binary
\"How to compute
S. Paterson,
M.
1990).
pp. 37-83.
9 (1987)
Comparison,
Sequence
origin of
'The
Waldinger,
Computer Programming,
searches,\"
string
(January
Programming,
of Computer
Mathematics, SIAM, Philadelphia,
Logic
in Applied
series
Conference
for on-line
method
new
Discrete Algorithms
1986).
14 (December
Complexity, R.
1971.
and
its
variations,\"
Algorithmica,
Bibliography 459
E.
Myers
\"A four-Russians
W.,
Technical
October
pattern
expression
regular
matching,\"
of
Arizona,
1988.
for
algorithm
#88-34,
Report
on
\"Note
Planar
Hamiltonian
of Discrete
Annals
Algorithms,
1988.
Netherlands,
Holland,
circuit,\"
American Mathematical
Random
House,
67
Monthly,
(1960),
p.
55.
Uses,
V.,
How
is not
Arbor,
129, Springer-Verlag,Berlin,
I., Parallel
M.
C,
Sons,
York,
A.
\342\200\224
H.
and
Communications of the
G.,
Polya
Induction
Framework,
Transformation
Search
Intelligent
for Computer
Strategies
Problem Solving,
MA, 1984.
Reading,
Itai,
Recursion
& Sons,
Wiley
and
Algorithms
1988.
Addison-Wesley,
Perl Y.,
Design:
Algorithm
New
Volume
Science,
Computer
1984.
ComplexityTheory, John
J., Heuristics
Pearl
in
K. Steiglitz, CombinatorialOptimization:
Prentice-Hall, Englewood Cliffs,NJ, 1982.
Complexity,
Paull
Lecture Notes
on Foundations
Symposium
pp. 166-176.
C. H., and
Papadimitriou
Parberry
Matrices Faster,
to Multiply
Avni,
ACM,
and Analogy
in
\342\200\224
A
search
\"Interpolation
21 (July
search,\"
loglog/V
Mathematics,
University
Press, Princeton,
NJ, 1954.
to
It, Second
Solve
Edition,
Princeton
University
Press,
Princeton, NJ,
1957.
Discovery,
Combined
Edition,
Sons, New
York,
1981.
Polya
G.,
Polya
and
G. Szego,
und Lehrsatze
in Analysis
I, Berlin,
Volume
I, Springer-Verlag,
Berlin,
1972.
and M. I. Shamos,Computational
New
York, 1985.
Verlag,
F. P.,
Preparata
Prim
R.
C,
\"Shortest
TechnicalJournal,
connection
36
(1957),
networks
p. 1389.
Geometry:
and
some
An Introduction,
generalizations,\"
SpringerBell System
Bibliography
460
York,
New
The Analysis
and C. A. Brown,
1985a.
P. W.,
Purdom
Quinn
1985b), pp.
Parallel
for
Algorithms
average time,\"
polynomial
943-953.
McGraw-Hill, New
Computers,
1987.
York,
M.
Quinn
Efficient
J., Designing
and
rule
literal
& Winston,
Holt, Rinehart
of Algorithms,
J.,
N.
and
\"Parallel
Deo,
graph
16
Surveys,
Computing
algorithms,\"
Rabin
and
\"Probabilistic
O.,
New
algorithms,\"
Traub, ed.,
J. F.
Directions,
Science,Toronto
A. G.,
Ranade
M.
R.
and
Processing,MIT
Reif J. H.,and
L. G.
ACM, 34
of the
Rivest
J. Hansen,
W.
28th
memory,\"
Multicomputer Networks:
MA, 1987.
Message-BasedParallel
size networks,\"
Journal
and public-key
1983.
18 (Summer
SIGACT News,
bibliography,\"
on
Symposium
pp. 185-194.
\342\200\224
A
sorting
Annual
1987),
(October
Valiant,
D., \"Parallel
R. L., A.
Cambridge,
(January
Fujimoto,
Press,
pp. 21-39.
1976,
York,
algorithms:
Approximating
on Foundations of Computer
Symposium
of Computer Science,LosAngeles
Foundations
Reed D. A.,
deterministic
Recent Results
Complexity,
10-18.
shared
emulate
to
\"How
of
pp.
and
Algorithms
Annual
1986),
(October
in
AcademicPress,New
obtaining
1986).
signatures
digital
ACM, 21 (February
1978),
pp. 120-126.
F. S.,
Roberts
Rodeh
M.,
RonseC,
Philips
Rosenberg
News,
Runge
C,
on digital
bibliography
Research
distributively,\"
A. L., \"On
1984.
of Computer and
(1973), pp.
Journal
NJ,
System
pp. 162-166.
24 (1982),
\"A
median
the
\"Finding
Sciences,
Prentice-Hall, EnglewoodCliffs,
Combinatorics,
Applied
time
the
required
Manuscript
Ml85,
1987).
to recognize
properties of
graphs,\"
SIGACT
15-16.
and H. Konig,
Die
der
Grundlehrn
mathematischen
Wissenschften,
Springer,
Berlin, 1924.
Ryser
H.
J.,
\"Combinatorial
Journal of Mathematics,
properties
9 (1957),
of matrices of
pp. 371-377.
zeros
and
ones,\"
Canadian
461
Bibliography
\"Fast
W.,
Rytter
Sankoff
W.
J.,
Schaback R.,
the
\"On
17
Theory of
A.,
N.
and
the
Linear
Pippenger,
and
Sedgewick
New
Sedgewick
R.,
Edition,
Second
Algorithms,
SedgewickR., and J. S.
(1986),pp.31-48.
Journal
of
184-199.
John
& Sons,
Wiley
Vitter,
1978.
York,
\"Shortest
and D. Hoey,
on
Foundations
Symposium
M.
I.,
Addison-Wesley,
in
paths
Euclidean
M.
I., and D. Hoey, \"Closest-pointproblems,\"
Foundations of Computer
Science,
(October
Berkeley
Shamos
the median,\"
\"Finding
1986.
Chichester,
Shamos
SI AM
algorithm,\"
Boyer-Moore
Programming,
Integer
tape
pp. 177-192.
648-658.
13 (October1976),pp.
System Science,
and
of
1988), pp.
(August
deterministic
and
4 (1970),
Sciences,
System
sublinearity
expected
SchonhageA., M. S. Paterson,
Schrijver
nondeterministic
between
Reading, MA,
Addison-Wesley,
Comparison,
The
Macromolecules:
and
Edits,
String
Wraps,
Journal on Computing,
Computer
Time
\"Relationships
complexities,\"
languages/'
pp. 12-22.
Sequence
of
context-free
and
automaton
pushdown
67 (1985)
Control,
J. B. Kruskal,
and
D.,
of
recognition
and
Information
\"Geometric intersection
Science,
of Computer
Annual
16th
Algorithmica,
graphs,\"
on
Symposium
17th
problems,\"
Houston
1976), pp.
(October
208-215.
Shell D. L., \"A
1959), pp.
Shiloach
Y,
sorting
high-speed
U. Vishkin,
and
N. Francez,
L.,
\"Finding
algorithm,\"
Computing,
Ottawa
SiegelH. J.,Interconnection
Studies,
D. D.,
Sleator
G.
and
and
32 (July
ACM,
Smit
and M.
distributed
Case
maximum,
Algorithms,
ACM
1983), pp.
Networks
for
merging,
3 (March
the
2 (July
ACM,
and
in
sorting
a parallel
From a sequential to a
on
Principles of Distributed
Symposium
Rodeh, \"Distributed
Second
(August
the
k selection:
143-153.
Large
Scale Parallel
V.,
of
30-32.
Communications
procedure\"
matching
Processing:Theory
and
1985.
binary
search
algorithms,\"
trees,\"
Journal
Software
of
\342\200\224
Practice
the
Bibliography
52
test for
R., and V.
olovay
(ominskii I.
'The
E. H.,
Jpafford
TR-823,
average
1988).
(November
in a
paths
graph of
positive arcs in
pp. 28-32.
2 (1973),
MA., 1980.
Reading,
Addison-Wesley,
Techniques,
University
Constructive Combinatorics,Springer-Verlag,
D. White,
and
D.,
Purdue
algorithm
An analysis,\"
program:
Science,
of Computer
new
time
worm
Internet
Department
M., \"A
D. C.
of Mathematical
Method
1963.
Boston,
Heath,
Spira P.
S., The
p. 118.
1978),
(February
on
Journal
SIAM
primality,\"
New
York,
1986.
R,
Stout Q.
of
Strassen
Computing, 4
Distributed
and
is
elimination
\"Gaussian
V.,
algorithms
divide-and-conquer
\"Supporting
Parallel
not
for image
(1987),pp.95-115.
Numerische
optimal,\"
processing,\" Journal
13 (1969),
Mathematik,
pp. 354-356.
Takaoka
\"An
T.,
22 (May
on-line
E., Data
Tarjan R.
of
E.,
J.
and
E., and
Journal
U.
van
graph
linear
not
SIAM
algorithms,\"
set union
on
Journal
algorithm,\" Journal
of the
complexity,\"
des
\"Almost
PA, 1983.
Philadelphia,
SIAM Journal
Leeuwen,
Vishkin,
SIAM,
Algorithms,
on Applied and
pp. 306-318.
1984),
labyrinths,\"
Geometry,
of set
union
algorithms,\"
pp. 245-281.
\"An efficient
on Computing, 14 (November
\"Leprobleme
J. S.,
Network
computational
parallel
1985),
Nouvelles
North
(1988),pp.63-82.
Tutte
but
6 (1985),
a good
Tarjan R.
Tarry G.,
146-160.
Structures and
E., \"Amortized
Journal
linear
and
pp.
1972),
DiscreteMathematics,
Tarjan R.
Letters,
22 (April
R.
Processing
Information
algorithm,\"
matching
search
first
E., \"Efficiency
ACM,
Tarjan
pattern
pp. 329-330.
1986),
algorithm,\"
biconnectivity
SIAM
pp. 862-874.
Ann. de Math,
Holland,
are easy to
(1895),
p. 187.
Amsterdam, 1984.
color,\"
Volume
Journal
of Algorithms,
21, Addison-Wesley,
Bibliography 463
Uhr
Parallel
L.,
Ukkonen
E.,
J.
for
\"Algorithms
1987.
York,
string matching,\"
approximate
and
Information
Control,
pp. 100-118.
64 (1985),
Ullman
Computer
Aspects of
D., Computational
VLSI,
Press, Rockville,
Science
Computer
MD,
1984.
for parallel
schemes
\"Efficient
E.,
Upfal
Principles of Distributed
and A.
E.,
Upfal
the
Vacca
\"Maurolycus,
P. M.,
\"Geometry
helps
of Computing,
Chicago
(May
L.
Valiant
\"Parallelism
G.,
L.
a distributed
of the
of mathematical
principle
and
20th
422-425.
SI AM
problems,\"
comparisons
1982),
Journal on
Computing,
on
Journal
SI AM
communication,\"
pp. 350-361.
J. Brebner,
G.
induction,\"
Annual
matching,\"
1988), pp.
in
L. G.,
Annual
Journal of
system,\"
348-355.
scheme
\"A
G.,
Computing, 11 (May
Valiant
in
1975), pp.
(September
Valiant
in
American
of the
Bulletin
Vaidya
(August
to share memory
Wigderson, \"How
1987), pp. 116-127.
on
Symposium
34 (January
ACM,
G.,
Computing,
ACM
First
communication,\"
Ottawa
parallelcommunication,\"
Milwaukee
13th
1981),
(May
pp.
263-277.
van
der
1979),p. 681.
van
von
Leeuwen
J., \"Graph
Science,
Neumann
J., Collected
U.,
Vitter J.
J.
S.,
Neumann
Wagner
the
R. A.,
ACM,
W. C.
New
Press,
J.,
and
analysis
York,
list
ranking,\"
Collected
M.
of dynamic
of Parallel
and
Huffman
&
Analysis
OR
coding,\" 26th
(October
Annual
1985), pp.
of
Coalesced
York,
1963.
Symposium
293-302.
Hashing,
Oxford
1987.
Works, 5,
Macmillan, New
\"The string-to-string
1974), pp. 168-173.
J. Fischer,
21 (January
Journal
91-99.
pp. 319-333.
Chen, Design
York,
1963, pp.
New
of Computer Science,Portland,
and
University
von
4 (1987),
S., \"Designand
Macmillan,
speedups for
parallel
Computing,
on Foundations
Vitter
Works, 5,
\"Randomized
Distributed
(December
of
algorithms,\" Technical Report RUU-CS-86-17,Department
The Netherlands,
Utrecht,
(October 1986).
University of Utrecht,
Computer
Vishkin
ACM, 22
of the
correction
problem,\"
Journal
of
464
Bibliography
H. S., \"A
Warren
for
Communications
relations,\"
Warshall S.,
matrices,\" Journal
on Boolean
theorem
\"A
of
closure
transitive
binary
pp. 218-220.
1975),
of the
9 (January
ACM,
1962),
pp. 11-12.
and applications
of hash functions,\"
classes
N., and J. L. Carter, \"New
1979),
Symposium on Foundations of Computer Science, (October
M.
Wegman
Annual
20th
pp.
175-182.
Automata
Wigderson
Wilf H.
S.,
An
\"Backtrack:
W.
J.
0(1)
expected
Processing
Letters,
\"A new
S.,
C-17 (1968),pp.
S.,
Winograd
of
coloring,\"
\"On
time
algorithm for
inner
IEEE
product,\"
the
for
algorithm
coloring
graph
pp. 119-121.
18 (1984),
the ACM,
of
Transactions
7 (June
on Computers,
693-694.
of
the number
to
necessary
multiplications
Pure
on
Communications
functions,\"
approximate graph
232: Heapsort,\"Communications
J., \"Algorithm
1964),p. 701.
Winograd
Journal
American
graphs,\"
ACM, 30 (October
problem,\" Information
Williams
of
the connectivity
and
graphs
Switching
(1932),pp. 150-68.
\"Improving
of the
Journal
algorithm,\"
matching
pattern
Theory,
certain
compute
23 (1970a),pp.
165-170.
Winograd
Winograd S.,
Sequential
New
York
\"Someremarks
(1973),
pp.
N.,
Yao
A.,
Algorithms
\"An
0(\\E
Information
Yao
A.,
Annual
Symposium
Ziv
Algorithms,
of
Press,
log
\\V
of Computations,
|)
Prentice-Hall, EnglewoodCliffs,
algorithm
for
finding
minimum
1986.
NJ,
trees,\"
spanning
Letters, 4 (1975),pp.21-23.
\342\200\224
toward
computations
on Foundations
a unified
of Computer
Science,Providence,
RI
(October
222-227.
J.,
of polynomials,\" in Complexity
J. F. Traub,
ed., Academic
multiplication
International
181-196.
Complexity
Processing
\"Probabilistic
1977), pp.
fast
on
of the
via variable-rate
coding,\"
(September 1978), pp. 530-536.
sequences
IT-24
INDEX
347-349, 351-354,356-363,369,
41
O 39
414-415,426-434
370-374,
00 41-42,174,316,410
greedy 210-211,235,237,363
0 41
for
ft
networks
interconnection
378,
389-403
41
co 41
for
notation
numeric 293-320,379-380,398-^01,
Aanderaa, S. O.
data
Abstract
Ackerman's
404-407
113
types
402-403
Acyclic
Acyclic
Addition,
84
Adjacency matrix representation
Adleman,L. M. 295,316
A. V. 85, 174, 175,305,316,336
Aho,
Ajtai,
Angluin,
R. 411
293-320,436-438
37-60
of
definition
distributed
Arborescence
357, 363-367
approximation
208, 376
158,
169, 172,
409-410,
431
of 341
efficient, definition
geometric 102-104, 265-291,331-333,
434-436
graph
95-96,98-101,
357, 363-367
186
66,
24-26,
35-36
divide-and-conquer 103-104,113-114,
379-380,383,
289
131,137,141,144,
175
A.
293-320, 436-438
of
369
D.
Apostolico,
analysis
322
Anti-Gray codes 33
Algorithm
algebraic
119-183
M. 410
Algebraic algorithms
42
236
path
Alternating
Akl, S. G. 409
Aleliunas,
sets
systolic 404-409,411,416
All shortest
paths problem 212-214,
G. S. 409
Almasi,
84
list representation
Adjacency
sequencesand
G. M. 75
Aderson-VePskii,
of an
time
running
379-380,411
parallel
158-164, 175,287,320,
probabilistic
86
function
214,216,304,375-416
parallel
62
185-264,325-326,
Arithmetic sum
11-12,31,53
V. L. 317
Arlazarov,
of 63
Array, definition
Articulation point 218,223,260,430
ASCII
Assembly
Assignment
146
line 404
problem
330-331
of
38
466 Index
Booleanmatrix
410
M.J.
Attalah,
Augmenting
path
L. 248
Auslander,
A.
Borodin,
130
AVLtree 75-77, 104,114-115,
174
Avni,H.
multiplication
316,410
338
Bach, E. 297
Breadth-first-search
369, 373
358-363,
Baer, J. -L. 86
75, 104-106,248
Balance factor
R. 369
Bar-Yehuda,
Batcher, K. E. 394,410
L. 369
Beame,P. W.
410
begin 5
Blelloch, G.
409
Bentley,
Berge,
31, 114
217
Binary
357, 363-365,
conversion
numbers,
Binary
search
120-125,285,342,431
molecular
149,155
matching
117,187,372
235-238,
Bisection 125
R.
Blum, M.
Bohr,
N.
55
Bolzano
175,247,248
method
Bondy,J. A.
247
125
409
K. M.
Chandy,
247,432
31
Chen,CW.
86
D. 247
Cheriton,
Chiba,
50
equation
Chavey, D.
Christofides,
174
79
Chand,D.R.286
N.
Choueka,Y.
369
Bollobds, B.
337, 357
98-101,113-114
Chaining, separate
Chartrand,G.
71-77, 86,285
248
problem
Characteristic
34,263-264
Biology,
24
A. L.
Celebrity
34, 115
search tree
L. 86
Cederbaum, I.
edge-biconnected259
packing
175
L.
Carter, J.
Cauchy,
Binary
411
Capobianco,M. 247
259
complete
113
Capacity 238
search
component 217-223,249,259-260,430
definition of
142,164
127-129,
network
Butterfly
Carrol,
Binary tree
55,369
Bussey,W.H.30
Biconnected
Bitner,J.
sort
Bucket
409
BFS,seeBreadth-first
Bipartite
C. A.
Burge,W.H.
Bertsekas, D. P.
Bin
network 256
Brualdi,R.A. 55,336
174,440
extension
317
396-398,406,412
Brown,
J. L. 55, 106,287
C. 247
graph,
Briggs,
Bre1az,D. 369
E. 31
R. E.
in a
F. A.
Broadcast
Beckenbach,
Bellman,
Bridge,
Broadcast
Bavel,Z. 55
Bent,S. W.
D. 369
Brigham, E.O.
Bates,J. L. 106
Baumert,
246, 249,426,427
Brebner,G.J.410
Brelaz,
87
155,174
358-363, 370
95, 189, 198-199,238,
Branch-and-bound
b-Matching
Bag
317,
Boyer-Moore algorithm
Backtracking
304-309,
319-320,325-326
239
247
175
N.
248,370
Circle 34,270,271,290,332
186
in a graph
Circuit,
Eulerian
187; see also Eulerian
graph
Hamiltonian
243-244; see also
Hamiltonian
cycle
467
Index
Conversion
binary to decimal
371,372
Closed
Convex hull
path,
Closestpair
287, 289
problem
Coalesced hashing
J. 304,317
Cook,S. A.
78-80
(in hashing)
175, 252,
14, 161-164,
Cooley,J. M. 317
262,432
Cover
edge 262
set 355
network
of a
Complement
Completebinary
tree
Complete graph
96,
Complexnumbers
CREW 378,382,410,412,415
Critical
node (in an AVL tree) 75
Crossedge, in depth-first search 194
38
complexity
344
Crossproduct
413
Cryptography
295-297
Cube-connected-cycles 390
42
Component
Culberson,
biconnected
187, 192,415
connected
strongly connected
86
241
Cycle(in
186
226-233,247, 250,
a graph)
even-cycle problem
Hamiltonian
230
248, 372
odd-cycleproblem
Compression
data
145-148,
path
82,86
344
complexity
186
security
Concurrent-read
88
see CRCW
concurrent-write,
Concurrent-read exclusive-write,
Condensation graph 226
form
normal
Conjunctive
see
346
187,
192,415
machine
co-NP 368
R. L.
CREW
Data
structures
Context-free language
317
62
252
Deadlock 6
31
representation
Decimal to binary
conversion
26-28,
36
106
61-90
Connected component
Connection
246
233-234,
Concatenate operation
Constable,
J.
251,260
Computational
363,
CRCW 378,383,410,412,413
space 42
Computer
348, 355-356,358,
371,373,432-433
55, 83
computational
372
3-sets
vertex 262-263,
318
asymptotic
by
34, 263-264
244, 263, 356
Complexity
amortized
414
circuit
Counter
14
352-354,370,371,373
3-coloring
Communication
316
D.
Corasick, M. J. 174
369
plane coloring
time
344,346,368,410
Coppersmith,
k-coloring
266
theorem 347
Cook's
369
edgecoloring
of
polygon, definition
Convolution 405-407
Cole,R. 410-411
Coloring problems
266,276
path
Convex
86
Cohen,
Collision
289
a 273
stretching
Convex
346
286-289
273-277,
approximate
80
Clustering
CNF
36
26-28
to binary
decimal
147
223
Index
468
Decreasing first
364
fit
Decryption
Domino principle
186
Dekel,E. 410
in a
in a
hashing 80
Double
induction
Deo,N.
for 234-235,
95, 189-197,221-225,
Dynamic
Dershowitz, N.
Deterministic
coloring 262,432
cover 262
4, 113
193,388
definition
algorithm,
of
in
158
search
Diameter
of a graph
Edit
113,247
E. A.
247, 254
204-208,
algorithm
317
376
316-319,
379-380,
383,
394-396,
50-51
382
Divide-and-crush
149
Dobkin,
Dodecahedron
D. P.
175
6
graph
relations
259
paths 23-24
155
steps
149
of parallel
Efficiency
Efficient
algorithms
Election, 1988Presidential
definition of 62
376
of 341
definition
algorithm,
Element,
D. F.
317
336
trees 263
Embedding
377
Emulation
146
Encoding
295
Encryption
end 5
Eppinger,
409-410,431
Divide-and-conquer recurrence
186
Ellipsoid algorithm
algorithms 103-104,
Divide-and-conquer
map 17
Elliott,
371,378,430-431
Distributed algorithms
of a planar
316
Dijkstra, E. W.
83,
Edge-disjoint
340
71
Dictionary
266
the plane
of a graph
Edge-biconnected
115,389
of a set of points
Dijkstra's
286
H.
Edelsbrunner,
Edge
W.
247
192,223
192-196
tree
114,
34
decomposition
Ebert,J.
194
108-111,
155-158,407
Ear
nonrecursive implementation
410
programming
429
194
edge
66
S. 174
Dvorak,
Dwork, C.
crossedge 194
numbers
S. E.
variable
Durian, B. 174
227-233, 246-250,388,426-427,
back edge 194
forward
409, 411,
123, 386-389,
114
Dreyfus,
Dummy
378
search
Depth-first
tree) 75
an AVL
440
247,410
Depth of a circuit
DNA
18
technique
Doubling
114
E. V.
Denardo,
73-74
Dense graphs,
algorithms
244-246
Dinic,
91
Double
Deletion
Diffie,
of points
Domination
Degree of a vertex
Degree
set 348-349
289
Dominating
295
J. L.
Equations, system
Erdos, P. 175
EREW
86
of
315
378,380-381,383-389,410,
412-415
Errors
in
design
by induction
111-112
Index
in
induction
in
reductions
proofs 28-29
334-336
Franco, J. 369
294, 297-298,316,318
Euclid's algorithm
187,247
Euler,L.
formula
17-18,30
387-389,409-411,414
187-190, 247, 251-252, 373,
graph
230, 369
372
problem
Even-cycle
3-sets
Full-history
see EREW
96-98
one-to-one
N.
247,
of polynomials
317
transform
Fourier
Fast
287
M.G.
Federal Express
Feedback-edge
257,
Girth
Goldberg,
309-317,443
numbers 34-35,46,
62
155
comparisons
Golovina,
174
First
fit 364, 370, 374
M. J. 86, 175, 317, 369,380, 410
Fischer,
Flajolet, P. 55
network
238-243,
246, 248, 327, 329,
Flow,
338
Flynn,
M.J.
410
Forest
174,248
Four-Russian
H.
Gonnet,G.
248
247,248
85,86,
175
Goodrich, M. T. 410
416
problem
409
R. L.
55,286
95-96,98-101,185-264,
algorithms
356-363,369,
(in DFS)
194
algorithm
306-109,
A. S.
R. E.
Gondran, M.
325-326, 347-349,
Fox,G. C. 409
Fraenkel,
31
Graph
178
186-187
edge
369
Fortune, S. 287
Forward
Gomory,
Graham,
Johnson sorting
L. I.
Gottlieb, A.
113, 174,247
S.
Golumbic, M. 247
Gossip
R. W.
Ford,L.R.
Goldstine, H. H. 113
Golomb, S. 369
Fingerprinting
Floyd,
of a graph 255
A. V. 248
Goldwasser,
FIFO (first-in-first-out)
File
49, 60,318
410
E.
274-275, 286
Gillis,J. 30
373, 429
FFT
Fibonacci
265-291,
247,409
Gift-wrapping algorithm
Feng,T. 411
Fich,F.
A.
Gibbons,
321
set
102-104,
331-333,434-436
Geometric series 53
inverse 314-316
Faust,
318
Geometricalgorithms
309-317,443
363, 369
356,
409
E. F.
Gehringer,
Faradzev, LA.
431
86
B. A.
86
R.
248, 366,
Galil,Z. 174,175,247,248
Galler,
294-297
Extendible hashing 86
Fagin,
51-53, 57,
relations
137
Gabow, H.
Exclusive-read exclusive-write,
Exponentiation
248
recurrence
Function,
Exact coverby
409
D. R.
336, 369
247, 248,
S.
146
R. M.
Fulkerson,
432
Even,
Freetrees 83
Fujimoto,
tour technique
Eulerian
410
247
Frequency
constant 53
Euler's
G.
Fredman, M. L.
Frederickson,
365-367
salesman
410
N.
Francez,
351-354,
370-374,
414-415,
426-434
316, 317
biconnected 217;seealso
Biconnected
component
175
469
bipartite
111-112, 117,187,372
470 Index
problems
coloring
code
Gray
370,371,373
356
226
186
directed
definition
edge biconnected,
415-416
embedding
Euclidean
of 259
365
254,
255
Hamiltonian
369
243-246,
induced 95-96,
114, 187,251
Haken, D.
55
Half
291
plane
336
Hall, M.
323, 336
P.
theorem
Hamilton's
89
234-238,248,251,260-261,
291, 338,
342, 366,431;seealso
Matching
problems
by induction,
involving
18-19,
representation
in a
strongly
subgraph
subgraph
traversal
connected
206,428
260
443
tree 104
E. 316
Height of a node
M.
Hellman,
174
443
S. T.
Hedetniemi,
undirected 186
unipathic
137-141,
Hedetniemi, S. M.
induced 187
189-199
weighted, definition
86
of a
260
80
double
86
86
extendible
186
unilateral
78-80,
coalesced
186
251,260
86
universal
263
53-54
79
functions
Hashing
simple 186
split
85
247
Harmonic series
of 83-84
residual 242
selfloops
F.
373
31
R. W.
Harary,
Hash
372
regular
356, 372,
258,
Hansen, W. J.
34, 35
369
path
338
248,
319
243-246,
cycle
Hamming,
186
multigraph
23-24,
puzzle.
Hamiltonian
proofs
6, 243
Hamilton's quaternions
255
matching
336
324,
Hamilton, W. R.
planar
287
H.
Hall's
263,433
kernel
369
Guting,R.
Hall,
253
injective
join
174
Guibas,L.J.
187-190,247,251-252,373,
of a
interval
M. 336
Grotschel,
Gusfield, D.
432
girth
290
Gries,D. 4,31,113,175
Guessing 11-14,43-44,47-49, 58,345
graph
Eulerian
55
Grid polygon
115,389
definition of 186
see Directed acyclic
acyclic,
directed,
Mesh
see
Grid,
318
398,410
Greene, D. H.
186-187
diameter
A.
Greenberg,
234-235, 244-246
210-211,
algorithms
Greedy
297-298,
318
of polynomials
of
connected, definition
217-234
decomposition
definitions
33-34
ami
320
31, 33-34,
20-23,
open 22,320
348
of a
complement
dense
31
Gray,F.
(continued)
Graph
in a
Heuristic 211,254,369
of
187,201
HEX
369
471
Index
Hibbard,T.
W.
Hillis,
102-104, 282
Hinrichs, K. 287
Hoare,C. A. R.
Hoey, D. 287
Inner
174
248,
House,
design 2
to algorithm
174
Huffman's
334-335
Hwang, K. 409
Incident
J. 174
325
matrix
174,336
Iyengar,
S. S.
(and
virtually
choosing
the base of
choosing
the order
of
everywhere)
137,167, 172
17, 30, 92-94,
96-101,112-113,
128,
200,
411
John, J. 174
247
D. B.
Johnson,
Johnson, D. J.
344,
Join graphs
28-29,
reversed
369, 370,
86
202-203,
111-112
217
k-connectivity
element
*th-smallest
143-145, 174,179,
396-399
usage 14
114
generalization
356, 363,
89
Jones, D. W.
18
double
double
355,
174
S. M.
Johnson,
221,282-283
common errors
definition 10
314-316
410
H. V.
Jagadish,
of 186
definition
Indegree of a vertex,
Independent set 19,262-263,356, 371, 433
Induced subgraph 95-96, 114,187,251
3, 9-36
113
248, 369
graph
Itai,A.
134,
113-114,
167-169,
175
Induction
24,
paradox
Incerpi,
281-285,
263,433
graph
Inventor's
389-390,401-403,415
Hypercube
Interval
175
J. W.
Hunt,
267-270,
problems
287,289-291
145-148,
encoding
125-127
search
Intersection
148, 174
D. A.
378,
310
Interpolation
Interpolation
B.C.
Huffman,
network, algorithms
389-403
Hu,T. C. 248
Huang,
Interconnection
369,410
analogy
331,336,
54
Integral
92-94,
71-72
361-363,371
261, 287,
305,316,336,410
E.
178
69-70
linear programming
Integer
Hopcroft, J. E.
rule
174,
130
sort
410
H.J.
Horowitz,
in a heap
Hong,J.-W. 409
Horner's
305
product
in a
Hofri, M. 55
Hoover,
142
bound
Insertion
248
CM.
Hoffman,
10-11,25
Information-theoretic lower
Injective graph 253
127, 131,135,137,
In-placesort
Input size 38,294
175
D. S.
Hirschberg,
of
variations
86
N.
elimination
D. 410
Hidden-line
25,244-246
22-24, 93,
strengthening the hypothesis
105-109, 113, 145,162, 167-169,
281, 282, 379, 418, 423,428, 435,
437, 442
411
Kailath,T.
Kapur,
S. S.
Karatsuba,
286
A.
Karlin,A. R.
Karmarkar,
316
86,411
N.
336,370
M. 174,175,243,261,346,
255
Kernel of a graph
B. W. 370
Kernighan,
Karp, R.
Khachian,
L. G.
336
368,
370
472
Index
Killian,
J.
369
Linesin
Klee,V. 336,369
175
S. T.
Klein,
KMP algorithm
174, 323,
148-155,
Knuth,
155,
357,
160,
173, 174,175,316,369,436
Knuth-Morris-Pratt
148-155,
algorithm
323,427
R. J.
Litwin,
W.
B.
Kruskal,J.
411
86
Load balancing
380
Localreduction
356
54
rules of
subsequence
increasing
187
Loopinvariant
31, 36,
26-28,
175,247,410
411
D.
Lopresti,
31, 175,248,336
174, 178, 180, 331-334, 339
Lovdsz,L.
Lowerbound
86
142
i nformation-theoretic
75
M. A.
Langston,
Lower triangular
Lucas, E. 247
30
Landis, Y. M.
Language-recognition 342
Las Vegas
344
A.M.
248, 369
174
of a path
370
164-166,175
409
G. K.
Manacher,
174
Manber, U. 4, 55,86,
318
113,
Manna, Z. 4,
113, 173
Masek,W.J.317
247,432
problems
Matching
198
234-238,248,251,
35,234
Lin,S. 370
string
148-155
Line
3-dimensional 356
14,265
266
Linear
Linear
Linear
338-339
induction,
Mathematical
281-287,435
431
324,337, 357
perfect
Line-sweep
technique
398,
369
R.
Manber,
201
Line segment
175, 248,
410,440
Lenstra, J. K. 370
L.
336
problem
Malek, M.
114
Lempel,A. 174,248
Lesniak,
T.J.
Majority
Lawler, E. L. 248,366,
Least common multiple
Leiserson, C. E. 411
Length
E. M.
Lynch,
Maggs, B. M.
algorithms
33
Lattice point
Law,
Luks,
algorithms 159;seealso
Probabilistic
339
matrix
Lueker,G.S. 55
174
nondeterministic
141-143
for sorting
E. 369,380,410
I.
Lakatos,
113-114, 134,
176-177
Ladner,R.
167-169,
175
317
D.
T.
174,
Lipton,
Longest
Kurtz,
409
Lipson,J.D. 309,316
Logarithms,
Komlos,J. 410
Kbnig,
64
Lipovski, G. J.
427
8, 108-111,114-115,
370
D. E.
in the plane 13
position
general
lists
Linked
problem
Knapsack
of 328
form
standard
see Induction
Matrix
339
Boolean
304-309,317,
multiplication
319,320,325-326
and
graphs
246, 325-326,
336, 339-340
Index
Molecular biology 149,155
J. C. 247
Molluzzo,
325
incident
336
inversion
lower triangular
Monotonically increasing
Monte Carlo algorithms
Moore, J. S. 155, 174
339
301-309,316,318,
multiplication
325-326, 333-334,336,339,
398-400,410,412,416
Multiple-instruction multiple-data
upper triangular
339
vector multiplication,
of Booleanmatrices
404-405
by
of general
242
Maximum-consecutive subsequence
in
problem
6, 247
Median
spanning
tree
Megiddo, N. 411
217-218
Menger,K.
Murty,
U. S.
Myers,
E. W.
(of a
389,398-400,414
A. R.
Meyer,
317,369
Miller, R. 297
MIMD
Minimum-cost spanning
tree
247,
208-212,
249, 255,257, 338,340, 365, 428, 429
Minimum
in a set 143-144, 169-171,
element
176,380-382
247-248
Minoux,M.
Minty,
G. L.
336
A.
Misra,J.
175,409
Modeof a multiset
A. 247
A.
410
246, 248,
I. 94
Nil
pointer
86,287
66
Nishizeki, T.
247
definition
of
Node,
Nonconvex
polygon
Nondeterministic algorithm
344-346,
language recognition 344
model
of computation
Notation
171-173
324, 327,
329,338
Nonuniform
173
Mirzaian,
238-243,
Nievergelt, J.
E. 248
Miniekas,
Network flow
Newton,
376,410
393-396
on a
sorting
of trees 390
329
241
cut
393
split
in 256
broadcast
communication 146,208,256,
parallel 394-396,413
Moitra,
shortest
Network
393-395,410
Moffat,
410
cost,
Negative
104,130-131,174,281,
mergesort
Mesh
164,171
4,175,317
Neighborhood
Merge
element
175,316,336
R. 247
Munro,I.
Nassimi, D.
286
K.
Mehlhorn,
309, 316-317,
171-173
Multiset
174, 179;
element
144-145,
smallest
of an
Multiplicity
McCreight, E. M. 175
MCST,seeMinimum-cost
McVoy, L. 4, 248
339,
319
176,380-382
Mazeproblem
316, 318,
301-309,
398-400,410,412,416
of polynomials 298-300,
143-144, 169-171
a set
matrices
325-326,333-334,336,
106-107
Maximum element
317,
304-309,
319-320
theorem
min-cut
Max-flow
410
376,
Multiplication
F. 30
Maurolycus,
378
186,259,262
Multigraph
333
transpose
155,174
Multicomputer
333-334
symmetric
48
function
159
174
B. M. E.
Moret,
Morris, J. H.
319
permutation
473
algorithms
graphs 83-84,
geometry
186-187
265-266
370
143
Index
474
NP-completeness 341-374
problems
NP-complete
345-357,368-374,429,
Parter, S. V.
246,
Partition
439-440
398-401,404-407
248
133-135, 144-145,
(for quicksort)
176
293-320, 379-380,
algorithms
Numeric
I. 409
Parberry,
175, 244,
Partition
357
problem
Pascal 4
o Notation
Pascal'striangle
41
39
O Notation
41-42,
OONotation
Objectivefunction
Octal tree
174, 316,
327,361
Odd-eventransposition
246
sort
411,
390-393,
Hamiltonian
174
see
shortest,
Pattern
96-98
22-23, 320
function
174, 179,396-399
287
Outdegree of a vertex,
definition
186
of
M. C.
Pearl,
J.
369
Perfect
matching
Perfect
shuffle
376
174-175
Pigeonhole
316
V.
Pan,
C. H.
Papadimitriou,
24
inventor
Paradox,
efficiency
398-401,404-405,
410,416
of
375-379,
routing
24,30,31,113,169
287-290
290
regular 291
simple 266,270-272,288,331-333
401-403
sequence
comparisons
407-409,
376
division
Polynomial
411, 416
sorting 390-396,410,411,413,414
tree operations
33, 266-277,
grid
385-387
of
165
public-opinion
selection 396-399,410,414
speedup
Poll,
Polygon
409-411
prefix 382-385,410,411,413,415
ranking
17
map
Polya,G.
merge 394-396,410,413
models
377
multiplication
174
Plotkin, S. A. 336
Plummer, M. D. 248
376
folding principle
matrix
375-416
319
N. 86,
Pippenger,
L.
369
Pitt,
248, 336
182
matrix
396-397,409
Pipeline
247
Pang,C.
142,
Philanthropist
35,234
390
Perfect speedup
Permutation
345
matching
Permutation
341
see String
matching,
Paull,
Perl,Y.
P (class of problems)
problem
problems
Shortest-paths
simple 186
l-in-3SAT371
P=NP
372-373
258,
201
length of a
Ofman,Yu. 316
T.
82, 86
compression
414
Ottmann,
239
augmenting
393-396,410,411
Odd-even mergesort
One-to-one
236
alternating
233-234,
A. M.
174,317
186
Path
390
Odd-cycleproblem
Odlyzko,
M. S.
Paterson,
410
31
O. 55
Patashnik,
evaluation
greatest
multiplication
387-389,
319
318
common
divisor
298-300,
318
309, 316-317,
Index
Polynomial-time
Reed, D. A.
369
hierarchy
Pool 87
155, 174
V. R.
Pratt,
146
constraint
Prefix
Prim,R.C. 247
testing 369
root of unity
Primitive
Priority
queue
62, 68,
Priority
search
207,246
Reingold,
E. M.
Repeated
squaring
problems
Pruhs,K.
175
algorithm
A. H.
Kan,
160
287
C.
Ronse,
295-297,316
165
poll
Purdom, P. W.
55, 369
line
Rotation
68, 86
62,
Queue, priority
Quicksort 131-137,144,174,176
problem
Routing
in
Raghavan, P.
175
search
285,435
in linked
number
Sankoff,D. 175
lists 385-387
SAT
317
46-53, 55,56,
divide-and-conquer 50-51
with
full
history
unfolding
Reduction
see
175,183
Recurrence relations
57, 58,
51-53, 137
194
problem
(satisfiability)
346-347,
350-354,357,370,371,440
186
sequence
317
N.
Santoro,
Reachable vertex
175
S. L.
Salzberg,
Rao,K.R. 317
Recursion,
369,410
Sahni,S.
Saks,M. 175
Rank 143
Realizable
317,409
Rytter,W.
Rational
42
411
A.G.
Ranking
of an algorithm
Ryser,H.J. 175
Random
Range
scheme
410
time
Running
127-129
sort
Ranade,
174,287,369
401-403
encryption
317
Runge,C.
Rabin, M. O.
329-330
a hypercube
L.
Rudolph,
183
tournament
Routing
RSA public-key
316
409-410
Quinn,M.J.
75
304,317
390
319
186
270
Rotating
Round-robin
Quaternions
66, 83,
of
113
A. L.
Rosenberg,
Roth,M.
Radix
55
F. S.
Rodeh,M.175,336,410
357
encryption scheme
Quadtree
G. 370
149
RNA
368
Public-opinion
244-246
155
410
D.
Roberts,
Pseudorandom numbers
83-84
Rivest,R.L. 174,295,316
278-281
Pseudopolynomial
85, 369
297, 317
Ring 305
Rinnooy
Proximity
291
410
Richards,
158-164,175,287,
175
proof
Public-key
polygon
Revision maintenance
320,402-403
PSPACE
Regular
Representation of graphs
Residual graph 242
Reversed induction
25,
313,315
86
Probabilistic algorithms
Probabilistic
graph 372
Reif,J.H.
409
Regular
59
SAT,2-in-4371
368
Savitch,W.J.
Saxe, J.
B. 55
Schaback,
Scheduling
Schbnhage,
174
R.
185
A.
174
295-297,
Index
476
Schorn,P. 287
Simplex algorithm
336
A.
Schrijver,
Schwab, B. 86
Single-source
Sink 99
Search
Sizeof an
120-125,285,342,431
binary
search trees 71
in binary
depth-first
227-233, 246-250,
Smit,G.V.
388,426-427,429
125-127
interpolation
linear 173
80
Seed 160
143-145, 174, 179,
problem
in a graph
Separate chaining
186
337,
407-409
119-183,380-383; seealso
Shiloach,
Shmoys,
295,316
365
Minimum-cost
spanning
Spencer,J. 175,247
Y. 410
D. B. 370
263
Square
of a matrix
Square
root
429
Siegel,H.J. 411
319, 334
272
304
Stable algorithm
255,
428,
247
P. M.
Split graph
185, 201-208,
Shortest-paths problems
212-214, 247, 249, 253,254,
257, 258, 322, 325, 338,427,
Shrira, L. 410
Stack 194
of linear
Standard
form
Standish,
T. A.
Stanton, D. 173
409
Steele,G.L. 410
Simple graph
Steiglitz,
path
Simple polygon
programming 328
85
SIMD 376,410
Simple
186
186
K.
248,336
Stirling's approximation
266,
see also
tree
Speedup 376
378-389
Spira,
D. P.
187
tree 35,186,371,429;
Spanning
270-272,
430
173
E. H.
Spanning forest
174
Siewiorek,
130
selection
Spafford,
297
Shared-memory algorithms
Shell, D. L. 174
Shellsort
topological 199-201,249,427,
Space complexity 42
algorithms
M. I. 286-287,
390-393, 411,414
127-129
radix
cover 355
Shamos,
transposition
390-396,413-414
quicksort
155-158,175,
181-182,317,324-325,
Sequence
141-143
393-396,410
parallel
407-409
337,
Set algorithms
128, 178
for
bound
network
106-107, 119-183,317,
Sequence comparisons
Shamir, A.
178
130
odd-even
algorithms
J.
137-141, 174
mergesort 130-131
79
324-325,
Shallit,
heapsort
lower
Self-adjusting trees 86
Sequence
Johnson
lexicographic
130
sort
Selfloops
142, 164
127-129,
Ford and
insertion
396-399
Selection
31
in-place 127,131,174
Segall,Z.409
Selection
369
I. S.
bucket
R. 174
Sedgewick,
B. 113
Sorting
186
Security
86
Smith-Thomas,
M. 410
Snir,
Sominskii,
Secondary
102-104,115
174
Solovay,R.
207,246
priority
paths
328
D. D.
Sleator,
189-197,221-225,
376
201-208
38,294
input
Slackvariable
249,426, 427
95,
shortest
Skyline problem
189, 198-199,238,246,
95,
breadth-first
336
multiple-data
Single-instruction
288, 331-333
Stockmeyer,
L.
369
54, 142
477
Index
Q. F.
410
Strassen, V.
369
Stout,
Strassen's
algorithm
the induction
hypothesis
10,
Strong
Strongly connected
component
250,251,260
247,
226-233,
complete
124
problem
subsequence
Subsequence, longest
increasing
Summation problems 11,15-16
Summation
Supporting line
System
378, 404-409,
323-324
411,416
T. G.
node
Tarry,
86
369,
352-354,370,371,373
3-Dimensional
complexity
42
Timetable
Towers
of Hanoi
Tractable problem
249,427,
430
116
248, 317,322,
matrix 339
Tremaux,
247
W. T.
of a matrix
333,414
114
346
assignment
Tsitsiklis, J. N. 409
TSP, see Traveling
317
Tuckey,J.W,
Turing machine
Turner, J. S.
W. T.
344,
369
247
2-in-4SAT 371
Uhr,
L.
396
see also
tree
365
inequality
339
Transpose
spanning
Triangular
Tutte,
341
closure 214-216,
Transitive
Triangle
Truth
357,370, 371,439
374
Topological sort 199-201,
Toussaint, G. 286
Minimum-cost
Trotter,
356
Matching
3SAT 350-354,
Time
247, 248,
174,
247
G.
35,186,371,429;
spanning
55, 85,86,
410,431
247,
365,428,
208
shortest-path
R. E.
3-Coloring
208-212,
66,83, 186
rooted
30
67-68
390
quad
175
Takaoka,T. 247,426
Tarjan,
67
104
spanning
self-adjusting
Talmud
in
Szemeredi, E. 410
Szymanski,
representation of a
minimum-cost
175
A.
263
for
implicit representation of a
insertion
into a 71-72
machine 396-398
a 73-74
from
of a
height
376
algorithms
Systolic
192-196
193
free 83
of equations 315
of distinct representatives
System
depth-first
explicit
matrix 333-334
Symmetric
in
embedding
274
Synchronization
descendant
deletion
54-55
43-46,
technique
167-169
438-439
426,
335,424,
114, 187
95-96,
198-199
breadth-first
Subgraph 186-187
induced
71-77, 86
binary search
109, 157
induction
Stuttering
115
34,
binary
86
H. R.
221
biconnected
180-181,323,427
Strong,
75
balancing
177,
174-175,
148-155,
189-199
ancestor in a 193
AVL 75-77,104, 114-115
442
435,437,
428,
matching
365-367
Euclidean
113,145, 162,
22-24, 93, 105-109,
167-169, 281, 282, 379,418,423,
String
390-393,411,414
326
Strengthening
sort
Transposition
salesman
346-347
problem
429
Index
478
Voronoi diagram
175
UkkonenE.
85, 86,175,305,
J. D.
Ullman,
316,
336, 409,
287
164
Voting
411
definition
graph,
Undirected
Union of
of 83,
186
260
graph
Unilateral
290
polygons
Upfal,E. 86,411
matrix
triangular
Upper
Wagner, R. A.
175
Waldinger,R.
173
248
Warren, H. S.
248
S.
Warshall,
Wegman, M. N.
86
Weighted
187
graph
Weiner,P. 175
339
173
D.
White,
Whitney, H. 217
Vacca,G. 30
A.
Wigderson,
Vaidya,
P. M.
366
Wildcard
Valiant,
L. G.
369,402,410
Wilf,
Nat, M. 174
J. 86, 247,
Leeuwen,
van
Vertex
83,
336
186
Wirth, N. 85
cover 262-263,348,
355-356,
371,373,432-433
degreeof 186
outdegree of
of a planar
17
Vishkin,
U.
Neumann,
38-39
X3Cproblem
372
I. M.
Yaglom,
410
55,86,174
Ziv,J.
J.
31
Yao,A. 175,247
VLSI 263
von
173
186
Virus 173
Vitter,J.S.
Worm
Wrap-around 398
266
of
reachability
Wood, D. 287
186
map
of a polygon
358, 363,
Worst-case
186
of
369
algorithm
Winograd's
basis 260,431
indegree
H. S.
Williams, J. W. J. 174
Winograd, S. 316
der
van
175,369,411
174
113, 174
174
Zorat, A.
410
301,316,319
\342\200\242
Creative
Ap
i roach
i
of Arizona
University
This book
emphasizes the creative aspects of algorithm design. It teaches the thinking behind the
of algorithms. Algorithms are describedby following
the steps leading to their development
(sometimes
including wrong steps). The heart of this creative process lies in an analogy between
theorems
combinatorial
The book contains
provingmathematical
algorithms.
by induction and designing
creation
hundreds
understanding
the
reader's
abilities and
problem-solving
HIGHLIGHTS
coverage of algorithms.
known
many
techniques for designing
over 500 mostly nonroutine
exercises,
\342\200\242
Current,
up-to-date
\342\200\242
Covers
\342\200\242
Contains
\342\200\242
Discusses
\342\200\242
Covers
implementation
parallel
\342\200\242
Includes
The
of efficient
introduction
ABOUT
algorithm
and unifies
many
of them.
solutions to about
a quarter
design
of them.
ideas.
examples
statistics,
mathematics,
including
an
and gives
algorithms.
pseudocode
design
algorithms,
to
THE AUTHOR
Manber
1987. He
90000
9\"780201\"120370
ISBN
Theory
Addison-Wesley
Publishing
Company
0-201-12037-E