APPROXIMATION ALGORITHMS FOR COMBINATORIAL PROBLEMS
David S. Johnson
Massachusetts Institute of Technology
SUMMARY
Simple, polynomial-time, heuristic algorithms
for finding approximate solutions to various poly-
nomial complete optimization problems are analyzed
with respect to their worst case behavior, measured
by the ratio of the worst solution value that can
be chosen by the algorithm to the optimal value.
For certain problems, such as a simple form of the
knapsack problem and an optimization problem based
on satisfiability testing, there are algorithms
for which this ratio is bounded by a constant, in-
dependent of the problem size. For a number of set
covering problems, simple algorithms yield worst
case ratios which can grow with the log of the pro-
blem size. And for the problem of finding the max-
imum clique in a graph, no algorithm has been found
for which the ratio does not grow at least as fast
as 0(ne), where n is the problem size and e > 0
depends on the algorithm.
I. INTRODUCTION
Cook and Karp in [1,5] have shown the exis-
tence of a class of combinatorial problems, the
"polynomial complete" problems, such that if any
of these problems can be solved in polynomial time,
then they all can. Many of these problems such as
tautology testing and the traveling salesman
problem, have become notorious for their compu-
tational intractability, and it is widely conject-
ured that none of them can be done in polynomial
time. Many problems of this class are in the form
of optimization problems. Some examples of poly-
nomial complete optimization problems are:
SUBSET-SUM: Given a finite set of positive
numbers and another positive number called the
"goal", find that subset whose sum is closest to,
without exceeding, the goal.
Work reported herein was supported in part by
Project MAC, an MIT research program sponsored by
the Advanced Research Projects Agency, Department
of Defense, under Office of Naval Research Con-
tract Number N00014-70-A-0362-0006 and the Nat'l.
Science Foundation under contract number GJ00-4327.
Reproduction in whole or in part is permitted for
any porpose of the United States Government.
38
BIN-PACKING: Given a finite list of numbers
between 0 and I and a sequence of unit-capaclty
bins, find a packing of the numbers into the bins
such that no bin contains a total exceeding I and
the number of non-empty bins is minimized.
MAXIMUM SATISFIABILITY: Given a set S of dis-
junctive form clauses, all of whose disjuncts are
either literals or their negations, find a truth
assignment to the literals which satisfies (makes
true) the most clauses.
SET COVERING I: Given a finite cover of a
finite set, find that subcover which uses the few-
est sets.
SET COVERING II: Given a finite cover of a
finite set, find that subcover which has the least
overlapping.
GRAPH COLORING: Given a graph, find a color-
ing of the nodes so that no two adjacent nodes have
the same color, and the total number of colors used
is minimized.
MAXIMUM CLIQUE: Given a graph, find the max-
imum subgraph all of whose points are mutually
adjacent.
Work on the above-mentioned BIN-PACKING pro-
blem [2,3,4] has suggested an approach that may be
applied to the others. Since no fast algorithm
for finding an optimal solution could be found,
simple heuristic algorithms that seemed likely to
generate fairly good packings were studied. It was
found that certain of these simple algorithms could
guarantee near-optimal results. For instance, the
FIRST FIT DECREASING algorithm guarantees that,
asymptotically, no more than 11/9 times the optimal
number of bins will be used.
This paper examines polynomial-time heuristic
"approximation algorithms" for the other optimiza-
tion problems listed above, in an attempt to get
similar results. We discover that a wide variety
of worst case behaviors are possible.
Some algorithms, such as the ones we present
for SUBSET-SUM and MAXIMUM SATISFIABILITY, have the
same type of behavior as FIRST FIT DECREASII~ in
that they guarantee that the solution they generate
will be no worse than a constant time the optimal,
and indeed that constant can be quite close to I.
 For other algorithms, such as the ones we mation algorithm for problem P, then the perform-
present for the two SET COVERING problems, there ante A(U)p of A for input u is defined
is no fixed bound on the ratio of the worst case
results to the optimal, but that ratio can grow A(U)p = WORST {mp(x): x 6 SOep(U) and x is
no faster than the log of the problem size. choosable by A on i n p u t
u},
For still others, such as some of the coloring
algorithms we give and all of the algorithms for
where WORST is M I N if BEST is MAX, and vice versa.
MAXIMUM CLIQUE that have been suggested, the ratio
As a measure of the relative worst case behavior of
grows at least as fast as O(ne), where n is the algorithm A on input u, we shall use the ratio
problem size and e > 0 depends on the algorithm.
rp(A,u) = Up*/A(U)p, if P is a maximization
We also consider the possibility of using the problem,
reductions which show that problems are polynomial
A(U)p/Up*, if P is a minimization
equivalent to translate our results from one optim-
problem.
ization problem to another Karp's reductions in
[5] are all between language recognition problems,
Note that this ratio has been defined so as to al-
but we can often extend them directly to the assoc-
ways be greater than or equal to I.
iated optimization problems. Difficulties arise,
however, since a "good" approximate solution for
For a given input u, the problem size lul is
one problem may map to a solution for the other
the number of symbols required to describe u in
which is not at all good according to the measure
some standard notation. Our measure of the overall
for that problem.
worst case behavior of algorithm A with respect to
In Section 2 we introduce a formal framework problem P will be a function R[A,P]:N+ ~ R defined
within which to discuss the above problems, defin- by
ing what we mean by "optimization problem",
"approximation algorithm", "worst case behavior", R[A,P](n) = MAX(rp(A,u): u 6 INPUTp and
etc. Each of the next six sections is devoted to
one of the problems, and finally Section 9 presents
some general observations and conclusions.
As defined, R[A,P] is an increasing function of n.

When there is no possibility of confusion, we

2. APPROXIMATION ALGORITHMS
will often delete the subscripts P from our notat-
ion, and let R[A] stand for R[A,P]. Of particular
An optimization problem P consists of
interest are algorithms A for which R[A] has a
fixed upper bound independent of n. Examples of
i. A set INPUTp of possible inputs,
these are given in the next two sections.
2. A map SOLp which maps each u 6 INPUTp to a fin-
ite set of approximate solutions, 3. SUBSET-SUM

3. A measure mp:SOLp(INPUTp) ~ Q defined for all This maximization problem is a simple form of
the well known "knapsack problem". We shall de-
possible approximate solutions.
note it by SS, and restate it in terms of our for-
mal definitions:
In addition, the problem P is specified as a
maximization problem or a minimization problem,
INPUTss = [<T,s,b>: T is a finite set, s:T-~
depending on whether the goal is to find an appro-
is a map which assigns to each
ximate solution with maximal or minimal measure.
x 6 T a size s(x), and b > 0 is a
For each u E IINPUTP, the optimal measure Up* is
single rational number].
defined by

Up* = BEST Imp(X): x E SOLp(U)}, SOLss (<T,s,b>) = (T' = T: ~ s(x) ~ b}.

xET'

where BEST stands for MAX or MIN depending on mss(T') = ~ s(x).

whether P is a maximization or minimization pro- x6T'
blem. Since SOLp(U) is finite, there must be at
The optimal measure for input <Tf,s,b> is (with sub-
least one solution x E SOLp(U) such that mp(x) =
scripts omitted)
Up*, and such a solution will be called an optimal
<T,s,b>* = max(m(T'): T' ~ T and m(T') ~ b }
solution.
We shall study a series of approximation algor-
An approximation algorithm for problem P, or
ithms [Ak: k ~ 11, where algorithm Ak is defined as
simply an algorithm, is any method for choosing
follows:
approximate solutions, given u 6 INPUTp. (Since
the algorithms we will study are not always com- I. Let SUB be that subset of (x E T: s(x) > b/(k+l)}
pletely determined, more than one solution may be whose measure is closest to, without exceeding,
ehoosable for a given input). If A is an approxi- b. Let SUM be this measure,&ndset L E F T = T - S U B .

39
2. If for all x E LEFT, s(x) + S U M > b, halt and For the lower bound on the limit, consider the in-
return SUB. put <T,s,b> where
3. Let y be an element of LEFT for which s(y) +
SUM is closest to~ without exceeding, b. T = ia I . . . . . ak+2} ,
4. Set LEFT = LEFT - {Y}, SUB = SUB U iY], SUM =
S U M + s(y). s(ai) = [I + E, for i = I,
5. Go to 2. l I, otherwise.

Remark. Algorithm A1 is closely related to b=k+l.

the FIRST FIT DECREASING algorithm for BIN-PACKING
[4], since the set chosen by it consists of those
elements which would be assigned to the first bin
if T were being packed into bins by that algorithm.
However, the BIN-PACKING results are apparently of
no use to us here, since they dealt with the number
of bins used, not the sum of the pieces in the first
bin.
The majority of the effort in these algorithms,
at least for k ~ 2, is concentrated in the first
step. Since the set found in this step has ISUBI
k, the step can be accomplished using at most
0(n k) comparisons and additions, where n = ITI, and
we do not know how to do any better. The Ak thus
are a series of seemingly increasingly expensive
algorithms. However, they yield increasingly better
results:
Theorem I. For k ~ I and n > 0,
R[Ak](n) ~ (k+l)/k, and
lira R [ A k ] ( n ) = ( k + l ) / k .
Clearly <T,s,b>* = k + I and Ak(<T,s,b>) = k + 6,
and thus r(Ak,<T,s,b>) = (k+l)/(k+6). Since E can
be made arbitrarily small by making the problem
size n large enough, we thus have I ~ R[Ak](n)
(k+l)/k and the theorem follows.
The above result shows that the SUBSET-SUM
optimization problem has the desirable property:
For any E > 0, there is a polynomlal-time algorithm
A_ such that R[AE] K i + E. Unfortunately, this
p~oblem and some minor variants are the only such
problems we have found.
In the next section we present a problem from
which we can obtain in a natural way a sequence of
sub-problems which are all polynomial complete.
We exhibit approximation algorithms BI and B2 for
~his problem with the property that for any E > 0
and i E 11,21, there exists a sub-problem Pi,E
with R[BI,Pi, E] ~ i + E.
4. MAXIMUM SATISFIABILITY

Proof. For t h e u p p e r b o u n d , s u p p o s e we a r e This maximization problem is not a naturally

g i v e n a n i n p u t < T , s , b > and T 1 = T i s c h o o s a b l e by occurring optimization problem, but instead was
derived from the language recognition problem
a l g o r i t h m Ak o n t h i s i n p u t . We a c t u a l l y p r o v e t h e called SATISFIABILITY in Karp's paper [5]. Al-
somewhat s t r o n g e r r e s u l t t h a t e i t h e r m(T1) = ~ r , s , b > * though such derived problems often seem to have no
o r e l s e m(T 1) m [ k / ( k + l ) ] . b . L e t T O be a n o p t i m a l practical applications, it is hoped that their
analysis can help us learn more about the nature
solution for the given input. We now p a r t i t i o n T. of polynomial complete problems in general. MAXI-
1
for i ~ i 0 , 1 ] as f o l l o w s : MUM SATISFIABILITY has been chosen as an example
= TBIG U _SMALL because our results illustrate another type of be-
Ti i Ti ' havior for approximation algorithms.

where T_BIG
i = {x E ri: S(X) > b/(k+l)} ' and r~iMALL = Let L = U Ixi,xi] be a set of literals. A
i>O
T. - ~BIG. We thus have, for i E [0,I}, clause will be any finite subset C c_ L. A truth
i 1 assignment will be any subset T c L such that for
SMALL. :n~ i > 0 is [xi,xl} c_ T. Truth assignment T sa___~-
m(T i) = m(T~ IG) + m(T i ).
isfies clause C if C N T ~ . The MAXIMUM SATIS-
FIABILITY problem, denoted by MS, is given by:
By Step I of algorithm Ak, m(T~ IN) ~ m(T~Ig).
INPUTMs = iS: S is a finite set (CI,C2,...,Cp}
If T SMALL
I D T~MALL we would also have m(T SMALL
I ) of clauses].
SMALL.
m(T 0 ), and so T 1 w o u l d be o p t i m a l . SOLMs(T) = IS' = S: there exists a truth
assignment T which satisfies every
On t h e o t h e r h a n d , if some x ~ T~
MALL- w e r e e x - clause C E S'}.
cluded from T SMALL
1 t h e n by S t e p 2 o f Ak, s ( x ) +

m(T1) > b, and s o

~Ms(S') = IS'I.

m(Tl) > b - s(x) ~ b - b/(k+l) = kb/(k+l) MS(k) will denote the sub-problem with inputs
restricted to sets of clauses, each clause of which
[k/(k+l)]<T,s,b>*. contains at least k distinct elements. MS(k) is
polynomial complete for all k ~ i.
Thus in all cases we have r(Ak,<T,s,b>)
(k+l)/k, and the upper bound is proven. Each of our algorithms will work by generating
a truth assignment TRUE, while keeping track of the

t0
associated set SUB of satisfied clauses. The first, 3. Let y be any literal occuring in both LIT and
Bl, is a simple heuristic requiring time propor- a clause of LEFT. Let YT be the set of clauses
2 in LEFT and containing y, YF the set of clauses
tional to n .
in LEFT containing y.
i. Set SUB = @, TRUE = @, LEFT = S, LIT = L. 4. If ~ w(C) ~ ~ w(C), set TRUE = TRUE U {y},
2. If no literal in LIT is contained in any clause C6YT CEYF
of LEFT, halt and return SUB. SUB = S U B U YT, LEFT = L E F T - YT, and for each
3. Let y he the literal in LIT which is contained C E YF, set w(C) = 2w(C).
in the most clauses of LEFT, and YT he the set Otherwise, set TRUE = TRUE U [y}, SUB = SUB U
of clauses in LEFT which contain y., YF, LEFT = LEFT - YF, and for each C E YT, set
4. Set SUB = SUB U YT, LEFT = LEFT - YT, TRUE== w(C) = 2w(c).
TRUE U {y], and LIT = LIT - [y,y), (where x. = 5. Set LIT = LIT - [y,y], and go to 2.
xi). i
5. Go to 2. Note that since the choice of which literal to
consider at step 3 is left largely undetermined,
Theorem 2. For all k m I and n > 0, there is room in this algorithm for the insertion
of heuristics which might improve its average case
R[BI,MS(k)](n) ~ (k+l)/k, behavior. Its worst case behavior is already a
perhaps surprising improvement on that of algorithm
Bl:
with equality for all sufficiently large n.
Theorem 3. For k m 1 and n > 0,
Proof. Suppose algorithm Bl is being applied
to a set S of clauses, all of which contain at
least k literals. Each time a literal is added to R[B2,MS(k)](n) ~ 2k/(2k-l),
TRUE, the number of clauses saved, i.e.~ added to
SUB, is by Step 3 at least as large as the number
of clauses remaining in LEFT which are wounded, with equality for all sufficiently large n.
i.e., have one of their literals removed from LIT
without being added to TRUE. When the algorithm Proof. Suppose algorithm B2 is applied to a
halts, the only clauses left in LEFT are t h o s e set S, all of whose clauses contain at least k lit-
which have been wounded as many times as they con- erals. Initially, the total weight of all the
tain literals, and hence are dead. This means clauses in LEFT cannot exceed !S~/2 k. During each
that when the algorithm halts there are at least iteration, the weight of the clauses removed from
k'ILEFT wounds, and hence ISUB I ~ k. ILEFT I. Thus LEFT is, by Step 4 of algorithm B2, at least as
S* ~ S = ISUBI + ILEFTI ~ [(k+l)/k].ISUB I The large as the weight added to those remaining clauses
upper bound follows. which receive new wounds. Thus the total weight of
the clauses in LEFT can never increase, and so, when
For the lower bound, we give an example when the algorithm halts, it still cannot exceed IS]/2 k.
k = 3. Similar examples can be constructed for any But each of the dead clauses in LEFT when the algor-
other k > 0. Consider the input ithm halts must have been wounded as many times as
it had literals, hence must have had its weight
doubled that many times, and so must have final
S = {{Xl,X2,X3~, [Xl,X4,X5],
weight = I. Therefore ILEFT I ~ ISI/2 k, and so
(x2,x6,x 7} ,
ISUBI ~ ISI(l-i/2 k) and the upper bound follows.
{X3,X8,X9} }
We again give a lower bound example for k = 3
which can be generalized for arbitrary k. Consider
Clearly S* = 4 since the truth asslgmnent T = [Xl,
the input
x4,x6,x8] will satisfy all four clauses. However,
each of the literals occurs exactly once, so that S = [[Xl,X2,X3}, {Xl,X4,x5 ],
Bl might choose TRUE = [Xl,X2,X3} , thus killing
the clause {Xl,X2,X3]. Hence BI(S) = 3 and so [Xl,X2,X3}, {Xl,X6,X7 ],
r(BI, S) = 4/3 = (k+l)/k. [Xl,X2,X3}, [Xl,X8,X9 },

Obviously this algorithm leaves much room for {*l,X2,X3 ], (Xl,Xlo,Xll}}

improvement. Since the worst case examples require
that all the wounded clauses eventually die, it
apparently could be to our advantage to have our where x I occurs in clauses with all possible com-
algorithm save as many of the wounded as possible.
binations of a literal from [x2,x2] and one from
Algorithm B2 incorporates this idea and still only
2 [x3,x3] , and Xl occurs in an equal number of clau-
requires time proportional to n :
ses, each filled out with new literals. S* = 8
I. Assign to each clause C E S a weight w(c) = since the truth assignment T = {Xl,X4,X6,Xs,Xl0 ~
IT
2 -ICI. Set SUB = TRUE = ~, LIT = L, and LEFT satisfies all 8 clauses. However, if algorithm B2
= So
chose literal Xl first, we would have ~ w(C) =
2. If no literal in any clause of LEFT is in LIT,
halt and return SUB. C6YT

41
1/2 ~ ~ w(C) = 1/2, and so Xl would have been with equality for all sufficiently large n.
C6YF
added to TRUE, with the consequence that one of the Proof. First we show that the bound is attain-
four clauses containing x I must eventually die. able for sufficiently large n. Let the set to be
covered, T, consist of k.k! points, as shown in
Thus B2(S) = 7, and so r(B2,S) = 8/7 = 2k/(2k-l). Figure i. F will consist of k! sets making up the

An interesting fact about the proof of Theorems
2 and 3 is that they are independent of the value
O f S*, and hence actually prove stronger results
than are given by the statements of the theorems.
The proof of Theorem 3, for instance, guarantees
that B2 will y l e l d a s e t s at least [~I(2k-I)/2~,
no matter what S* is, and the reader may verify
that S* may be as small as [!SI(2k-i)/2K].--
optimal subcover F 0, and k!o(~ (l/j)) additional
J=l
sets which will make up a subcover F I that CI might
choose. Divide T into k segments of k! points
each, labeled from I to k, as shown.
The optimal subcover Foconslsts of kl dis-
joint sets, each containing one point from each of
the k segments. The choosable

5. SET COVERING I

This section deals with a minimization prob-

[o T
lem, denoted by SC, from which we can again obtain
a natural sequence of polynomial complete sub- k! disjoint SEGMENT k disjoint
problems. However for this problem the restric- k! Points k-element
tion on the input is relaxed as the index k of the sets
sub-problem SC(k) increases, and although we have
an algorithm C1 such that, for each k, R[CI,SC(k)] k!
k-~ disjoint
is bounded by a constant independent of the size of SEGMENT k-I
k-element
the input, this constant can grow arbitrarily large k-~ element
k~ Points
as k increases. sets

In our format, the problem is described as

follows: sets with

INPUTsc = IF: F is a finite family [SI,S2,...,

Sp} of finite sets}.
one point
SOLsc(F) = {F' G F: U S = U S}.
S6F' SEF

msc(F' ) = IF'[.
per

If we consider the input F to be a finite cover

of the set T = U S, then an optimal solution to the k! disjoint
S6F SEGMENT I
segment 1-element
problem is a minimum cardinality subcover. This k! Points sets
problem has practical applications in areas such as
logic design and fault testing.
FIGURE I
SC(k) will denote the sub-problem with inputs
restricted to families, no set of which has more
than k elements. For k ~ 3, each of these sub- subcover FliS made up of k!/k disjoint k-element
problems is itself polynomial complete. Our al- sets forming a cover of segment k, k!/(k-l) dis-
gorithm CI is again a straightforward heuristic, joint (k-l)-element sets forming a cover of seg-
requiring time proportional to n and given by the ment k-l,..., and k! disjoint single element sets
following: forming a cover of segment i. CI could choose the
k!/k sets covering segment k first, since each will
I. Set SUB = ~, UNCOV = U S, N = IFI, cover k new points. After these have been chosen,
S6F no remaining set covers more than k-i new points,
SET(i) = Si, 1 K i ~ N. so CI could next choose the k!/(k-l) sets covering
2. If UNGOV = ~, halt and return SUB. segment k-l. This process could continue until Cl
3. Choose j ~ N such that !SET(j)I is maximized. had chosen the entire sub-cover FlWith its k l ( I / k +
4. Set SUB = SUB [J {Sj], UNCOV = UNCOV - SET(J), k
SET(i) = SET(i) - SET(j), I ~ i ~ N. I/(k-l) + ... + I) sets, and so r(Ci,F) ~ ~ (l/j).
5. Go to 2. J=l
For the upper bound, let us consider the oper-
Theorem 4. For all k ~ i and n > O, ation of the algorithm. The only internal variables
k which are interrogated when decisions are to be
R[Ci,SC(k)](n) ~ J~l (l/j) made are N, UNCOV, and SET(i), I ~ i ~ N. We shall

4e
thus define a configuration K of the algorithm to Suppose the Lemma holds for h = k-l. We shall
be a triple show it also holds for h = k. So let KI be a legal
configuration, MO be a set such that MAX[IKI[i]]:
~NK, UNCOVK, <Krl],K[2] ..... K[NK]>>, i e MO] = k and U Kl[i] = UNCOVKi, and suppose
i6M0
the set M1 is selectable from KI. Since M1 is
where N K is a value for N, UNCOV K is a value for selectable, there is a run R = <~[l,Jl,K2,...Jt_l,
UNCOV, K[i] is a value for SET(i), i ~ i ~ NK, and Kt> such that M1 = numbers(R). Let r be the index
of the first ]i in the run for which IKirJi] I < k,
the values satisfy N~ K[i] = UINIOVK.
i=l that is, the first Ji for which- ISET(Ji) I was less
A run R from a configuration K will be a se- than k when Ji was chosen, (if none exists, let
quence r = t).
R = < KI,Ji,K2,J2 .... K(t-l),Jt_l,Kt> If we let MI' = {Ji: i < r] and COV = UNCOVKi-
UNCOVKr , then, since every j chosen prior to Jr
of alternating configurations and integers such had ]SET(j)I m k at the time it was chosen, we must
that KI = K, UNCOVKi ~ ~, i < t, UNCOVKt = ~, and have
such that if the algorithm enters Step 2 in con-
figuration Ki, it can choose Ji at Step 3, and the IMI'I ~ Icovl/k. (5.1)
resulting configuration, after choosing Ji and up-
dating at Step 4, is K(i+l). We may think of a run Now let us turn to configuration Kr. The set
as a possible pass through the algorithm, and say Mr = fJi: i ~ r} = M1 - MI' is selectable from Mr,
for instance that when Ji was chosen, the value of since R' = <Kr,Jr,...,Jt_l,Kt> is a run from Kr,
SET(Ji) was Ki[ji]. inheriting this property from R. Furthermore,
consider the sets Kr[i], i c M0. None of these
Note that all the Ji's in the sequence must
sets can have more than k-I elements, since Jr can
be distinct, since once Ji is chosen, SET(Ji) is be chosen from gr at Step 3, and so for all i
set to @, and so Ji cannot be chosen again. Let i ~ NKr = Nkl, |Kr[i]I ~ IKr[Jr] I < k. Moreover,
numbers(R) be the set of the Ji's in the run R. since Kr[i] = Kl[i] - COV, we have that
We say that a set MI of integers is selectab~e from
U Kr[i] = [J KI[i] - COV = U~OVKr
configuration K if there is a run R from K such i~0 i~0
that Mi = numbers(R). The following lermna is an
obvious consequence of these definitions: Thus the induction hypothesis for k-I applies
to Kr, Mr, and M0, and we have
Lermna I. Suppose F I ~ F is a subcover and
MI = [i: S i e Fi]. Let K be the configuration of IKrrill
C I after it has been given input F and initialized IMrl ~ ~ (~ (i/j)). (5.2)
itself via Step I. Then FI is ehoosable by CI i6MO J=l
given F if and only if MI is selectable from K.
The desired upper bound will be a consequence Now for each i 6 M0,
of a second, less trivial, lemma:

Lenmm 2. Let K be any configuration, MI be IKl[i]l IKrri]! IKl[i]I

any set selectable given K, and M0 be any set such (~/j).
that U K[i] = UNCOV K. Then I (l/j) = ~ (l/j) + I
i6MO |Kri]I J:l J=l I~rrt]l+l
IMll~ I ( Z (l/j)). Since Krri] c Kl[i], and IKlri]l ~ k for i MO by
i6M0 J=l
assumption, and since (l/j) ~ (l/k) for 1 ~ j ~ k,
the above quantity is thus
Proof of Lenmm. We proceed by induction on
h = MAX{IK[i]I: i e MO}. The Lemma holds for h =
0, because if no Kill, i e M0, contains any points, IKiri]l
and yet U K[i] = UNCOVK, we must have UNCOV K = (I/j) + IKl[i]-Kr[i]I.(1/k).
6M0
J=l
~, and hence for a,ny MI selectable from K
Then, since ~ IKlri]-Krri] I ~ [ U (KI[i]-
i6M0 i6M0
i6MO J=l
Kr[i])I = IUNCOVKi- UNCOVKr ] = ]COVI, we have

since the inner sums on the right are sums of 0 terms.

43
 Since k can be taken to be arbitrarily large,
it follows that R[Ci,SC] must be at least 0(in(n)).
IKl[i][ IKr[i]l

iCMO J=l iEMO J=l T: 3"2 k POINTS

IM*'I + I~I = IMll,

by (5.1) and (5.2). Thus the le~mna holds for h = One Set
k and, by induction, is proved.
3.2 k'l
To conclude the upper bound proof: Let F be Points
any family, no set of which contains more than k
elements, F 0 an optimum subcover, and F 1 a choos-
able subcover with CI(F) = IFiI. Let K be the con-
figuration of C1 after it has been given input F
and initialized itself via Step I. Let M0 = {i:
S i e F0} and M1 = [i: S i e FI]. Since F 1 is choos- One Set
able given F, M1 must be selectable from K, by 3.2 k-2
Lemma I. Moreover, since F 0 is a suhcover, Points

[J K[i] = UNCOV K. Thus Lenmna 2 applies and we

i6MO One Set
have 3.2 k'3
L Points
el(F) = IF11 = IMII r

IK[i]I lSl >

i6MO J=l S6F 0 a=l I Set I Set I Set

k k 2k P t s 2k P t s 2k P t s

SEF 0 J=l J=l FIGURE 2

and the upper bound follows.

As a consequence of Theorem 4, we can con-
clude that, for the general problem SC with no re-
striction on inputs, R[CI,SC](n) ~ in(n) + I,
where n is the size of the input. This is because
if it takes n symbols to describe the family F,
surely no set in F can have more than n elements,
and
k k
(l/j) <l+ln(k) < ~ (l/j) +
J=l J=l
That R[CI,SC] actually is 0(In(n)) can be seen
from the generic example given in Figure 2 [6].
Here T consists of 3"2 K points.
cover F 0 consists of 3 disjoint sets of 2K points
each, so F* = 3. The subcover Fi found by CI how-
ever conslsts of k+l sets of size 3.2 K'I, 3.2 K'2,
..., 3.4, 3.2, 3, and 3 respectively. Since F can
be described by giving a bit vector of ler~th ITI
for each element of F, we may say n ~ IFI'ITI =
(k+4)3.2 K, and so log2(n) ~ k + log2(k) ~ k.
r(CI,F) = (k+l)/3 ~ (log2n)/3 ~
As an example of how results from one poly-
nomial complete optimization problem can be dir-
ectly applied to another if the reduction between
the two is simple enough, we now briefly consider
the problem NC of finding a minimal node cover.
This minimization problem has inputs which are
graphs G = (N,A), and approximate solutions are
subsets N' = N of the nodes such that for each arc
in A, one of its endpoints is in N'. The measure
is m(N') = [N'I. NC(k) refers to the subproblem
1/2. with inputs restricted to graphs, no node of which
is connected to more than k other nodes.
NC is equivalent to SC with inputs F restric-
ted so that every point in U s is in precisely
S6F
two sets of F, for in this case the points may be
made to correspond to arcs of a graph and the sets
The optimal to nodes. Since our examples in Figures 1 and 2
obeyed this restriction, all our lower bound re-
sults for SC and SC(k) apply to NC and NC(k). (The
upper bounds apply because NC is a subproblem of
SC).
However, if we still further restrict the in-
puts, and let SC'(k) and NC'(k) be the problems
where exactly k elements are in each set of the
Thus input family F, then although the lower bound re-
suits for SC(k) can still be shown to hold for
(.48)(In(n)). SC'(k), we have for NC'(k) an entirely different
and stronger result:

44
 Theorem 5. For all k ~ I and n > O, Surprisingly, however, there exists an algorithm
C2 which, when applied to problem EC, will have be-
havior quite similar to that of CI applied to SC.
R [ C 1 , R c ' ( k ) ] ( n ) ~ 2,
C2 also requires time proportional to n 2 and is
described as follows:
and for all sufficiently large n, R[CI,NC'(k)](n)
(2k-l)/k. I. Set SUB = , LEFT = F, UNCOV = U s.
SEF
The upper bound given here is better than the 2. If UNCOV = , halt and return SUB.
one guaranteed by the SC(k) result for all k ~ 4. 3. Let S' LEFT be that set S whlch minimizes
However, note that if all points of T occur in pre- Ratio(S) = Is - UNCOVl/IS n U N C O V l
cisely two sets of F and all sets have precisely k 4. Set SUB = SUB U (S'}, UNCOV = UNCOV - S',
elements, IF I = (2[T!)/k = 2(Irl/k ) ~ 2F*. Al- LEFT = LEFT - (S').
though this constitutes a short proof of the the- 5. Go to 2.
orem's upper bound, its major significance is that
it shows that the algorithm which simply says, Theorem 6. For all k ~ I and n > 0,
"Given input F, return F," can guarantee this same k
upper bound without doing any work at all! R[C2,EC(k)](n) K I + in(k) ~ ~(I/j) + 1/2,
J=l

6. SET COVERING II and for all sufficiently large n, R[C2,EC(k)](n)

k
The following set covering problem, although y, (l/j).
differing from SC only in the measure chosen, does J=l
not seem to be directly reducible to it. It is a
minimization problem based on the EXACT COVER re- Proof. The lower bound follows from an example
cognition problem presented in [5], which asks much llke that given in Figure I, except that all
whether an input family F has a disioint suhcover. sets in F I are filled out with points from segment
Our optimization problem will hence be denoted by k so that each has exactly k elements.
EC:
For the upper bound, let F be any input, all
INPUTEc = IF: F is a finite family (SI,$2,..., of whose sets have k or fewer elements, and let F 0
Sp} of finite sets}. be an optimal subcover. If we set T = [J S, we
S6F
SOLEc(F) = [F' = F: U s = U s}. then have m(F O) = F* = aITI, for some a, I ~ a ~k.
S6F' S6F
Now suppose F I is choosable by C2 on input F,
mEc(F'> = Z Isl.
S6F' and let us restrict our attention to some particu-
lar run of C2 during which ~ is chosen. For each
S Fi, let the overlap ov(S) be the value of
An optimal solution is a subcover of the set
T = U S with the least possible overlapping. IS-UNCOVl when S was added to SUB by C2. The
S6F cumulative overlap OV(Fi) is defined to be ~ov(S),
EC(k) will be the sub-problem with inputs restric- SEF I
ted to families, no set of which contains more than and we have
k points. For k ~ 3, each of these sub-problems
is again polynomial complete. m(Fl) = !T I + OV(Fi).

Now if all the sets in the family F had

exactly k elements, then for each subcover F' we Now let us consider the action of C2 on F, and
would have mEc(F') = k.msc(F') , and the optimal its relation to the sets of the optimum cover F 0.

solutions for both SC and EC would be the same.
However, for less restricted inputs, the two pro-
blems diverge, and optimal solutions for one can
be bad for the other. For instance, if OSC is an
algorithm which always chooses an optimal solution
for the SC problem, then (proof omitted):
lim R[OSC,EC(k)](n) = k.
If at a given time a set S' is chosen with RatiOS')
= IS' - UNCOVl/Is' N UNCOV I ~ y, then at this time
each S F 0 can have at most I/(y+l) of its points
in UNCOV. For if S e F 0 N SUB, S has n__opoints in
UNCOV; otherwise we must have Ratio(S) ~ Ratio(S')
y, so IS O UNCOVI/Is I ~ I/(y+l). Thus

Iu~c0vl ~ z Isn uNG0vl

R[OSC,EC] = O(n). S6F 0

Conversely, an algorithm OEC which always chooses
an optimal solution to the EC problem will have
reciprocal inapplicability:
R[OEC,SC(k)] = k, for all sufficiently largen.
R[OEC,SC] = O(n).
F*/(y+l) = aIrl/(y+l ).
And therefore, at least ITi(l-a/(y+l)) points, and
quite possibly more, must be outside of UNCOV, i.e.,
already covered. Thus we cannot choose an S' with
Ratio(S') ~ y until l-a/(y+l) of the points of T

45
have been covered. Conversely, if x = IT-UNCOVI/
ITI, then Ratio(S') for the next set S' chosen can
not exceed a/(l-x)-l.
Ratio(S') gives, for each new point covered
by the set S', the number of points in the set
which overlap points already covered. This fact
leads to interpretation of OV(Fi) as the area under
a curve. See Figure 3.
We divide the X-axis into a sequence of inter-
vals of length I/IT I corresponding to the points of
T, laid out along the X-axis in the order the points
were covered by the algorithm. The Y-value for
each interval is the number of overlaps contributed
by the corresponding point, as measured by RatiOS')
for the first set S' that covered it. The function
f so defined is an increasing step function on the
interval (0,I), with maximum value k-l, and
OV(FI) = IT I ~f(x)dx.
0 is a modification of that given in Figure 2, where
Y bring its total cardinality up to 3.2 k'l.
[ -MIIi!ii fill "X
0 1.0
FIGURE 3
Now an X-value in the gr@ph can be thought of
as representing the value of IT-UNCOVI/ITI at the
time the point was chosen which corresponds to the
interval containing x. Thus by our reasoning a few
paragraphs above, f(x) ~ a/(l-x)-l, for all x c
(0,I). Thus an upper bound on OV(Fi) will be ITI
times
l-a- I number of colors.
a
~J~[ I ~ x - l]dx + a~ [k-l]dx,
where we have replaced g(x) = a/(l-x)-I by k-I for
x > 1-a/k, since we have g(x) > k-i for such x,
even though f(x) never can be. Transforming and
evaluating, we find that the above equals
alj~ adz . [Z]o1 - a/k + (k-1)~
z
46
= a[0-1n(a) + In(k)] - I + a/k + a - a/k
a[In(k) + I] - I.
Thus OV(F I) ~ IT1(a[In(k) + I] - I), and so
m(Fl) ~ Irl + Irl(a[In(k) + I] - I)
= alTI[In(k ) + I] = F*[in(k) + I].
The upper bound follows.
Thus, despite the differences in measures and
the differences in algorithms, our results for
R[C2,EC(k)] are practically the same as the results
for R[CI,SC(k)]. Continuing the parallel, we also
have R[C2,EC] = 0(In(n)). The lower bound example
F 0 is as before, and each of the smaller sets of
has points added to it from the largest one to
Thus
C2(F) = (k+l)(3"2k-l), and so r(C2,F) = (k+l)/2 =
0(In(n)), where the problem size n is the same as
in the original example.
7. GRAPH COLORII~
The results of the next two sections are neg-
ative in character, as we can show only lower
bounds on the worst case behavior of the approxim-
ation algorithms we study. However, these lower
bounds will be sufficiently large to serve as a
warning that such simple heuristics may well have
drawbacks. In our format, GRAPH COLORING, denoted
by GC, is a minimization problem given by the
following:
INPUTGc = (G = (N,A): G is a finite undirected
graph with nodes N and
arcs A].
SOLGc(G) = (h: N ~ [1,2 ..... INI}: if there is
an arc between nodes x and y,
h(x) ~ h(y)}.
mGcCh) =][z: z = h(x) for some x e ~1"
An approximate solution can be interpreted as
a coloring of the nodes of the graph, such that no
two adjacent nodes have the same color. A n optimal
solution is a coloring with the minimum possible
If k ~ G*, we say that G is
k-colorable. This problem is often called the
"timetabllng problem", and occurs in practical sit-
uations such as scheduling exams. Welsh and
Powell [7] and Wood [8] have described what are
essentially approximation algorithms for the prob-
lem, although they did not analyze their worst case
behavior. The Welsh and Powell algorithm, which 2
we shall call DI, requires time proportional to n
and is the same sort of simple heuristic we have
studied above. It constructs h as a set H of
ordered pairs in N X Z:
I. Set DOM = CHROM = H = ~, COLOR = I, REST = N.
2. If DOM = N, halt and return H.
3. If each node in REST is connected to some node
in CHROM, set COLOR = COLOR + I, CHROM = @.
4. Let x be a node in REST not connected to any
node in CHROM.
5. Set H = H U [(x,COLOR)}, CHROM = CHROM U [x},
DOM = DOM U Ix], REST = REST r Ix].
6. Go to 2.
This algorithm behaved reasonably for Welsh
and Powell on the small examples they considered;
however, there is a sequence [Gk} of 2-colorable
graphs such that graph G k has only 2k nodes but
DI(Gk) = k. Figure 4 shows G 3. Graph G k = (Nk,Ak)
has nodes and arcs:
N k = [a I ..... ak,b I ..... b k] ,
Ak = [(ai,bj): i ~ j}.
aI a2 a3
b1 b2 b3
FIGURE 4: G 3
G k has 2-coloring h(ai) = I, h(bi) = 2, I
i ~ k. Algorithm D ~ however, could color the nodes
in the order al, bl, a2, b2, ..., ak, bk, and so
set h(a I) = h(bl) = I, h(a2) = h(b2) = 2, .... and
h(a k) = h(b k) = k, thus using k colors.
r(DI,Gk) = k/2. Since the number of arcs in G k is
proportional to k 2, and we can describe a graph by
listing its nodes and arcs, we thus can conclude
that R[DI] ~ o(nl/2).
One obvious way to try to improve this algor-
ithm is to give more specific rules for choosing x
in Step 4. For instance, consider algorithm D2
which replaces Step 4 in the above by:
4. Let POSS be that subset of REST made up of all
the points not connected to any member of
CHROM. Let x be the node in POSS connected to
the fewest other nodes in POSS.
It is easy to see that RID2] ~ O(logn). There
is a sequence [Hk} of 2-colorable graphs such that
H k has 2k points and D2(Hk) = k+l.
H I thru H 4. Hk+ I is obtained from H k by adding 2 k
new points and arcs, each new point connected by a
new arc to a different one of the 2k points of H k.
The new points are the circled points in the Figure.
Algorithm D2 applied to Hk+ I would choose these new
points first, coloring them all the same color, and
then would have to start with a second color on the
remaining graph, which is just H k. Since H I re-
quires two colors, we thus can conclude that D2(Hk)
= k+l, r(D2,Hk) = (k+l)/2, and R[D2] ~ O(log2n ).
HI H2 H3 H4
FIGURE 5
In fact, these examples can be improved upon,
so that the size of the graph requiring k+l colors
grows more slowly than a k for any a > I (but faster
than any polynomial in k), and so the above in-
equality is a strict one. We can also show that
the Wood algorithm [8] is at least 0(logzn), and of
course, since we have no upper bound proofs, both
algorithms may be considerably worse.
However, one algorithm, which we shall call
D3, has been proposed for which R[D3] definitely
i__s 0(log n). Let an independent subset of the nodes
of G be any subset, no two nodes of which are ad-
jacent in G. D3 can be described as follows:
I. Set NODES = N, i = I, H = ~.
2. If NODES = @, halt and return H.
3. Let IND be the maximum sized independent sub-
set of NODES.
4. Set H = H U [(x,i): x IND], NODES = NODES -
Hence IND, i = i+l.
5. Go to 2.
D3 is really only our set covering algorithm
Ci in disguise, applied to the cover of N by the
family of its independent subsets, and since an
optimal coloring corresponds to a minimum cardin-
ality subcover of this family, our upper bound re-
suits for SC apply, and tell us that R[D3,GC] s
0(log n). The graphs H k given in Figure 5 can be
used to show that equality holds.
Unfortunately, it is unlikely that D3 can be
implemented in polynomial time. Step 3 is equival-
ent to the problem of finding the maximum sized
clique in a graph, and hence is itself a polynom-
ial complete problem. And note the results of the
following Section.
Figure 5 shows
8. MAXIMUM CLIQUE
For this maximization problem, we have not
found any polynomial-time algorithms which even
might be as good as 0(log n). Denoting the problem
by MC, we have:
47
 is of size 2. Thus r(E2,G) = k /2, and again
INPUTMc = (G = (N,A): G is a finite undirected R[E2] ~ o(nl/2).
graph with nodes N and
arcs A].

SOLMc(G) = [N' c N: there is an arc in A be-

tween each pair of nodes aI a2 ak
in N' } .

mMc(N'> = {N'{.

The approximate solutions are the cliques of

the graph. An optimal solution is a maximum sized bI b2 bk
clique. A straightforward heuristic for this pro-
blem, requiring time proportional to n 2, is algor-
ithm E1 given below; FIGURE 7

i. Set SUB = ~, REST = N.

2. If REST = @, halt and return SUB.
3. Let y 6 REST be that element connected to the
most other elements of REST.
4. Set SUB = SUB U [y~,
REST = REST-[points not connected to y].
5. Go to 2.
Figure 6 shows an example of a graph with 2k+l
points, k of which form a clique of size k, for
which algorithm E1 will find only cliques of size 2.
P would be chosen first as it is the only point
connected to k other points, but this foredooms the
eventual clique found to have just 2 points. Thus
r(Ei,G) = k/2. Since the graph has proportional
to k 2 arcs, we thus have R[EI] ~ 0(n I/2)
Minor attempts at modifying El and E2 may
avoid the pitfalls of the particular examples we
have given, but to date every proposed n -time al-
gorithm E based on such a modification has been
shown to also have R[E] ~ 0(nl/2).
The only way to (apparently) improve on this
lower bound is to use algorithms requiring more
than n 2 time. For instance, we might try to im-
prove algorithm E1 by adding back-tracking on a
large scale, much as we improved algorithm AI in
Section 3. Let Fj be the algorithm which enumer-
ates all the cliques of size j in the graph G, and
then runs algorithm E1 once for each one, each time
initializing SUB to the clique in question. F
then returns the largest of the cliques found 3by

E1 in this process.
PI
Our best lower bound on R[FJ] is 0(nl/j+l).
P2
Figure 8 shows a graph with N k j+l points (k >> j),
k of which form a clique of size k, for which FI
will only find a clique of size j+2. For each sub-
clique J of size j in the k-clique, there are k
P0 Fk additional points PJ,I thru PJ,k" PJ,I and the
nodes of the clique J are mutually connected to the
k-I other Pj,i's. Since each point Pi outside of
FIGURE 6
J but still in the k-cllque is only mutually con-
nected with the nodes of J to k-j-I other points,
algorithm E1 started on SUB = J will next choose
The alternative approach of starting with N PJ,I and thus have to settle for a clique of size
and deleting nodes until a clique remains is just
J+2. Thus r(FI,G) = k/(j+2). Since there are N k j
as bad. Consider algorithm E2:

I. Set SUB = N.
2. If SUB is a clique, halt and return SUB. PJ,I
3. Let y e SUB be the node connected to the fewest
other nodes in SUB.
4. Set SUB = SUB - [y] and go to 2. ~ ~ , PJ, 2

Figure 7 shows a graph with 3k points, k of

which form a clique of size k, for which E2 will
choose a clique of size 2. The right-hand compon-
ent is much like the graph G. we presented in the
previous section; each of th~ k points in the top
line is connected to each of the k points in the
bottom line, but no two points in the same line subclique PJ ,k
are connected. Since each of the points in the k- of size J
clique is connected to only k-I other points, the
first k points deleted by E2 will be the points of
the k-clique, leaving a graph whose largest clique FIGURE 8

48
subcllques of size j in a clique of size k, our 5. R.M. Karp, "Reducibility among combinatorial
graph has N k j+l points, thus our example yields problems", COMPLEXITY OF COMPUTER COMPUTATIONS,
R.E. Miller and J.W. Thatcher, eds. Plenum Press,
the lower bound R[Fj] o(nl/j+l). New York, 1972.

The cost for this apparent improvement in 6. J.H. Spencer, private co~nunicatlon.
worst case behavior is a running time proportional
to n j+2. If we wish to further increase our ex- 7. U.J.A. Welsh and M.B. Powell, '%n upper bound
pense, we could modify E1 so that in Step 3 it to the chromatic number of a graph and its
looked for a clique of size j instead of a single application to time-tabling problems", The
point. This however, would only lower the constant Computer Journal, Vol I0 (1967), p. 85.
of proportionality in the lower bound, as can be
seen by substituting a clique of size j for each of 8. D.C. Wood, "A technique for coloring a graph
the points PJ,i in the above example. applicable to large scale time-tabling problem,S
The Computer Journal, Vol 12 (1969), p. 317.
We have found n_qpolynomlal-tlme algorithm E
for this problem for which there does not exist an
> 0 such that R[E] > 0(nE), and no way to reduce
the ~ without significantly increasing the algor-
ithm's cost.

9o CONCLUSION

The results described in this paper indicate

a possible classification of optimization problems
as to the behavior of their approximation algorithms.
Such a classification must remain tentative, at
least until the existence of polynomial-tlme algor-
Ithms for finding optimal solutions has been proved
or disproved. In the meantime, many questions can
be asked. Are there indeed 0(log n) coloring al-
gorlthms? Are there any clique finding algorithms
better than O(n E) for all @ > O? Where do other
optimization problems fit into the scheme of things?
What is it that makes algorithms for different
problems behave in the same way? Is there some
stronger kind of reducibility than the simple poly-
nomial reducibility that will explain these results,
or are they due to some structural similarity be-
tween the problems as we define them? And what
other types of behavior and ways of analyzing and
measuring it are possible?

ACKNOWLEDGMENTS

The author wishes to thank Professors Michael

J. Fischer and Albert R. Meyer for their encourage-
ment and suggestions as to the presentation of
these results.

BIBLIOGRAPHY

I. S.A. Cook, "The complexity of theorem-provlng

procedures", PROCEEDINGS 3rd Annual ACM Sympo
on the Theory of Computln~.

2. M.R. Garey, R.L. Graham, and J.D. Ullman,

"Worst-case analysis of memory allocation
algorithms", PROCEEDINGS 4th Annual ACM S~mp.
on the Theory of Computln~.

3o D.S. Johnson, "Fast allocation algorlthms",

PROCEEDINGS 13th Annual IEEE Symp. o__n Switching
and Automata Theory

4. D. So Johnson, "Near-optlmal bin packing algor-

ithms", Doctoral Thesis (in preparation).

49