1962

IRE

TRANSACTIONS

ON INFORMATION

THEORY

203

A New Upper Bound for Error-Correcting

SELMER M. JOHNSONt
Summary-By refining Hamming s geometric sphere-packing model a new upper bound for nonsystematic binary error-correcting codes is found. Only combinatorial arguments are used. Whereas Hamming s upper bound estimate for e-error-correcting codes involved a count of all points <e Hamming distance from the set of code points, the model is extended here to include consideration of points which are >e distance away from the code set. The percentage improvement from Hamming s bounds is sometimes quite sizable for cases of two or more errors to be corrected. The new bound improves on Wax bounds in all but four of the cases s he lists.2

Codes*

The results for these sphere-packing models are compared in Table I. It is seen that the new estimate improves on Wax results in all but four cases. s

TABLE

COLLECTEDRESULTSFOR UPPERBOUND ESTIMATES

e I 16 18.2 7 8 :i 28.4 20.3 8 25.6 20 51.2 39.7 12 51.2 El 82.2 2.9 68 170.7 154.8 17* lG0 128 315.1 346.8 2%.5 256 585.1 806 5iE.l 512 1092 1913 10;: 1024 2048 4656 35 2048 2048 38G0 11600 36% 2048 7280 29600 44% 7090 4096 2 4.4 3.1 ii.0 2 6.9 5.3 2 4.6 A 9.2 6 6 18.3 16.6 6 13.4 E.6 26.5 8: 2.8 46.7 K3 ii.0 85.1 19* 70 32 154.6 160 * 1:: 64 270.8 309.8 42 256 128 478.4 617.3 4ii.3 128 851.1 1264 68 851.1 256 3 2 2.1 1 ; 2.8 3 ;.3 i.9 4.2 i.9 2 5.8 6.1 i.8 9.8 8.6 2 5.3 4 13.7 12.7 3 9 5 4 21.7 19.3 li.3 2.9 30.1 2t.9 E.9 48.9 ii.2 32 94.0 82.0 22 68.3 32 157.2 140 37* 118 F-l 2 2.7 1 2 2 2.7 3.4 1 2.2 i.6 4.3 1 2.8 5.2 5.7 k.6 2 7.5 7.2 1 4.8 2 11.1 9.8 5.9 4 16.9 13.8 3 11.e 4 26.0 24.2 4 17.2 4i.8 37.5 6 26.0 8 2 2.8 1 i 2.6 3.3 B 2 3.4 4.0 1 2.7 2 4.7 4.9 i.4 i.6 6.2 1 4.5 2 9.5 8.8 1 6.1 2 13.9 12.1 2 9.9 4 4 5

6l

E SHALL ASSUME the reader is familiar with the standard references on error-correcting w codes. This paper is concerned with developing new upper bounds on the size of nonsystematic binary error-correcting codes. ,4 geometric statement of the problem to be considered is as follows: Find t.he maximum subset of vertices of an n-dimensional unit cube such that any two vertices in this subset are at least Hamming distance d apart, i.e., the distance is measured along the edges of the cube. For e-error-correcting codes d = 2e + 1. These vertices are the code points which Hamming surrounds by spheres of radius e. He counts t,he vertices which are < e Hamming dist)ance away from a code point, i,e., inside the sphere. Dividing 2 by this count gives an upper bound to the size of an e-error-correcting binary code of sequences of n symbols. Wax on t,he other hand applies a continuous sphere-packing model with a variable density function to obtain upper bounds which improve on Hamming for certain ranges of n and e. His calculations are quite involved and were carried out on an electronic computer. The present paper goes back to the discrete point model of Hamming and extends the point count to include lower-bound estimates of the number of points which are outside the space occupied by the Hamming spheres packed in the n-dimensional cube. While this method can be described formally for sphere-packing problems for any mebric distance, the special properties of the Hamming metric lead to useful results using only elementary combinatorial arguments and hand computations.
* Received by the PGIT, June 12, 1961; revised manuscript received, August 15, 1961. Any views expressed in this paper are those of the author, and should not be interpreted as reflecting the views of the RAND Corporation or the official opinion or policy of any of its governmental or private research sponsors. t The RAND Corporation, Santa Monica, Calif. 1 R. W. Hamming, Error detecting and error correcting codes, Bell Sys. Tech. J., vol. 29, pp. 147-160; April, 1950. 2 N. Wax, On upper bounds for error detecting and error correcting codes of finite length, IRE TRANS. ON INFORMATION THEORY, vol. IT-5, pp. 168-174; December, 1959.

12

13

2 2.9 B 2 2.5 3.3 1 2.2 i.3 3.9 A.6 2 4.4 4.7 1 3.3 ii.0 5.9 :.2 2

16

H = Hamming bound for A(n, d). W = the Wax soft-sphere bound for A(n, d). R = the best bound for R(n, d, e), using (5), (6) and (7). J = the bound for A(n, d) using (2), (5), (6) and (7). N = the best code size actually found. * Improved values of ZZ for these cases. See Appendix.

204

IRE

TRANSACTIONS

ON INFORMATION

THEORY

April

There are other methods usually giving better upper bounds such as Plotkin s, but they apply only to cases where n < 2d or sometimes slightly more. It should be not that the present method could be combined with ed special arguments to give improved results for many cases where n < 2d, but these arguments depart from the main development of our sphere-packing model, and will not be exploited at this time. Asymptotically the present method improves over all methods known to the author in a significant way for large n and e in the general region roughly where 4e < n < e2 along with smaller improvements as n exceeds e These asymptotic estimates will be presented . in a second paper.
DEVELOPMENT
OF THE NEW BOUND

one vector has a one while the other has a zero. This simple relation will be used repeatedly in our investigation. Our first improvement over Hamming bound is as s follows : Theorem 1:

Ah 4

While the motivation of this paper is geometric, the techniques are mainly algebraic. The algebraic statement of the problem follows: Let A(??., d) = the maximum number of rows in a matrix of n columns with elements of zeros and ones, with the property that any two row vectors differ in at least d places. We seek improved upper bounds for A (n, d). As is well known this function is the maximum size of an e-error-correct,ing binary code of length n, where d = 2e + 1. We shall need some definitions of auxiliary functions. Consider a vector of m elements of zeros and ones, where r is the weight of the vector, i.e., the sum of its coordinates or the number of ones in it. Then let R(m, r, X) = the maximum number of such vectors of order m and weight r such that the inner product of any pair of row vect ors in 5 X. Thus any two vectors must not have more than X ones in the same positions. For example it can be checked that R(5, 3, 1) = 2. The vectors (1, 1, 1, 0,O) and (0; 0, 1, 1, 1) would be such a pair. Our upper-bound estimate for A(n, d) will depend critically on the function R(m, T, X) which is closely related to work in block designs in combinatorial mathematics (see Ryser4) but differs from the usual definition in that any two rows need not have exactly X ones in common but rather at most X ones in common. Two elementary upper-bound estimates for R(wz, r, X) will be presented in this paper. Further study of this function is being pursued. For two vectors identified by subscripts i, j we have the relation ri + ri = 2Xi; + dgi (1)

where the standard notations for combination symbols and integral value symbols are used here and throughout the paper. Our proof is motivated by a refinement of the Hamming geometric model. Let E, be the set of 2 vertices of the n-dimensional unit cube to be divided into disjoint subsets as follows : Let S, = the desired maximum subset of vertices at least Hamming distance d apart, S, = the subset of vert,ices at distance k from the set S,. Then E, = S, + S, + ... S,-,. (3)

For if t)here is any point in S, for k 2 d, it should be transferred to the set S, which is to be made as large as possible. Recall that the Hamming upper bound considered only the point sets S,, S,, . . . , S,. We improve the Hamming result first by considering in addition the set S c+l. Further extension to the sets S, for k > e + 1 will be discussed later in this paper. To estimate a lower bound for the number of points in S,+, we consider any chosen point P in S,. We translate this point to the origin by proper choice of coordinate axes; geometrically, by rotating the unit cube. This is accomplished algebraically by changing the entry of the column of the matrix of coordinates wherever a one appears in the coordinate vector of P. This clearly does not affect the Hamming distance between any two vertices. We concentrate on the neighborhood of t,his origin code point P, = (0, 0, . . . , 0) in S,. In the unit cube there are z vertices of weight Ic and therefore of distance 0 L from P,. The set of such points form a hyperplane W,. Any point in Wk thus belongs to some Xi, for i 5 k. Code points of weight r will be denoted by P, where P, is in W, A S,, while noncode points of weight r will be denoted by QV where Q7 is in W, A Xi, for some i > 0. Let c(X) = the cardinality (point, count) of a point set X. Then the number of points in both S, and lVd is c(W, r\ S,) 5 R(n, d, e) from (1). If two points are in S, and W, then ri = ri = d and thus dii must be even. Since d is odd, we must have

where ri and ri are the row sums, xii is the number of positions where there are ones in both vectors, i.e., the inner product, and d<i is the number of positions where

3 M. Plotkin, Binary codes with specified minimum distance, IRE TRANS. ON INFORMATION THEORY, vol. IT-6, pp. 445-450; September, 1960. 4 H. J. Ryser, Matrices of zeros and ones, Bull. Amer. Math. Sot., vol. 66, pp. 442-464; November, 1960.

1966

Johnson: Upper Bound for Error-Correcting

Codes

205

dij 2 d + 1. This implies 2X;; + d + 1 5 2d or Xii 5 e. d Each of these points P, has points Qe+, in We+1 n S, 0e which are e distance from Pd. This can be seen by deleting e of the d ones in the coordinate vector of Pd. From (1) since d + e + 1 = 2Xii + e, we have Xii = e + 1. Thus every one in the coordinate vect.or of Qe+, occurs also in the coordinate vector of the corresponding Pd. Moreover, these e-neighbors of the set W, n X, are all distinct in W e+1 since any two points in Wd n X, are at least d = 2e + 1 apart, actually at least d + 1 apart. From W, + 1 = (W,,, r\ S,) + (W,,, n S,,:) we obtain

A point Qefl in W r\ A,,., is e + 1 dist#ance from at ,+1 most [n/(e + l)] points in 8,. This can be seen by translating Q,+1to the origin for the moment and asking how many points can be of weight e + 1 with mutual Hamming distance at least d, or rather d + 1 since d is odd. Her& 2(e + 1) = 2Xcj + 2(e + l), or Xii = 0.

This can be seen by considering an R by 112 matrix whose elements are zeros and ones with the maximum number R(m, r, X) = R of ?n-order row vectors of weight r, with the inner product of any pair of rows _< X. The maximum number of ones in any given column or the maximum column sum is 5 R(nz - 1, r - 1, X - 1). For consider the vectors with one in this column. After s deleting this column of one these vectors will be des creased by one in order, weight, and pairwise row inner product. Since each column sum must be 5 R(m - 1, r - 1, X - l), the total sum by columns is 5 mR(m - 1, r - 1, X - l), the same as the total row sum = rR(m, r, A). Repeated application of (.5) gives Theorem 2 since finally R(n - e, d - e, e - e) = [(n - e)/(d - e)]. From an analogy of the combinatorial study of balanced block designs (for example, see Ryser*) another upper bound for R = R(m, r, X) is possible when r2 > mX. Consider an R by ~2 matrix with row sums = r and maximum pairwise row inner product = X. The row calculations for the sum of all the row inner products summed over ordered pairs of (i, j) gives $ z hi = R(R 16 I R@ 1)X

Thus we find R(n, e + 1, 0) = [n/(e + l)], the number of ways we can choose disjoint sets of e + 1 ones from n. This factor [n/(e + l)] was pointed out by Lloyd Shapley. Now when we sum up over all points in X,, counting their k-neighbors for lc 5 e + 1, all k-neighbor points for k 5 e are uniquely assigned to one point in S, and therefore counted just once. On the other hand the points in S e+, may be counted as many as [n/(e + l)] t imes, so we must divide the lower bound estimate of e + l-neighbors

where X is the average row inner product. The column calculation for the same sum of all the row inner product,s (if kj is the j-th column sum) is

since
$ 1~~= $ ri = Rr.

by [nl(e + 111= Wn, e + 1, 0).

Thus we have the inequality

j Ah, 411+ ($ + ($ + . ($ .
+ (e y 1) -

Thus R(R 1)X 2. R(R 1)x = g

1~: -

Rr.

r n 1
Lel

(%)R(n dle)i.

r 2

giving Theorem I. Next, we look for good upper bounds for R(n, d, e). Two simple techniques will be developed here. More refined est imates are being studied and may be presented in a later paper. Theorem 2:

Now, subject to cy 1~; = Rr, cy Icy is the ki are as nearly equal as possible. If integral value of rR/m, so that rR = mk we have an upper bound for R(m, r, largest R satisfying R(R 1)X 2 (m t)k2 + t(k +

minimized when k = [rRlm], the + t, 0 5 t < m, X) given as t he 1) - rR.

(6)

First an approximate value to R can be found by allowing kj to be equal, neglecting the restriction that s they must be integers. In this case nz& = rR, c; i? = r2R2/m and R(R 1)X >_ z - rR.

Rb,d,e)I [2[&[...

pz]

. ..I]].

(4)

Note these are nested integral values. First we prove a more general result.

Solving for R gives the following: Theorem 3: If r2 > mX, 1, X 1)

R(m, r, A> 5

;R(

m -

1,r -

(5)

(7)

206

IRE

TRANSACTIONS

ON INFORMATION

THEORY

April

This approximate value can be used for R(m, r, X) when r2 > mh. Or we can test it in (6) which sometimes further decrease our upper-bound estimate. Thus we can use a combination of (5), (6) and (7) whenever applicable in any particular numerical case giving a relatively simple calculation for R(n, d, e). Theorem 3 is almost always a stronger result than Theorem 2 so that we wish to find out when it applies. If r2 > mX, we use (7) and then (6). If r2 < mX, we use (5) repeat*edly, reducing r, m, and X by one each time. Eventually we either get Theorem 2 or after t applications of (5) for some smallest t < e we have (d - t) > (n - d)(e - t) where Theorem 3 applies. Sometimes t + 1 applications of (5) gives better results if (d - t) (n - t)(e - t) happens to be too small a positive integer. Thus we get major improvement over the Hamming bound for the cases roughly bounded by n < e2 + 5e + 2. Otherwise we get only the minor improvement obtained by taking repeated integral values in (5). To illustrate the use of all these relations (5), (6) and (7), consider the example R(20, 7, 3). Here (7) does not apply. By (5) we have g6 R(18, 5, 1) By (7) we find R(18,5, However, since 1) _< [y;5:l;)] = [$+I = 10. (6)

own and other techniques to improve upper-bound estimates for group codes. In this connection the improved bound given in this paper can sometimes be used to give further improvements to group code bounds. For example, Griesmer shows that his function N(18, 13) 2 42 by using Hamming bound to give s N(17, 7) 2 29, while our bound gives N(17, 7) > 30 and thus N(18, 13) > 43.
EXTENSION TO

Sk FOR k > e + 1

For the case k = e + 2 there are four sets of points to be subtracted from n the number of points in ( e+2 > in W,+z must be in one of the sets

W 6+2* Any point se-,, se, &,I, x,+2. Reasoning as before from (1) we can conclude that the points in W,+z fI Se--l) i.e., in both W,+z and S,-,, are formed by deleting e - 1 ones out of d ones in the coordinate vector of each code point in Wd. Thus there are at
most

R(n, d, e) points in W,,, n LY?,-~. The points

11

in Wc+2 A S, are formed by deleting e ones from the d + 1 ones in the coordinate vector of each code point in W d+l. Thus there are at most in We+2 + 1, e + points in in Wd or d t ( n S, since from 1) code points in W,,, n S,,, are e W,+z or both. R(n, d + 1, e + 1) ) (I) there are at most Wdfl. + 1 distance from code

points R(n, d The points

if R(18, 5, 1) = 10 we would contradict = 90 < 92 = 14(3 + 4(2 - 50. ) )

f T 4 R(n, d + 2, e f 2) ( > points which are e + 1 distance from some code point in Wrdt2. Thereare at most (t) ( 1 R(n, d,e) pointsin )

Thus there are at most

lO(9)l Thus

R(18, 5, 1) < 9 R(20, 7,3) By Theorem 1 A(20, 7) 5

and 5 [y [; (9)]] = [T] = 80.

W,,, n S,,, which are e + 1 distance from code points in Wd, since by (1) we have e + 2 + d = 2Xii + e + 1 so Xii = e + 1. Thus we may delete e ones out of the d ones in the vector for the code point P, and add one new one to form the coordinate vector for Qe+, which is e + 1 away. The problem here is to estimat,e lower bounds for the points in TV,,, r\ S,,, which are e + 1 distance from several code points in either or both W, and TYd+s.This has not been solved as yet. By ignoring this overlapping we can still get a somewhat crude lower bound on the number of points in W,+? n S,+? given as follows: (e y 2) _ ( (, f l)Rh d, 4 (3(n 1 )Rh 4 4

and we have reduced the upper bound from Hamming s value 776 down to 595. As a method of comparison of the upper bounds for A (n, d) derived from these sphere-packing models, we shall partially reproduce the table employed by Wax. Professor Bose has furnished some newer values of bestknown codes. Griesmer has combined Hamming bounds with his s
5 J. H. Griesmer, A bound for error correcting codes, IBM J. Res. and Dev., vol. 4, pp. 532-542; November, 1960.

d T R(n, d + 1, e + 1) > df2 e + l R(n, d + 2, e + 8. >

This number must be divided by R(n, e + 2, l), the maximum number of code points which could be at distance e + 2 from a noncode point in W,,, f3 S,,,.

1962

Johnson: Upper Bound for Error-Correcting

Codes

207

This follows from (1) by similar argument to that used for points in W,,, n X,,,. In general a lower-bound estimate for the number of for points in W,,, n A!Y,+~, t 2 1 involves subtracting These t.erms n ( e-l-t > represent, upper bounds to point counts in sets which may overlap and hence give somewhat crude estimates. Setting r, = e + t, we have our general improvement for -4(n> d) as follows: Theorerll 3 : t2 terms of the type indicated from

This can be interpreted as a curve cutting under both Plotkin and Hamming asymptotic upper bounds as s s represented in Peterson Fig. 4.1.6 s This work will appear in another paper.
APPENDIX

The following inequality estimates of R(n, d, e): R(m,r,X) 5 ;%R(m

sometimes

sharpens

our

l,r,X)

1 (11)
.

Ah, 4 5
( ) where

This can be seen by first changing all zeros to ones and ones to zeros in t,he (0, 1) matrix, then applying (5), then reversing the zeros and ones again. Thus R(m, r, A) = R(m, m - r, m - r - (r - A))

cm, n s,,)2.0 ry

m-R(m-l,m-l-r,m-l-r-(r-X)) m-r

1
p+]].

for each rl > e such that c(W,, r\ AS > 0. ,,) From (1) and the relation rZ - r1 5 d,, 5 r, - 1 we see that [(rz + 1)/2] 5 x,, 5 rl. Thus for each code point P-, we have ($ (c 1 i:,) points in W,, which are at

This relation applies to a variety of cases. In particular we can show that for n = 6 u - 1, u > 1, R(n, 3, 1) < [i In contrast to this R(n, 3, 1) = [a p+]] for n = 6 u + 1, or 6 u + 3, for u 2 1, since Steiner triple products exist for such cases (see Ryser4). Special combinatorial arguments are needed to show that R(12, 5, 2) is only 12 rather than 14. We can also show that R(13,5,2) = 18, A(13,5) < 69; R(14, 5, 2) 5 27, A(14, 5) 5 127; R(l5, 5, 2) < 40, A(15, 5) 5 248. In Theorem 1, the term [n/(e + l)] can be replaced by

distance rl + rZ - 2X,, away from P,,. Such a point Qr, is formed by ret,aining XIZ ones from the rZ ones in the coordinate vector for P,, and adding r, - X,, new ones in positions chosen from the n - rZ positions which have zero coordinates for P,,. The estimate for the case A(lOO, 41) was carried out and the results are presented to give insight on the relative size of the sets W.,, r\ AS.,,. Hamming s sum 2
t=O

The first correction term involving

c(W21 &J/RUOO, 0) n 21,

Subsequent contributions give gives 5.1 X 10. 14.6 X 10Zo, 40.9 X lo, 104 X lo, 130 X 10 for (, -;- 1> - ($Rh d, 4 = 2, 3, 4, 5. Almost all points in W,,, are also in Xe+L = 1 and 2, at, least 98 per cent for t = 3, 91 per cent for whenever this is smaller. = 4, 48 per cent for = 5, and our crude estimates Thus, for example, A(16, 7) 5 62. gives 0 per cent for > 5. Thus neglecting overlapping as Corresponding generalized expressions may replace the indicated, we obtain an upper bound A(lOO, 41) < l/42 terms R(n, rl, rl - e - 1) in Theorem 4, giving, for exof Hamming value. s ample, A(lOO, 41) < l/190 of Hamming value. Also s we can show that in the asymptotic case, any e-error FURTHER WORK ON ASYMPTOTIC ESTIMATE correcting code will correct almost all sequences with e + t More detailed arguments lead to the result that as e errors for t < g(F), where g(F) was described earlier in and n grow large almost all points in W,,, are in S,+t this paper. Details will be presented later. for 0 < t 5 g(F), a complicated function of F = n/e.

t t t

Thus, for example, g(4) % e, g(5) % 0.3e, g(6 M )0.2e, !m) = O.O9e, etc. where the decimal coefficients are only approximate.

6 W. W. Peterson, Error-Correcting and John Wiley and Sons, Inc., New York,

Codes, M.I.T. Press N. Y., p. 56; 1961.