KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 617

DIGITAL IMAGE STABILIZING ALGORITHMS BASED ON BIT-PLANE MATCHING

Sung-Jea KO,Senior Member IEEE, Sung-Hee Lee and Kyung-Hoon Lee
Department of Electronics Engineering, Korea University
5-1 Anam-Dong, Sungbuk-Ku, Seoul 136-701, Korea
Email: sjko@dali.korea.ac.kr

Abntract-In this paper, we present a new digital image The full-search(FS) BMA under the mean absolute dif-
stabilization (DIS) scheme based on bit-plane matching ference (MAD) and mean square error (MSE) criteria
(BPM). The proposed DIS system performs motion es-
timation using 1-bit planes which are extracted from a
can be considered as an optimal solution for motion es-
video sequence. This motion estimation technique can be timation [6],[7]. However, the FS BMA requires large
realized using only Boolean functions which have signif- amount of computations which causes time delay, and
icantly reduced computational complexity, while the ac-
requires complex hardware architecture [7]-[9].In this
curacy of motion estimation is maintained. In the sec-
ond part of this paper, a median-based motion correction paper, we present a new motion estimation technique
scheme is proposed which is robust to various irregular based on the bit-plane matching (BPM) for the DIS sys-
conditions such as moving objects and intentional pan- tem. The proposed algorithm performs binary motion
ning. Simulation results show that the proposed DIS al-
gorithm exhibits better performance compared with ex-
estimation using 1-bit planes which are extracted from
isting other algorithms when applied to real video signal. a video sequence. This motion estimation technique can
be realized using only Boolean functions which have sig-
nificantly reduced computational complexity, while the
I. INTRODUCTION accuracy of motion estimation is maintained. Simulation
Image stabilization is the process of generating a com- results show that the performance of the BPM-based mo-
pensated video sequence where image motion by the tion estimation algorithm is comparable to that of the FS
camera’s undesirable shake or jiggle is removed [1]-[5]. BMA.
The recent DIS systems are realized using digital im- Various algorithms have been developed to estimate
age processing techniques instead of mechanical motion the global motion of a frame from local motion vectors
detection techniques using gyro sensors or fluid prism [3]-[7],[11]. Most of these algorithms are complicated,
PI-[51. and thus are not simple to implement. In this paper, we
The image stabilization task can be subdivided into propose a simple and robust decision algorithm for deter-
two basic systems, namely: i) the motion estimation sys- mining the global motion vector. In the proposed algo-
tem and ii) the motion correction system. In general, the rithm, the global motion vector of a frame is determined
motion estimation system generates several local motion based on the order statistics of current local motion vec-
vectors from subimages in the different position of the tors and past global motion vectors. It is shown that the
frame using a block matching algorithm (BMA). The DIS system using the proposed motion estimation and
motion correction system determines the global motion correction algorithms is less sensitive to irregular condi-
of a frame by appropriately processing these local mo- tions such as moving objects and intentional panning.
tion vectors, and decides whether the motion of a frame This paper is orgainzed as follows: The proposed mo-
is caused by undesirable fluctuation of the camera or in- tion estimation and correction algorithms are presented
tentional panning. The stabilized image is generated by in Sections I1 and 111, respectively. Simulation results
reading out the proper block of fluctuated image in the are given in Section 1V and concluding remarks are in
frame memory [3]-[5]. Section V.

This work was supported by SAMSUNG Electronics Co.

Manuscript received June 17, 1998 0098 3063/98 $10.00 1998 IEEE

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
 618 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

Fig. 1. Example of generating bit-planes from a gray-sclae image, (a) an original image, (b) eight bit-plane images.

11. MOTIONESTIMATION USINGBIT-PLANE IMAGES the bit-plane as shown in Fig. 2. Each motion vector of a
Before describing the proposed method to estimate lo- subimage in the current bit-plane image is determined by
cal motion vectors, we introduce the bit-plane decompo- evaluating bit-plane matching over subimages in the pre-
sition of a gray-scale image. vious bit-plane and selecting the subimage which yields
the closest matching. This approach assumes that all
A . B i t - p l a n e Decomposition of a Grayscale I m a g e pixels within the subimage have uniform motion and the
Let the graylevel of the pixel at location (z,y) in the range of the motion vector is constrained by the search
t-th image frame with 2K graylevels be represented as window.
Let the size of each subimage be M x N and a search
+
window be ( M 2p) x ( N + 2q). For bit-plane matching,
we define the correlation measure given by
where a k , O 5 h 5 K - 1, is either 0 or 1. Let the k-
th order bit-plane image be denoted by b t k ( z , y ) . This ~ ( m n),= C
b k t ( z , y) CB bkt-'(z m, y n) (2) + +
plane contains all the k-th order (ak)bits. For the case (",Y)ES.

of the 8-bit image, an image is composed of eight 1-

where bkt(a,y) and bkt-'(z, y), respectively, are the cur-
bit planes bk(z,y) N b$(z,y), ranging from plane 0 t o
rent and previous k-th order bit-planes, and @ is the
plane 7. Fig. 1 shows eight bit-planes decomposed from
exclusive-OR operation.
a grayscale image. bi(z,y) contains all the least signif-
At each ( m , n ) ,-p 5 m 5 p and -q 5 n 5 q, within
icant (lowest order) bits comprising pixels in the image
the search range, the proposed matching method calcu-
and b$(a, y) contains all the most significant (highest-
lates C i ( m , n )which is the number of unmatched bits
order) bits. Note that only the higher order bit-plane im-
between the reference subimage in the current bit-plane
ages contain visually significant data whereas the other
and the compared subimage in the previous bit-plane.
bit-planes contribute to more subtle details within the
The smallest Ci(m,n) yields the best matching for Si,
image.
and thus local motion vector V; from Si is obtained as
B. M o t i o n E s t i m a t i o n Based o n t h e Bit-plane M a t c h i n g
vit = arg min{Ci(m, n),- p 5 m 5 p , -q 5 n 5 q } . (3)
Local motion vectors are estimated from four subim-
ages (SI,S2, Ss, 5 4 ) placed in appropriate positions in

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
 KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching
! preuioiis bit-plane image
I-_-._
_ _ _ _ - - _ _ I - _ _ _
Fig. 2. Estimation of local motion vectors from four subimages in a bit-plane.
This motion estimation technique can replace the
arithmetic calculations of BMA's based on conventional
MAD and MSE criteria with simple Boolean exclusive-
OR operations, and thus has significantly reduced com-
putational complexity.
Since the proposed DIS system performs motion esti-
mation using a single bit-plane, it is important to select
an appropriate bit-plane for bit-plane matching. In this
paper, the 4-th order bit-plane, b;(z,y), is utilized to
estimate the local motion vector since it contains both
the global information and details of the original image.
To show that b:(z, y) is suitable for motion estimation
based on BPM, an example for BPM using 1-D signals
is presented in Fig. 3.. Fig. 3(a) shows an 1-D signal
which is one horizontal scan line of a real image with
256 graylevels, and Fig. 3(b) is the same scan line from
the previous frame. Using (2), we computed the correla-
tion measure between two binary signals from the scan
lines at each bit-level. Fig. 3(c) illustrates the simulation
results. It is seen that the correlation measure from the
4-th bit-level exhibits the steepest gradient around the
minimum point.
111. MOTIONCORRECTION UNDER IRREGULAR
CONDITIONS
In this section, we introduce the motion correction sys-
tem to cope with irregular conditions such as moving
objects and intentional panning that degrade the perfor-
mance of the DIS system.
Fig. 4 shows the proposed motion correction system
which consists of the decision unit, the integration unit,
Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
619
i
and the motion compensation unit. The motion correc-
tion system determines the global motion of a frame by
appropriately processing local motion vectors, and de-
cides whether the motion of a frame is caused by un-
desirable fluctuation of the camera or intenti6nal pan-
ning. The stabilized image is generated by reading out
the proper block of fluctuated image in the frame mem-
ory.
In an image with motion, some subimages with mov-
ing objects can produce motion vectors which are signif-
icantly different from the other motion vectors. Fig. 5
shows an image which has moving objects in some subim-
ages. Fig. 6(a) and (b) show the correlation measures
calculated using (2) from subimage SI which has no
moving object and subimage S4 with moving objects,
*
respectively. In Fig. 6, for display, the correlation mea-
sures are normalized using l - where C,,, is the
maximum Cj(m,n) within the search range. It is seen
that there does not exist a distinct maximal correlation
value in S4.
In general, motion vectors from the subimages with
moving objects are not reliable and should be excluded
from the global motion decision process. Moreover, since
the hand movement is relatively slow than the frame rate
of the video camera, two successive frames fluctuated by
camera's shake should have similar global motion.
Based on these properties of camera's motion, we pro-
pose a simple and robust motion correction scheme where
global motion decision is performed using current local
motion vectors (V:, Vi, Vi,V:) and the previous global
620 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

-8 .o -4 -2 0 2 4 0 B

horizontal displacement

(c>
Fig. 3. An example of BPM using 1-D signals, (a) a 1-D signal which is one horizontal scan line of a real image, (b) the same scan line
from the previouse frame, ( c ) the correlation measure between two binary signals from the scan lines at each bit-level. (+: bit-level
6, 0: bit-level 5, *: bit-level 4, 0 : bit-level 3, X : bit-level 2).

Fig. 4. Basic structure of the motion correction system.

motion vector V,"-'. In the proposed algorithm, the
global motion vector is obtained by
Vi = median{Vi, Vi, V i ,V i ,v,"-'}
Here the median of vectors is determined by seperately
selecting medians of each vector elements.
Local motion vectors affected by undesirable condi-
tions such as moving objects can be viewed as impulses.
It is known that the median filter is very effective in elim-
inating impulses. Therefore, the median-based method
in (4) can exclude such abrupt local motion vectors and
(4) produce a global motion vector similar to the previous
one.
After determining the global motion vector, the mo-
tion correction system decides whether the motion of a
frame is caused by camera's motion or intentional pan-
ning. For this decision, the global motion vector of a
KO, Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 62 1

Fig. 5. An image with moving objects.

(4 (b)
Fig. 6. Correaltion measures, (a) from subimage S1 which has no motion, and (b) from subimage Sb which contains moving objects.

frame is integrated with a damping coefficient, and the TABLE I

integrated motion vector designates the final motion vec- RMSE’S ASSOCIATED WITH LOCAL MOTION YECTORS.

tor of a frame for motion correction. The integrated mo-

Sequence number RMSE
tion vector V, for estimating intentional panning is given
I
RPM I
I
EPM I BPM
I
by 1 I 0.0334 I 0.3285 1 0.0205
vat = DIVat-l + V,t (5) 2 0.0373 0.4886 0.0333
3 0.0296 0.3000 0.0219
where Q t is a global motion vector and D1(0 < D1 < 1) 0.0518 0.3409 0.0362
4
is a damping coefficient for smooth panning. I

1 0.0373 1 0.5002 1 0.0229

5

IV. SIMULATION
RESULTS
To evaluate the motion estimation performance of the
proposed BPM algorithm, we compare it with two exist-
ing DIS algorithms, namly, representative point match-
ing (RPM) [3] and edge pattern matching (EPM) [4].
The performance is evaluated using the root mean square
error (RMSE) based on the FS BMA under the MAD
criterion. The RMSE is given by
where (zn,yn) is the motion vector from the FS BMA,
and (&, ijn) is that from aformentioned algorithms. Five
real image sequences with a resolution of 640 x 240 and
150 frames are utilized for simulation.
The RMSE’s associated with the local motion vector
are summarized in Table I. It is seen that BPM exhibits
better performance than RPM and EPM.
Table I1 summarizes the RMSE’s associated with
global motion vectors. It is interesting to observe that
the RMSE’s of BPM are always smaller than those of
RPM and EPM.
These experimental results indicate that the proposed
BPM-based DIS system exhibits good performance com-
parable to the FS BMA.
622
TABLE I1
RMSE’S ASSOCIATED WITH GLOBAL MOTION VECTORS.
Sequence number
RMSE
RPM EPM BPM
1 0.0285 0.1223 0.0178
2 0.0308 0.2082 0.0260
3 0.0212 0.0585 0.0164
4 I
0.0349 I
0.0827 I
0.0242
5 1 0.0374 I 0.1690 1 0.0242
V. CONCLUSION
In this paper, we proposed a BPM-based DIS system
which performs motion estimation using 1-bit planes.
It was shown experimentally that the performance of
BPM is very close to that of the FS BMA in terms of
RMSE. Moceover, BPM can be realized using only sim-
ple Boolean functions, and thus is more suitable for VLSI
implementations than existing algorithms. It was also
shown that the median-based motion correction scheme
is robust to irregular conditions such as moving objects.
Simulation results show that the proposed DIS algorithm
is a computationally efficient alternative to existing DIS
algorithms.
REFERENCES
M. Oshima, T. Hayashi, S. Fujioka and T. Inaji, “VHS cam-
corder with electronic image stabilizer,” IEEE Trans. on Con-
aumer Electronics, vol. 35, no. 4, pp. 749-758, Nov. 1989.
K. Sato, S . Ishizuka, A. Nikami, and M. Sato, “Control tech-
niques for optical image stabilizaingsystem,” IEEE Trans. on
Consumer Electronics, vol. 39, no. 3, pp. 461-466, Aug. 1993.
K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Ki-
tamura, “Automatic image stabilizing system by full-digital
signal processing,” IEEE Trans. on Consumer Electronics,
vol. 36, no. 3, pp. 510-519, Aug. 1990.
3.-K. Paik, Y.-C. Park, and De-W. Kim, “An adaptivemotion
decision system for digital image stabilizer based on edge pat-
tern matching,” IEEE Trans. on Consumer Eleclronics, vol.
38, no. 3, pp. 607-615, Aug. 1992.
T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T.
Imaide, “Electronic image stabilizer for video camera use,”
IEEE Trans. o n Consumer Electronics, vol. 36, no. 3, pp.
520-525, Aug. 1990.
H. G. Musmann, P. Pirsch, and H. J. Gralleer, “Advances in
picture coding,” Proc. IEEE, vol. 73, no. 4,pp. 523-548, Apr.
1985.
B. Liu and A. Zaccarin, “New fast algorithms for the estima-
tion of block motion vectors,” IEEE Trans. on Circuits and
Sydems for Video Technology, vol. 3, no. 2 , pp. 148-157, Apr.
1993.
3. Lu and M. L. Liou, “A simple and efficient search algo-
rithm for block-matching motion estimation,” IEEE Trans.
IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998
o n Circuits and Systems for Video Technology, vol. 7, no. 2,
pp. 429-433, Apr. 1992.
[9] H. Gharavi and M. Mills, “Blodunatchingmotion estimation
algorithms-new results,” IEEE B a n d . o n Circuits and Sys-
tems, vol. 37, no. 5, pp. 649-651, May 1990.
[lo] Y. T.Tse and R. L. Baker, “Global zoom/pan estimation and
compensation for video compression,” Proceedings of ICASSP
91, Toronto, Ontario, Canada, pp. 2725-2728, May 1991.
[113 M. Okada, “Video camera and video signal reproducing ap-
paratus with shake detection and correlation operation,” U.S.
Patent, no. 5,502,484, Mar. 1996.
S u n g - J e a K Oreceived the Ph.D. degree in
1988 and the M.S. degree in 1986, both in
Electrical and Computer Engineering, from
State University of New York at Buffalo,
and the B.S. degree in Electronic Engineer-
i r g at Korea University in 1980. In 1992,
he joined the Department of Electronic En-
gineering a t Korea University where he is
currently an Associate Professor and Grad-
uate Chairman. From 1988 to 1992, he was an Assistant Pro-
fessor of Electrical and Computer Engineering at the University
of Michigan-Dearborn. From 1981 to 1983, he was with Daewoo
Telecom Corporation where he was involved in research and de-
velopment on data communication systems. He received the Hae-
Dong best paper award from the Institute of Electronics Engineers
of Korea (1997), and the best paper award from the IEEE Asia
Pacific Conference on Circuits and Systems (1996). Dr. KO is cur-
rently a Senior Member in the IEEE and a programs chair (1996
- Present) of the IEEE Seoul Section of Korea Council. His cur-
rent research interests are in the areas of digital signal and image
processing, and multimedia communications.
S u n g - H e e Lee received the B.S. degree
in information engineering and M.S. degree
in computer science form Korea University
in 1993 and 1995, respectively. He is now
a Ph.D. candidate in electronic engineer-
ing with the Department of Electronic En-
gineering a t Korea University. In 1995, he
joined the Research Institute for Informa-
tion and Communication Technology, where
he is currently a research engineer. His research interests are in
the areas of wavelets and nonlinear signal and image processing.
K y u n g - H o o n Lee received the B.S. degree
and M.S. degree in electronic engineering
form Korea University in 1992 and 1994,
respectively. He is now a Ph.D. candidate
in electronic engineering with the Depart-
ment of Electronic Engineering at Korea
University. In 1993, he joined the Research
Institute for Information and Communica-
tion Technology, where he is currently a re-
searcher. His research interests are in the areas of nonlinear signal
and image processing including morphological and order statistics
filtering.