THPM 18.

4
PICTURE ENHANCEMENT IN VIDEO AND BLOCK-CODED
IMAGE SEQUENCES
Gaetano Scognamiglio*, Andrea Rizzi**, Luigi Albani**, and Giovanni Ramponi*
*D.E.E.L, University of Trieste, Trieste, Italy
**Philips Research Monza, M o n a (MI), Italy

ABSTRACT are the outputs of a Laplacian filter along the horizontal

We propose a sharpness enhancement technique for video and the vertical direction respectively, and
and decoded image sequences. A non-linear function
generates a correction signal that is added to the
luminance edges. Spatio-temporal information is used to
enhance details, avoiding blocking artefacts and noise.

INTRODUCTION
Where TV and PC systems merge, high-quality display of
are the related control functions. The rational function
video images on PCs becomes a challenge for industries
oriented to the consumer market. Improving the poor
sharpness of video images is probably the most obvious
feature for the end user, who usually compares the video
images displayed on a PC monitor to very sharp graphics.
Despite the many algorithms for 2-D sharpness assumes values depending on the sensors:
enhancement already proposed, their effectiveness drops in
d,(n,,n,) = s(n, - I ,nY)- dn, + I +,) (2)
this context because they usually introduce many small
artefacts that, while perfectly acceptable in still pictures, dY (aX ,nY ) = s(n,,ny - 1) - s(n,,ny + 1) (3)
become very visible and annoying in image sequences.
With PCs, even more attention has to be paid to The parameters k and h set the maximum of the rational
decompressed images (MPEG, AV1 etc.), where blocking function at the abscissa d p The value of do is chosen to
elfects are often visible. The high frequencies artificially achieve the best trade-off between enhancement of details,
arising along the borders of the blocks themselves are reduction of noise amplification and limiting of excessive
usually amplified by sharpness enhancement algorithms, overshoot. Both c, and cy are IIR filtered along the
emphasizing blocking artefacts. In this paper, we propose a direction orthogonal to the control action to avoid false
sharpness enhancement technique that takes image details from appearing in highly detailed areas. A temporal
blocking into account. The enhancement is achieved by filtering for noise reduction is also included in Eq.1 by
adding a correction signal to the luminance edges in an subtracting the following high-pass signal from the input signal:
Unsharp Masking-like way, following an approach u,(n,,ny,n,) = ~rzl(~,,~y.~r)c,(n,,ny,n,)
successfully used for still pictures [I], and exploiting both
spatial and temporal information. where zt is a temporal Laplacian filter and c, is the temporal
sensor function:

0 if Id,( > d o

d,(nx,ny,n,) = x(n,,ny,nt- 1) --(n,,ny,nt + I )

The controlling function c, avoids filtering when dr >> 0,
whereas the temporal Laplacian Z, is non-null when d, 9 0 .
So noise is suppressed, while the sharpness of moving
objects is preserved. On their own Eq.2 and Eq.3 would
mean that one-pixel-wide lines would not he enhanced or
(because of their actual variable thickness) would be

334
0-7803-5123-11991 $10.00 0 1999 IEEE

Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on August 17,2010 at 05:42:45 UTC from IEEE Xplore. Restrictions apply.
enhanced in a non-homogeneous way; this would
emphasize the staircase effect. When noisy pixels are
adjacent to a line, the output of the sensor varies along the
line. Temporal changes due to noise, too, would give rise to
a non-uniform enhancement, appearing as an annoying
flicker effect close to the line. For a very thin line, the
following three cases are possible:
horizontal line /z,I 2 0 Izy( B 0
vertical line /zxI >> 0 lzyl 4 0
diagonal line /zxI n 0 lzyl )* 0
From these relationships, it is straightforward to
discriminate between a noisy pixel and a pixel belonging to
a line simply by setting a threshold. Once a line is detected,
the only need is to avoid excessive overshoot, and we have
therefore modified the control function. For example, in
the case of a horizontal line, cy is replaced by:
while cx remains unchanged
ENHANCEMENT IN DECODED IMAGE
SEQUENCES
A simple modification to the algorithm described above
must be introduced when processing pixels belonging to
the borders of‘ a block [ 2 ] .Here, hotb the rational function
and the sensor which operate orthogonally to the border
itself are modified. For instance, along the vertical borders
cx is replaced by:
while the sensor becomes:
Fig. I Result of a test on a detail of “Mobile & Calendar”, after M P E G coding of the 720 x 576 frames at 3 Mb/s and reconstruction.
Original image (leff),enhanced version with linear Unsharp Masking (middle) and rational Unsharp Masking (right).
Authorized licensed use limited to: MULTIMEDIA UNIVERSITY. Downloaded on August 17,2010 at 05:42:45 UTC from IEEE Xplore. Restrictions apply.
where ci2 is the variance of the pixels belonging to a 3 x 3
window centered on the current pixel, and lslh is a constant.
The constant -h’ in the numerator of c introduces a low-
Xb .
pass effeqt aiming to reduce the block visibility. In uniform
areas, 0 E O ; therefore cXi,<O so that s +hc3,z,is a
low-pass signal. In detail areas ls1>> 0, and then d, I d,
When dx >> 0, cx I cx and the original behaviour is left
(4) unchanged. On thg other hand, when d, e 0 (and ls2 >> 0)
(5) the area is uniform in the horizontal direction, so the low-
pass filtering is applied.
(6)
A similar rational function is also used inside the macro
block to reduce ringing and motion-compensation
mismatch effects. For this task, cx is replaced by:
d,(n,,nJ
2
-h’
cxi (nX‘ ny ) =
k;lx(n,,nJ4 + 6h’
CONCLUSIONS
We have presented a non-linear contrast enhancement
technique for video applications. In coded image
sequences, the behaviour of the filter ranges from a quasi-
linear low-pass one in uniform areas to a strongly non-
linear high-pass one in detailed areas. A specific
component of the operator allows this technique Lo he
employed for decoded sequences in a PC environment, also
thanks to its low cost of hardware andlor required
processing power. The algorithm has proven to he robust
against the heterogeneous contents of real sequences, and
its results clcarly outperform the existing algorithms (an
example is shown in Fig. I).
REFERENCES
[ I ] G. Ramponi and A. Polesel, “A Rational Unsharp
Masking Technique,” Jonrnal of Electronic Imaging,
vol. 7 , no. 2 , April 1998, pp. 333-338.
[2] S. Marsi, R. Castagno, and G . Ramponi, “A Simplc
Algorithm for the Reduction of Blocking Artifacts in
Images and its Implementation,” lEEE Trans. O J ~
Consumer Electronics, vol. 44, no. 3, Aug. 1998.
335