Lossless compression of hyperspectral images

using hybrid context prediction

Yuan Liang,
1,*
Jianping Li,
1
and Ke Guo
2

1
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu,
Sichuan, 611731, China
2
College of Management Science, Chengdu University of Technology, Chengdu, Sichuan, 610059, China
*tacal@163.com
Abstract: In this letter a new algorithm for lossless compression of
hyperspectral images using hybrid context prediction is proposed. Lossless
compression algorithms are typically divided into two stages, a
decorrelation stage and a coding stage. The decorrelation stage supports
both intraband and interband predictions. The intraband (spatial) prediction
uses the median prediction model, since the median predictor is fast and
efficient. The interband prediction uses hybrid context prediction. The
hybrid context prediction is the combination of a linear prediction (LP) and
a context prediction. Finally, the residual image of hybrid context prediction
is coded by the arithmetic coding. We compare the proposed lossless
compression algorithm with some of the existing algorithms for
hyperspectral images such as 3D-CALIC, M-CALIC, LUT, LAIS-LUT,
LUT-NN, DPCM (C-DPCM), J PEG-LS. The performance of the proposed
lossless compression algorithm is evaluated. Simulation results show that
our algorithm achieves high compression ratios with low complexity and
computational cost.
2012 Optical Society of America
OCIS codes: (100.4145) Motion, hyperspectral image processing; (110.4234) Multispectral and
hyperspectral imaging.
References and links
1. B. Aiazzi, L. Alparone, and S. Baronti, Near-lossless image compression by relaxation-labeled prediction,
Signal Process. 82(11), 16191631 (2002).
2. E. Magli, G. Olmo, and E. Quacchio, Optimized onboard lossless and near-lossless compression of
hyperspectral data using CALIC, IEEE Geosci. Remote Sens. Lett. 1(1), 2125 (2004).
3. B. Aiazzi, S. Baronti, and L. Alparone, Lossless compression of hyperspectral images using multiband lookup
tables, IEEE Signal Process. Lett. 16(6), 481484 (2009).
4. J . Mielikainen, Lossless compression of hyperspectral images using lookup tables, IEEE Signal Process. Lett.
13(3), 157160 (2006).
5. B. Huang and Y. Sriraja, Lossless compression of hyperspectral imagery via lookup tables with predictor
selection, Proc. SPIE 6365, 63650L, 63650L-8 (2006).
6. X. Tang, W. Pearlman, and J . Modestino, Hyperspectral image compression using three-dimensional wavelet
coding, Proc.SPIE/IS&T Electron, Imaging 1, 10371047 (2003).
7. B. Penna, T. Tillo, E. Magli, and G. Olmo, Progressive 3-D coding of hyperspectral images based on J PEG
2000, IEEE Geosci. Remote Sens. Lett. 3(1), 125129 (2006).
8. J . Zhang and G. Liu, An efficient reordering prediction-based lossless compression algorithmfor hyperspectral
images, IEEE Geosci. Remote Sens. Lett. 4(2), 283287 (2007).
9. M. J. Weinberger, G. Seroussi, and G. Sapiro, The LOCO-I lossless image compression algorithm: principles
and standardization into J PEG-LS, IEEE Trans. Image Process. 9(8), 13091324 (2000).
10. J . S. Mielikainen, A. Kaarna, and P. Toivanen, Lossless hyperspectral image compression via linear prediction,
Proc. SPIE 4725(8), 600608 (2002).
11. F. Rizzo, B. Carpentieri, G. Motta, and J . A. Storer, Low-complexity lossless compression of hyperspectral
imagery via linear prediction, IEEE Signal Process. Lett. 12(2), 138141 (2005).
1. Introduction
In recent years, the use of hyperspectral images has become more and more widespread due to
the current development of hyperspectral sensors technology. However, the growing scientific
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8199
and technological demands in spatial and spectral resolutions have drastically increased the
data volume of hyperspectral images. Hyperspectral images contain large amounts of
information. Spectral information is quantized into 224 contiguous bands, of approximately
10 nm each, with a spatial resolution of 20 m at operational altitude. Spectral components are
sampled with a 12-bit ADC and then represented with 16-bit precision after calibration and
geometric corrections. The unit size of the recorded image is called a scene, a data cube of
512 lines by 614 columns by 224 bands, for a total of 140 MB. However, large data volume of
hyperspectral images introduces difficulties in transmission and storage; there is the need of
reducing the data size in order to match the available bandwidth. Undoubtedly, data
compression is the appropriate solution to this problem. However, any distortion caused by
lossy compression of the image data is unacceptable in many of the corresponding
applications such as automatic feature extraction, classification, target detection, and object
identification. As a result, only lossless compression of hyperspectral images can guarantee
both the requirements of data reduction and the original quality of the data.
Unlike 2D images, hyperspectral images have two types of inherent correlation: intraband
correlation between neighboring pixels in the same band and interband correlation between
pixels in adjacent bands. Hyperspectral image coding schemes explore data correlations for
compression. Traditional approaches to the hyperspectral images compression are mainly
based on differential pulse code modulation (DPCM), direct vector quantization, or
dimensionality reduction through principal component analysis (PCA) [1]. DPCM basically
consists of a prediction followed by entropy coding of quantized differences between original
and predicted values. A unit quantization step size allows reversible compression to be
achieved as a limit case. Recently, an adaptation of the context based, adaptive, lossless image
codec (CALIC) to on-board hyperspectral data compression has been presented in [2]. The
look-up table (LUT) [3] approach exploits the calibration-induced data correlation that is
specific to hyperspectral images. To predict a pixel, the pixel value of the collocated pixel in
the previous band is used as a key to search an LUT.
In a recent publication [4], Mielikainen introduced a very simple prediction given by the
value taken on the current band by its nearest neighbor (NN), i.e., the spatially closest pixel,
previously encountered along the scan path, having the same value as that at the current
position on the previous band. Such a prediction, which is computationally very simple, can
be effectively implemented by means of a dynamically updated lookup table, which is
indexed by the value at the current pixel position in the previous band and contains the value
of the NN previously taken from the band being encoded. The rationale of prediction based on
LUTs has been later extended by Huang and Sriraja in [5] by exploiting two LUTs,
respectively containing the first and second NNs in the current band. To yield the current
pixel prediction, the choice between the two values contained in the two LUTs, indexed by the
radiance level of the current pixel in the previous band, is based on the similarity to a
reference prediction, which takes into account the cross gain between the current and the
previous bands, as indicated by its acronym locally average interband scaling (LAIS). There
are many excellent wavelet-based three-dimensional image compression algorithms such as
3D-SPIHT and 3D-SPECK for lossy or lossless hyperspectral images compression. 3D-
SPIHT is the benchmark for three dimensional image compressions. It has been applied on
multispectral image compression by Dragotti. Xiaoli Tang and William A. Pearlman use
vector quantization (VQ) and Karhunen Love transform (KLT) on the spectral dimension to
explore the correlation between multispectral bands [6].
In [7], Penna et al. apply a 3-D wavelet transform to decorrelate the image in the spectral
and spatial domains and develop a progressive coding scheme that works from lossy to
lossless coding and complies with the second part of the J PEG 2000 Standard. Transform-
based schemes can yield excellent coding gain for lossy compression at low bit rates while
their lossless coding performance is inferior to these specialized lossless compression
schemes. Recently, Zhang and Liu [8] proposed a two-step adaptive spectral-band-reordering
algorithm. First, the bands are classified into groups based on the correlation factor of
adjacent bands, and then, a reordering algorithm based on the Prim algorithm is applied to
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8200
each group. A prediction method called ABPCNEF is proposed to take advantage of the
similarity of structure and pixel relationship between two neighboring spectral bands and the
residual is coded using adaptive arithmetic coding. In [9], M.Weinberger, G. Seroussi, and G.
Sapiro proposed a J PEG-LS algorithm. J PEG-LS is a simple and effective algorithm, but in its
current form, it can only handle 2-D and not 3-D images. However, since J PEG-LS has been
developed to work with two-dimensional (2-D) data and do not exploit redundancy in disjoint
bands, their compression ratios are very poor. In [10], J . Mielikfiinen, A. Kaarna, and P.
Toivanen proposed an interband linear prediction approach (LP) based on least-squares
optimization. In [11], Rizzo et al. proposed Spectral-oriented Least Squares (SLSQ), in which
spectral correlation is exploited using linear prediction, and the prediction error is then
entropy coded.
As an extension to the LP algorithm, we propose a lossless compression method which
consists of intraband prediction and hybrid context prediction followed by arithmetic coding.
Our primary goal is to develop a fast and efficiency compression algorithm. Simulation results
show that our algorithm achieves higher compression ratios than others. This letter is
organized as follows: The algorithm based on hybrid context prediction is presented in
Section II; Section III describes experiments with AVIRIS images, together with
improvements to the baseline; Section IV concludes and discusses future research.
2. Proposed algorithm
Different from 2D images, hyperspectral images are treated as a three-dimensional (3D) data
set for the purposes of compression. Besides structural correlation can be found in 2D images,
hyperspectral images have two other types of intrinsic correlations: intraband correlation
between neighboring pixels in the same band and interband correlation between pixels in
adjacent bands. However, the dynamic range and noise level of AVIRIS data (instrument
noise, reflection interference, vibrations of the imager, etc.) are higher than those in
photographic images. Assuming that the behavior of every image pixel can be predicted
according to the information provided by its causal neighborhood, the interband redundancy
can be exploited using interband prediction. A salient property of hyperspectral images is that
strong spectral correlation exists throughout almost all bands. Due to the feature of spectral
correlation in hyperspectral images, interband (spectral) prediction outperforms intraband
(spatial) prediction in most cases. For this reason, a spatial predictor like the median predictor
in J PEG-LS tends to fail in this kind of data. Motivated by these considerations, a new
algorithm for lossless compression of hyperspectral images using hybrid context prediction is
proposed. The block diagram of the proposed compression scheme is shown in Fig. 1. It
consists of a decorrelation stage and an entropy coding module. The decorrelation prediction
stage supports both intraband and interband predictions. The intraband (spatial) prediction
uses the median prediction model, since the median predictor is fast and efficient. The
interband prediction uses hybrid context prediction. The LP and context prediction composes
the hybrid context prediction. Finally, the residual image of hybrid context prediction is coded
by arithmetic coding.
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8201

Fig. 1. Block diagramof the proposed lossless hyperspectral images compression scheme using
hybrid context prediction.
2.1 Intraband prediction
The decorrelation prediction stage supports both intraband and interband predictions.
Intraband prediction is applied only to the first image along the spectral line. The median
predictor is employed here due to its simplicity and efficiency for still images. The median
predictor is also the standard predictor for the first band marked for intraband coding (IB set).
Let x be the current pixel and NW (northwest), N (north), and W (west) denote three
neighboring pixels, respectively. The estimate of pixel x is given as follows:

int
min( , ), max( , )
max( , ), min( , )
,

ra
N W if NW N W
N W if NW N W
N W NW otherwise
x

(1)
2.2 Hybrid context prediction
In our algorithm, we propose a new approach that uses a new interband linear predictor for
bands marked for interband coding. The interband predictor relies on a small causal data
subset of the pixel x to compute the prediction. The interband prediction is formed by simply
adding the average difference between the current band and the previous one to the value of x.
LP assumes that the interband prediction is likely to perform poorly and corrects the
prediction by adding to it the average prediction error.
Through comparing the value of the previous pixel and the predicted value of the current
pixel, and analyzing the relations of the previous pixel and the current pixel, we propose the
hybrid context prediction. In general, the lossless compression based on hybrid context
prediction can be expressed in Eq. (2), in which the predictor is the linear combination of
pixels of the previous band. Where y and y are the actual and the predicted values of the
current pixel, x is the previous pixel, x
a
, x
b
, x
c
are the left, upper and upper left neighbor of x,
y
a
, y
b
, y
c
are the left, upper and upper left neighbor of y. If we have an accurate prediction, y
is expected to be close to y.
Suppose we have two adjacent bands. We use the previous band pixel value x
a
, x
b
, x
c
, x
and the current band value y
a
, y
b
, y
c
to predict the value of y. We can get horizontal gradients
(|x
b
x
c
|, |xx
a
|, and |y
b
y
c
|), vertical gradients (|x
c
x
a
|, |x
b
x|, and |y
c
y
b
|) and spectrum
gradients (|x
a
y
a
|, |x
b
y
b
|, and |x
c
y
c
|). Firstly, we compare the value of spectrum gradients
|x
a
y
a
|, |x
b
y
b
|, with |x
c
y
c
|.
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8202
If |x
c
y
c
| max [|x
b
y
b
|, |x
a
y
a
|], it means that the tendency of interband pixel transform
ratio of horizontal and vertical gradients will decrease. In this case we use [|x
a
y
a
| +|x
b
y
b
| +
|x
c
y
c
|]/3 to amend the prediction function.
If |x
c
y
c
| min [|x
b
y
b
|, |x
a
y
a
|], it means that the tendency of interband pixel transform
ratio of horizontal and vertical gradients will increase. Accordingly we use [|x
a
y
a
| +|x
b
y
b
|]/2
to amend the prediction function
Otherwise, if |x
c
y
c
| is in the range of min [|x
b
y
b
|, |x
a
y
a
|], max [| x
b
y
b
|, |x
a
y
a
|], it means
that the tendency of interband pixel transform ratio of horizontal and vertical gradients are
different. Therefore we use [|x
a
y
a
| +|x
b
y
b
| |x
c
y
c
|] to amend the prediction function.

[| | | | | |] 3, | | max(| |, | |)
[| | | |] 2, | | min(| |, | |)
[| | | | | |],
x x y x y x y if x y x y x y
a a b b c c c c b b a a
y x x y x y if x y x y x y
a a b b c c b b a a
x x y x y x y otherwise
a a b b c c

+ + +
= + +
+ +
(2)
3. Experimental result and comparisons
In order to test the performance of the proposed algorithm, some experiments are carried out.
The hyperspectral images for test are the four scenes of the sequences Cuprite Mine and Lunar
Lake in Nevada, Moffett Field and J asper Ridge in California. All images comprise of 224
bands recorded at different wavelengths in the range 380 to 2500 nm, with a nominal spectral
separation of 10 nm between two adjacent bands. Each image is constituted by a variable
number of scenes of size 512 lines by 614 columns. All data that have been considered for
compression are in radiance units, 16-bit format.
We compare the proposed lossless compression algorithm with some of the existing
algorithms for hyperspectral images. Besides 3D-CALIC, M-CALIC, LUT, LAIS-LUT, LUT-
NN, the other methods compared are the clustered DPCM (C-DPCM), J PEG-LS, LP, and the
spectral-oriented least squares (SLSQ) encoder. Table 1 shows the lossless bit rates that are
produced by these algorithms. All scenes of each image have been compressed. The standard
3D-CALIC has been evaluated for the lossless and near-lossless compression of hyperspectral
data. Because the standard 3D-CALIC algorithm switches between the interband and
intraband predictor, it achieves a compression rate of 5.11 bpp. Moreover, the M-CALIC
algorithm outperforms standard 3D-CALIC by more than 0.2 bpp. By testing M-CALIC with
and without the optimized model parameters and quantization thresholds, the gain is the same
due to the multiband predictor and to the optimizations. The LUT approach exploits the
calibration-induced data correlation that is specific to hyperspectral images. To predict a
pixel, the pixel value of the colocated pixel in the previous band is used as a key to search an
LUT. The LUT approach outperforms M-CALIC by more than 0.14 bpp. The LAIS-LUT is
an improved and optimized LUT approach, and it yields a slightly improved performance than
LUT. The LUT-NN approach is slightly worse than the LUT.
C-DPCM is the clustered DPCM and is a very complex predictor, consisting of cluster and
the optimal LP, both of which will consume much time. The bit rate is about 4.68 bpp. Thus,
LP and SLSQ are more complex prediction algorithms, consisting of spatial prediction and
spectral prediction. LP is applied with intraband (IB) (usually noisy and less correlated with
other bands) and prediction threshold. SLSQ uses the same IB set. But SLSQ compression
algorithm is worse than C-DPCM and LUT, which are the most advanced algorithms. The
proposed algorithm achieves a compression rate of 4.56 bpp. J PEG-LS is based on the
predictive coding technique, where the main compression phases are prediction, context
modeling, error encoding, and run mode. J PEG-LS encodes each of the scenes independently.
A bit rate up to 6.62 bpp can be observed. From the results in Table 1, the proposed hybrid
context prediction algorithm has the best bit-per-pixel performance. The hybrid context
prediction achieves the lowest compression rate among all the tested algorithms.
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8203
Table 1. Bit Rate Comparision of All Scenes (in Bits Per Pixel)
Algorithm Lunar Lake Cuprite J asper Ridge Moffett field Average
J PEG-LS 6.61 6.57 6.65 6.67 6.62
3D-CALIC 5.12 5.20 5.15 4.98 5.11
M-CALIC 4.90 4.89 5.01 4.82 4.91
LUT 4.67 4.59 4.87 4.92 4.77
LAIS-LUT 4.55 4.43 4.70 4.69 4.59
LUT-NN 4.72 4.66 4.87 4.90 4.79
DPCM(C-DPCM) 4.70 4.65 4.70 4.63 4.67
LP 4.66 4.71 4.66 4.70 4.68
SLSQ 4.97 4.92 4.97 4.96 4.96
Proposed 4.51 4.54 4.63 4.58 4.56
In order to further verify the effectiveness of the proposed algorithm, we calculate the
compression ratios. Table 2 shows the compression ratios of the proposed algorithm, as
compared with other schemes. The results show that the hybrid context prediction algorithm is
able to outperform in terms of compression ratio such state-of-the-art algorithms as LAIS-
LUT and J PEG-LS by 2.2% and 25%, respectively. We can see that the proposed algorithm
has the highest compression ratios, while the J PEG-LS algorithm is the lowest. Therefore, the
proposed hybrid context prediction algorithm is effective.
Table 2. Comparision of CR Result Achieved Using these Algorithms
Algorithm Lunar Lake Cuprite J asper Ridge Moffett field Average
J PEG-LS 2.51 2.52 2.59 2.58 2.55
3D-CALIC 2.85 2.80 2.83 2.93 2.85
M-CALIC 2.98 2.98 2.91 3.02 2.97
LUT 3.11 3.17 2.99 2.96 3.05
LAIS-LUT 3.20 2.29 3.10 3.11 3.17
LUT-NN 3.09 3.12 2.99 2.97 3.04
DPCM(C-DPCM) 3.10 3.13 3.10 3.15 3.12
LP 3.12 3.09 3.13 3.10 3.11
SLSQ 2.93 3.96 2.93 2.94 2.94
Proposed 3.23 3.21 3.15 3.18 3.19
Then we choose AVIRIS Lunar Lake images bands from 121 to 130 for testing. For
convenience, we resample it as 512 614, 16 bpp precision. It can be observed that the
hyperspectral images have strong interband correlation. We use LP, J PEG-LS, LUT, 3D-
CALIC, C-DPCM, and our algorithm to compress the images respectively. Table 3 shows the
entropy of several test images. Ten images from Lunar Lake B121, to Lunar Lake B130 have
been used for examination. They are respectively denoted as B121 B130 in Table 3 for
abbreviation. The results clearly show that the proposed algorithm outperforms any other
scheme examined here by an extent of 4% up to almost 12%. We can see that the proposed
algorithm has the lowest entropy among all other examined algorithms, while the J PEG-LS
algorithm has the highest. Therefore, the proposed hybrid context prediction algorithm is
effective for hyperspectral images compression. Figure 2 shows that our algorithm
outperforms the compared algorithms in reducing the spatial and spectral redundancy.
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8204
Table 3. Comparison of Bands' Average Entropy
Algorithm
Band
Proposed LP 3D-CALIC LUT J PEG-LS C-DPCM
B121 4.63 4.71 4.98 4.73 5.06 4.80
B122 4.62 4.70 4.97 4.72 5.07 4.81
B123 4.63 4.72 4.98 4.73 5.06 4.80
B124 4.62 4.71 4.97 4.72 5.07 4.81
B125 4.63 4.72 4.97 4.73 5.06 4.80
B126 4.62 4.71 4.96 4.73 5.07 4.79
B127 4.61 4.70 4.95 4.71 5.06 4.78
B128 4.62 4.67 4.94 4.70 5.07 4.77
B129 4.59 4.68 4.95 4.69 5.04 4.76
B130 4.58 4.67 4.93 4.67 5.03 4.75

Fig. 2. Performance of different algorithms for AVIRIS lunar lake images bands.
In order to evaluate the complexity of the proposed algorithm, we have run some of the
algorithms on a workstation with Dual Pentium Xeon 2.4-GHz processor and Linux operating
systems. We have measured the CPU time employed by each algorithm by using the clock ()
function and have averaged the obtained values over a number of trials. The results are
reported in Table 4. J PEG-LS was conceived with the aim of low computational complexity,
however its performance is not state-of-the-art for hyperspectral image coding. Other
approaches conceived specifically for hyperspectral image coding have better performances at
a cost of increased computational complexity, such as C-DPCM or 3D-CALIC. Our approach
outperforms all the other tested proposals, and still has a computational complexity
comparable to the simple LP encoder.
Table 4. Encoding Time of Various Algorithms
Algorithms

Band
Encoding time of 224 bands (seconds)
J P
EG-LS
Pro
posed
L
P
C-
DPCM
3D-
CALIC
Lunar Lake 98.2 184.6 191.5 1001.6 1993.5
Cuprite 94.5 177.7 184.3 963.9 1918.4
J asper
Ridge
96.8 181.9 188.7 987.4 1965.1
Average 96.5 181.4 188.2 984.3 1959
4. Conclusion
We have developed a new efficient lossless compression algorithm that consists of intraband
prediction and hybrid context prediction followed by arithmetic coding. It has been tested on
AVIRIS hyperspectral images and produced very high compression ratios due to the
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8205
successful hybrid context prediction. The new algorithm has been evaluated by comparing the
obtained results with results produced by a number of lossless compression algorithms tested
using the same set of hyperspectral images. The primary goal of this work was to develop a
fast and efficiency compression algorithm. Simulation results show that our algorithm
achieved this goal. Promising direction for the future work would be to study new lossless
compression algorithm through bands grouping and bands reordering for hyperspectral
images. As the high sensor-data rates of present and future hyperspectral missions call for
simple and fast compression techniques, the proposed algorithm proved to be a good option
for lossless hyperspectral image compression.
Acknowledgment
This paper is supported by 2009 open fund, Key Lab of Geomathematics of Sichuan Province
(SCSXDZ2009008), and School of Computer Science and Engineering, University of
Electronic Science and Technology of China provide me a lot of help. I thank their
supporting.
#163034 - $15.00 USD Received 16 Feb 2012; revised 12 Mar 2012; accepted 20 Mar 2012; published 23 Mar 2012
(C) 2012 OSA 26 March 2012 / Vol. 20, No. 7 / OPTICS EXPRESS 8206