Computers and Electronics in Agriculture 78 (2011) 3848

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

journal homepage: www.elsevier.com/locate/compag

Detection of common defects on oranges using hyperspectral reectance imaging

Jiangbo Li, Xiuqin Rao , Yibin Ying
College of Biosystems Engineering and Food Science, Zhejiang University, 388 Yuhangtang Road, Hangzhou 310058, China

a r t i c l e

i n f o

Article history:
Received 15 December 2010
Received in revised form 27 May 2011
Accepted 29 May 2011

Keywords:
Hyperspectral imaging
Principal component analysis
Defect detection
Oranges
Ratio images

a b s t r a c t
To detect various common defects on oranges, a hyperspectral imaging system has been built for acquiring reectance images from orange samples in the spectral region between 400 and 1000 nm. Oranges
with insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, phytotoxicity, heterochromatic stripe, and normal surface were studied. Hyperspectral images of samples were
evaluated using principal component analysis (PCA) with the goal of selecting several wavelengths that
could potentially be used in an in-line multispectral imaging system. The third principal component
images using six wavelengths (630, 691, 769, 786, 810 and 875 nm) in the visible spectral (VIS) and
near-infrared (NIR) regions, or the second principal component images using two wavelengths (691
and 769 nm) in VIS region gave better identication results under investigation. However, the stem-ends
were easily confused with defective areas. In order to solve this problem, representative regions of interest (ROIs) reectance spectra of samples with different types of skin conditions were visually analyzed.
The researches revealed that a two-band ratio (R875/R691) image could be used to differentiate stemends from defects effectively. Finally, the detection algorithm of defects was developed based on PCA
and band ratio coupled with a simple thresholding method. For the investigated independent test samples, accuracies of 91.5% and 93.7% with no false positives were achieved for both sets of selected wavelengths using proposed method, respectively. The disadvantage of this algorithm is that it could not
discriminate between different types of defects.
2011 Elsevier B.V. All rights reserved.

1. Introduction
Citrus is one of the major fruit products in China. Its annual production is over 23 million tons. For a long time, the agriculture
industry has attempted to automate fruit sorting in order to
decrease production costs and increase the quality of the production. Fresh market fruits are graded into quality categories according to parameters such as size, color, shape and external defects
(Leemans and Destain, 2004). The rst two quality criteria have
been automated on current commercial graders, but fruits grading
according to the presence of defects is yet very challenging,
although the most advanced machines are also capable of detecting blemishes (Aleixos et al., 2002; Leemans and Destain, 2004;
Blasco et al., 2007a).
Since the color is the natural sense we use to make our rst
evaluation of the quality of fruits, most of the inspection systems
use this information to detect skin defects of fruit (Miller and
Delwiche, 1991; Leemans et al., 1998; Mendoza and Aguilera,
2004; Blasco et al., 2007b). In terms of citrus, Recce et al. (1998)
identied orange defects based on gray value differences between
defects and sound peel in R and G component images. A neural
network classier on rotation invariant transformations was used
Corresponding author. Tel.: +86 0571 86971885.
E-mail address: xqrao@zju.edu.cn (X. Rao).
0168-1699/$ - see front matter 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.compag.2011.05.010

to recognize the radial color variation of the stem region. Diaz

et al. (2004) observed abnormal color on table olive was related
with low values of R and G co-ordinates. Defects were then segmented using a Bayesian model. Three different algorithms were
applied to classify the olives, and results showed that a neural network with a hidden layer was able to classify the olives with an
accuracy of over 90%. Leemans and Destain (2004) compared
defect segmentation results of three models that were generated
using the RGB, RGI and RGBI. The RGBI model, which was found
to be the best, correctly recognized 100% of the stem-ends and
99.1% of defects based on Bayesian classication model. In a recent
study, Blasco et al. (2007b) reported the application of nearinfrared, ultraviolet and uorescence computer vision systems to
detect the external defects of citrus fruits. Although proposed approach could detect different peel conditions, a complex image
processing algorithm affect the real-time application in packinghouse processing line. Kim et al. (2009) used 14 color texture features of grapefruit peel diseases to classify defects based on HIS
model. Lpez-Garca et al. (2010) developed a citrus fruits surface
detection algorithm, which was based on multivariate image analysis strategy and principal component analysis approach. The success ratio for the detection of individual defect was 91.5%. The
classication accuracy was acceptable. However, the complexity
of the algorithm restrained the detection speed.

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

From these studies, the researchers were more interested in

detecting the defects based on color images and more complex
algorithms. In agricultural production systems, the time available
to evaluate individual object normally ranges from tenths to hundredths of a second (Kim et al., 2002). In order to achieve high processing speeds, sometimes inspection system worked with low
resolution images, or used more advanced digital signal processor.
However, the low resolution images reduced the accuracy of the
system, especially for detection of some very small defects, such
as scale infestation on citrus surface, and the advanced hardware
increased cost of system.
Hyperspectral imaging method, which combines the features of
imaging and VIS/NIR spectroscopy to simultaneously acquire spatial and spectral information, has attracted the interest of researchers as a powerful tool for detecting a variety of agricultural
products. Examples include detecting bruises on apples (Xing
et al., 2005; ElMasry et al., 2008), bruises on pickling cucumbers
(Ariana et al., 2006), pork marbling (Qiao et al., 2007), contaminant
detection on poultry carcasses and cantaloupes (Lawrence et al.,
2006; Vargas et al., 2005), pits in tart cherries (Qin and Lu, 2005),
nematodes in cod llets (Heia et al., 2007), cracks in shell eggs
(Lawrence et al., 2008), and so on. In order to detect citrus surface
defects, Gmez-Sanchis et al. (2004, 2008) developed multispectral
imaging to detect infections caused by Penicillium digitatum in citrus fruits. To investigate the detection of canker on citrus fruit surface, Qin et al. (2008, 2009) utilized principal component analysis
(PCA) together with a simple threshold classier and spectral information divergence (SID) classication method to discriminate canker on grapefruit from other peel diseases using hyperspectral
images with an accuracy of 92.7% and 95.2%, respectively. These research works provided good references and resources for dealing
with various problems associated with detection of citrus surface
defects. However, to our knowledge no attempts were made to differentiate different defect types such as insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn,
phytotoxicity, heterochromatic stripe using hyperspectral imaging.
In addition, hyperspectral imaging technology has not yet been directly implemented in an in-line system for automated quality
detection because its time requirements for image acquisition
and analysis are too great (Mehl et al., 2002). Usually, the hyperspectral data can be used to determine optimal wavelengths for
developing multispectral imaging system. The multispectral imaging approach in conjunction with selected wavelengths is favorable
for rapid in-line assessments (Kim et al., 2002, 2005; Liu et al.,
2007).
The main objective of the present study was to investigate the
potential of using a hyperspectral imaging for detecting various
common defects on orange surfaces. For this purpose, the research
was conducted through (1) development of a hyperspectral imaging system with a spectral region from 400 to 1000 nm to detect
various common defects on orange surfaces; (2) determination of
effective wavelengths for skin defects detection based on principal
component analysis method; (3) development of an algorithm to
identify stem-ends from true defects; (4) development of a simple
algorithm to isolate defected areas from sound surfaces. The ultimate purpose was to develop a faster and more efcient multispectral method for real-time inspection of orange defects.

2. Materials and methods

2.1. Orange samples
The orange samples were hand picked from two different commercial orchards in Jiangxi during the harvest season of 2009 and
2010, respectively. A total of 460 samples were separated into

39

non-defect and defect groups by visual inspection. The defects included insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, heterochromatic stripe and
phytotoxicity. The diseases on the fruit surfaces show different
symptoms. Wind scar, which is caused by leaves, twigs, or thorns
rubbing against the fruit, is a common physical injury on the fruit
peel, and the scar tissue is generally gray (Qin et al., 2009). Scale
infestation, insect damage, and thrips scarring are caused by insect,
which generate surface blemishes during the fruit growing season.
The armor of the scale is 23 mm long, dark brown in appearance.
Lesions of insect damage and thrips scarring are caused by some
insect bites and the color of surface blemishes are gray and brown,
respectively. Heterochromatic stripe fruit is characterized by nonuniform surface color. Those regions commonly appear as deep
yellow stripe shape. Citrus canker is caused by bacteria, and it is
featured with conspicuous dark lesions. Most circular in shape,
canker lesions vary in number, and they are supercial (up to
1 mm deep) on the fruit peel (Qin et al., 2009; Schubert et al.,
2001). Number of the canker lesions on every orange surface in this
study was approximately in the range of 527. Copper burn scar, as
a non-infectious disease, is caused by high temperature from long
time sun exposure. The affected areas exhibit tan lesions in appearance. Phytotoxicity is caused by residual pesticide and exhibit
brown around fruit calyx (Li et al., 2010). The area of infected region is bigger than one-seventh of fruit surface area in this study.
Fig. 1 shows samples with various common peel defects and
stem-end.
Twenty samples of each type of skin defect and thirty samples
of normal oranges were selected from the 2009 data set as training
set and used to develop the algorithm. In addition, thirty samples
of each peel type were selected from the 2010 data set as test set
and used to evaluate the performance of algorithm for this study.
All the samples were washed and treated with chlorine and
sodium o-phenylphenate (SOPP) at Machine Vision Lab of College
of Biosystems Engineering and Food Science, Zhejiang University,
Hangzhou. These samples were then stored in an environmental
control chamber maintained at 5 C and they were removed from
cold storage about 2 h before imaging to allow them to reach room
temperature.
2.2. Hyperspectral imaging system
A schematic diagram of developed hyperspectral imaging system with a spectral resolution of approximately 0.58 nm is shown
in Fig. 2. The system consists of ve components: an imaging spectrograph (ImSpector V10E-QE, Spectral Imaging Ltd., Oulu, Finland)
coupled with a standard C-mount zoom lens (V23-f/2.4, Specim
Ltd., Finland), two 150 Watt (W) halogen lamps assemblies
(SCHOTT DCR III, SCHOTT North America, Inc., USA) provided a uniform VISNIR illumination for the sample in the eld of view of the
optics, a Hamamastsu monochrome linear CCD camera (C848405G, Hamamatsu Photonics, Japan) with 1344 effective pixels, a
sample transportation plate (PSA200-11-X, Zolix Instruments Co.,
Ltd., Beijing, China), and a computer (ACER, InterCoreTM2
4400@2.00 GHz, RAM 1.00G). The spectrograph has a prism-grating-prism (PGP) element, which is a holographic transmission grating. During measurements with the spectrograph-camera
assembly, the system is well shielded from the environment to
minimize interference from ambient light (Wallays et al., 2009).
The PGP disperses the incoming light corresponding to a pixel in
the scanned line into its spectral components and projects the
range from 400 to 1000 nm onto the CCD of the camera. The size
of the acquired images is 1344 by 700 pixels with a resolution of
9.25 pixels/mm. The wavelength range between 550 nm and
900 nm was used in this investigation due to inefciencies of the
system at certain wavelength regions (e.g., low light output in

40

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

Fig. 1. Different peel types. From top to bottom and left to right: insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, heterochromatic
stripe, phytotoxicity and stem-end.

Due to the uneven intensity of light source in different bands

and the existence of dark current in CCD camera, some bands with
less light intensity acquired the bigger noises (Polder et al., 2003).
Therefore, the hyperspectral images need to be calibrated with a
white and a dark references. The dark reference was used to
remove the dark current effect of the CCD detectors, which are
thermally sensitive. The dark image (with 0% reectance) was collected by turning off all light sources and covered the lens with a
black cap. A Teon white board with the 99% reection efciency
(Spectralon, Labsphere Inc.) was used to obtain white reference image. The corrected image (R) was calculated using Eq. (1) (Mehl
et al., 2002; Xing et al., 2005; ElMasry et al., 2009):

Ro  Rd
Rr  Rd

where Ro is the acquired original hyperspectral image, Rr is the

white reference image, Rd is the dark image.
Fig. 2. Schematic of hyperspectral imaging system.

the VIS < 550 nm, and low quantum efciency of the CCD in the
NIR > 900 nm).
2.3. Hyperspectral image acquisition
The laboratory-based system was operated in a darkened
inspection chamber where only the halogen light sources were
used. In order to acquire more accurate data, some parameters
need be adjusted before the images were acquired. In this study,
the exposure time was adjusted to 200 ms and the speed of fruit
movement was adjusted to 0.12 cm/s throughout the test. The object distance was set 42.5 cm. The average illumination intensity
from samples surface was 27333 lux acquired through a lux gauge
(TES1336A). During image acquisition, the orange samples were
placed on a tray painted with at black paint that were xed on
the positioning table (Fig. 2), manually orienting the side of the
fruit that contained the defects towards the camera. The camera
and spectrograph were then used to scan the oranges line-by-line
as the transportation plate moved the oranges through the eld of
view of the optical system. The line scan data were saved and processed later to create hyperspectral image cubes containing spatial
and spectral data. The spectral images were acquired based on
Spectral Cube_v2_75 software (Spectral Imaging Ltd., Finland).
However, after images were acquired, we found that several defective areas located on the border of the fruit in acquired images due
to human fatigue error. These images were also used in our study.

2.4. Data processing and analyzing

All data processing and analyzing were performed using the
Environment for Visualizing Images software program (ENVI 4.6,
Research System Inc., Boulder, CO, USA) and Matlab 2008a (The
MathWorks Inc., Natick, USA) with the image processing toolbox.
This research uses the ENVI software package for the application of PCA to the hyperspectral images of oranges. In the process
of creating the PCA images, a correlation matrix of the image is calculated. This correlation matrix is then used to compute the eigenvalues. The eigenvalues are equivalent to the variance of each
principal component (PC) image. These PC images are ordered in
the decreasing degree of variance sizes, where rst PC accounts
for the largest variance (Liu et al., 2007). The more details on this
method can be found in Malinowski and Howery (1980).
In this study, the PCA was used to reduce spectral dimensionality of the hyperspectral reectance images, and to aid in visualizing
the hyperspectral data to determine the several dominant wavebands responsible for discriminating defects from normal orange
surfaces.
Before applying the PCA and band ratio, a binary mask was created to produce an image containing only the fruit, avoiding any
interference from the background that could reduce discrimination
efciency. Image at 750 nm was used for this task because it appeared the best contrast between the fruit surface and the background and can be segmented easily by setting a simple
threshold value. Subsequently, individual principal component
(PC) images were visually evaluated to determine PC images with
(1) the least variation in normal orange surfaces and (2) the largest

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

contrast between defective areas and sample surfaces. Each PC

image is a linear sum of the original images at individual wavelengths multiplied by corresponding (spectral) weighing coefcients. Several wavelengths with high (local maxima) and low
(local minimum) weighing coefcients from the PC image were selected as the dominant wavelengths (Vargas et al., 2005). Principal
components images using only the selected dominant wavelengths
(multispectral images) were re-calculated. Because band ratio
images can effectively enhance the contrast between different regions and produce more uniform responses across the orange surface (Vargas et al., 2005), the two-band ratio method was also used
in this study. The two-band ratio was performed as Eq. (2):

Q t=k

Rt
Rk

where Qt/k represents a quotient of spectral reectances, and Rt and

Rk are reectance intensities at t nm and k nm, respectively.
Finally, the multispectral PC images and two-band ratio images
were combined and subjected to a simple thresholding method to
segment the defective areas from the normal areas.
In addition, morphological ltering was used during defects
segmentation with an aim of removing undesired small size pixels
(noises) in the binary images. One step morphological opening
operation based on a rectangle structuring element with a 3  3
kernel size in MATLAB morphological lter tools were used in this
study and dened as erosion of the image by structuring element,
followed by a dilation of the result by same structuring element.
Structuring element with a 3  3 kernel size was used mainly considering effectively removing the noises and retaining some small
defects like scale infestation as many as possible in defect binary
image.
3. Results and discussion
3.1. Hyperspectral reectance spectra
The representative regions of interest (ROIs) reectance spectra
of orange samples studied in the wavelength range between 550
and 900 nm are shown in Fig. 3. These spectra were extracted from
the hyperspectral image data of training set and were an average of
twenty spectra (one per orange) for each type of peel condition, except for stem-end and sound orange which were used to obtain the
mean spectra from fteen spectra (one per orange) per type. Each
spectrum was obtained from a rectangular ROI, varying in size
from 80 to 100 pixels.
The reectance of spectra depicted in the VIS region was lower
than in the NIR region over the entire spectral region. The spectra
of sound peel showed higher reectance comparing to the

Reflectance(%)

60

Insect damage
Stem
Scale infestation
Heterochromatic stripe
Copper burn

Wind carring
Thrips scarring
Phytotoxicity
Canker spot
Sound

50
40
30
20
10
0
550

defective peels and stem-ends in the spectral region between

600 and 775 nm. Therefore, the bands from VIS region could be
more adequate for defects detection than NIR region bands. However, the spectra shown in Fig. 3 do not account for the spatial variations (decreases) in intensities from the center portions toward
the edges. The single band image was also attempted to segment
different types of defects studied by using a simple thresholding.
The results showed that it was hardly to get the target due to variations in the gray level within the defective area and the surrounding surface, which was in agreement with results achieved by
Benneden and Peterson (2005).
On the one hand, it was reasonable to observe that the spectral
features of heterochromatic stripe fruit were similar to sound fruit
surfaces, which made the detection of this defect very difcult. On
the other hand, it was also practically very difcult to identify heterochromatic stripe area from sound skins based on RGB images
due to the color similarity (Fig. 1). In addition, one common feature
also observed from stem-end and phytotoxicity in the spectra is
absorption of chlorophyll a at approximately 689 nm. The spectra
of most defective surfaces have maxima located approximately at
810 and 875 nm, respectively. On the basis of these features, band
ratio algorithms could be promoting to identify stem-ends from
other defects.
3.2. PCA in the VIS to NIR region of the spectrum
The rst three images (denoted by PC-1 to PC-3) obtained from
PCA for the hyperspectral reectance images of oranges samples
using all 599 wavelengths in the region from 550 to 900 nm are
shown in Fig. 4. The PCA provides a means to reduce the high spectral dimensionality of image data. Features such as wind scarring
and heterochromatic stripe which are not readily visible in the
individual images are more apparent in these images. In the rst
principal component (PC-1) images, intensity decreases from the
center to the edges of oranges, and they do not provide more unique features than the original untransformed hyperspectral
images. The PC-2 images demonstrate more information of defects
on fruit surfaces. The defective regions could be clearly identied
in the PC-2, especially for insect damage, wind scarring, thrips
scarring, copper burn, phytotoxicity and heterochromatic stripe.
In the PC-3 images, only the insect damage and heterochromatic
stripe could be more effectively identied comparing to PC-2.
Starting from PC-4 (gures not shown), the transformed images
no longer possess meaningful information, and they are not useful
for oranges surface defects detection. Based on visual assessment,
the PC-2 images provide the better peel defects discrimination.
However, the noises are also observed in some sample surfaces,
such as insect damage, canker spot and stem-end. The main reason
may be due to too many bands used to perform PCA.
3.3. PCA in the VIS region of the spectrum

80
70

41

600

650

700

750

800

850

900

Wavelength(nm)
Fig. 3. Representative ROIs reectance spectra obtained from orange samples with
different types of peel conditions.

The gray levels variations between defective and sound skin regions were mainly affected by the spectra from VIS region. The
spectra in the NIR region were usually not sensitive to the variations. Thus, PCA on the full wavelength region (VIS to NIR) may
weaken the contrast of different regions on fruit surface. In general,
the band region from 380 to 780 nm is known as VIS region
(Hernndez-Andrs et al., 2001; Skoglund et al., 2004). In addition,
the image information below 600 nm shows poor features in our
study. The poor features may be brought because chemical components in the orange peel are not sensitive to this waveband region.
Therefore, the hyperspectral images in the wavelength range of
600780 nm were used to perform PCA for further analysis.
Fig. 5 illustrates representative PC-1, PC-2 and PC-3 images
obtained from the PCA of the 600780 nm hyperspectral

42

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

Fig. 4. First three principal component images obtained using the entire spectral region from 550 to 900 nm for (a) insect damage, (b) wind scarring, (c)thrips scarring,
(d)scale infestation, (e)canker spot, (f) copper burn, (g) phytotoxicity, (h) heterochromatic stripe and (i) stem-end. PC-1PC-3 are the rst, second and third principal
components, respectively.

reectance image data. The PC-1 images reected a weighted sum

and showed effects similar to those observed in the entire spectral
region (Fig. 4). Subsequent PC-2 images depicted the best contrast
between skin defects and normal orange surfaces for different
types of samples. PC-3 images did not give any useful information
for defects identication. Compared to PC-2 images in Fig. 4, the
PC-2 images shown in Fig. 5 were more effective to identify various
peel defects. It was also noticed that some obvious noises remained
on orange surfaces, such as normal surface with stem-end. These
noises were easy to be misclassied as defects (false positive). In
addition, using too many wavelengths (309 spectral channels with
a spectral resolution of approximately 0.58 nm) was also not effective to develop multispectral system for defects detection.
3.4. Selection of optimal spectral wavebands
The weighing coefcients for the PC-2 obtained by using images
across the entire spectral region are shown in Fig. 6a for wind scarring, insect damage, scale infestation and thrips scarring, and 6b for
canker spot, copper burn, heterochromatic stripe, phytotoxicity
and stem-end, respectively. The weighing coefcients for the PC2 obtained by using images from VIS region are shown in Fig. 6c
and d, respectively. The peaks and valleys indicated the dominant
wavelengths. Therefore, six wavebands from 550 to 900 nm were

chosen, which were centered at around 630, 691, 769, 786, 810
and 875 nm, respectively. In addition, two wavelengths at around
691 and 769 nm from VIS region were also singled out for further
analysis.
3.5. PCA on selected optimal wavebands
The principal components analysis was rst carried out on the
six optimal wavelengths (630, 691, 769, 786, 810 and 875 nm) instead of the full wavelength range. The rst ve PC images are
shown in Fig. 7. A visual inspection of the ve PC images revealed
the major features such as defected regions and stem-ends became
more evident in these transformed images except for PC-1 and PC5. The useful image features were enhanced from PC-2 to PC-4.
Since each PC image shown in Fig. 7 is a linear sum of the original images at six optimal wavelengths multiplied by corresponding
weighing coefcients, the intensity value of defective areas in the
PC images changed with different PC images. As shown in Fig. 7,
the defected regions of fruit surfaces appear dark in PC-2 and PC3 images, and turn bright in PC-4 images. The surface defects in
PC-5 images are not as evident as those in PC-2, PC-3 and PC-4
images because of the relatively low contrasts between the defective regions and the sound skins. The resultant multispectral PC
images (i.e., PC-1, PC-2 and PC-3 in Fig. 7) obtained from the

Fig. 5. First three principal component images obtained using the VIS region from 600 to 780 nm for (a) insect damage, (b) wind scarring, (c) thrips scarring, (d) scale
infestation, (e) canker spot, (f) copper burn, (g) phytotoxicity, (h) heterochromatic stripe and (i) stem-end. PC-1PC-3 are the rst, second and third principal components,
respectively.

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

43

Fig. 6. Weighing coefcients for the PC-2 that resulted from using the full wavelength region (550900 nm) and the VIS wavelength region (600780 nm), respectively: (a)
and (c) for wind scarring, insect damage, scale infestation and thrips scarring, and (b) and (d) for canker spot, copper burn, heterochromatic stripe, phytotoxicity and stemend.

multispectral PCA also gave similar characteristics in appearance

to those (i.e., PC-1, PC-2 and PC-3 in Fig. 4) obtained on the full
wavelength region. However, the multispectral PC images were
more effective to identify defects due to decreasing noises from
too many bands. In addition, less spectral bands were desired to
develop robust and rapid multispectral imaging systems suitable
for skin defects detection. The resultant PC-2, PC-3 and PC-4
images shown in Fig. 7 clearly demonstrated that they all may be
used to segment defects. However, compared to PC-3 images, the
PC-2 and PC-4 were more sensitive to illumination variations.
PC-2, for example, the lighter region (center portions) of original
images appeared lower intensities than other part after performing
the PCA. The sensibility suggested that the PC-2 and PC-4 may be
inadequate for very reliable detection of various skin defects. Based
on the inspection and the comparisons above, the PC-3 images
showed great potential for discriminating various common orange
peel defects from sound skins. Therefore, PC-3 images were chosen
for performing the images classication.
Fig. 8 illustrates the second principal component images (PC-2)
obtained by PCA of hyperspectral reectance images using two selected wavebands (691 and 769 nm) images in the VIS region. Because the PC-1 for two wavebands (691 and 769 nm) exhibited
similar results in appearance to those of the corresponding PC-1
images shown in Figs. 4, 5 and 7, only PC-2 images are shown in
Fig. 8. Based on visual assessment, the PC-2 (Fig. 8) appeared to
also provide the best orange peel defects detection regardless of
defect types and illumination variations (Note that a dome lamp
can be effective to prevent the effect from illumination variations,

which is not the aim of this work). Therefore, PC-2 (Fig. 8) images
were also chosen for further analysis.
3.6. Band ratio images for stem-ends identication
In the PC images (i.e., PC-2, PC-3 and PC-4 in Fig. 7 or PC-2 in
Fig. 8), it was very easy to observe that the characteristics of
stem-ends resembled those of the peel defects. Therefore, the
stem-ends could be misclassied as defects (false positive). Based
on the spectral responses in Fig. 3, the two-band ratio using 689
and 810 nm pair or 689 and 875 nm pair could be potential to
detect stem-ends because of the reectance spectra of stem-ends
region exhibiting local minimum value at 689 nm, and local maximum values at 810 and 875 nm. To develop robust and rapid multispectral imaging systems suitable for skin defects detection, less
spectral bands were desired. On the one hand, the wavelength at
689 nm was replaced by a wavelength at 691 nm. On the other
hand, two-band ratio images based on wavelengths at 691 and
769 nm were also observed. The ratio images were obtained and
shown in Fig. 9.
Compared to the resultant images, the R769/R691 and R810/
R691 ratio images were ineffective in producing images for distinguishing the stem-end from most surface defects. On the contrary,
the R875/R691 ratio images showed clear stem-end region (white
area in Fig. 9), which suggested much promise in the detection of
stem-ends because of most peel defects regions exhibiting darker
features. In the study, it was found that sometimes canker spots
and phytotoxicity regions could give similar intensity value to

44

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

Fig. 7. Principal component images based on the six selected wavebands (630, 691, 769, 786, 810 and 875 nm) for (a) insect damage, (b) wind scarring, (c) thrips scarring, (d)
scale infestation, (e) canker spot, (f) copper burn, (g) phytotoxicity, (h) heterochromatic stripe and (i) stem-end. PC-1PC-5 are the rst, second, third, fourth and fth
principal components, respectively.

Fig. 8. The second principal component images based on the two selected wavebands (691 and 769 nm) in the VIS region for (a) insect damage, (b) wind scarring, (c) thrips
scarring, (d) scale infestation, (e) canker spot, (f) copper burn, (g) phytotoxicity, (h) heterochromatic stripe and (i) stem-end.

Fig. 9. Representative two-band ratio images (R769/R691, R810/R691and R875/R691) for (a) insect damage, (b) wind scarring, (c) thrips scarring, (d) scale infestation, (e)
canker spot, (f) copper burn, (g) phytotoxicity, (h) heterochromatic stripe and (i) stem-end.

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

stem-end in R875/R691 ratio images due to the two types of defects usually appearing slightly green skin characteristic. However,
in most situations, the stem-ends were signicantly lighter than
canker spots and phytotoxicity region in the R875/R691 ratio
images due to greener color characteristic for stem-ends region.
3.7. Defects detection algorithm
Fig. 10 demonstrates major procedures for multispectral images
processing and detection for thrips scarring identication using an
orange sample with stem-end and thrips scarring scar on the fruit.
Based on analysis in Section 3.5 and 3.6, two wavebands images at
691 nm and 769 nm and two wavebands images at 691 nm and
875 nm are effective for detection of defects and identication of
stem-end, respectively. Therefore, three wavebands (691, 769
and 875 nm) images were used in Fig. 10. Firstly, a mask template
was created using a single-band image at 750 nm, and three wavebands (691, 769 and 875 nm) images were masked using the template to exclude the background that could affect the results. Then,
on the one hand, the images at 691 and 875 nm were rst used to
produce ratio image (R875/R691). Afterwards, a global threshold
value of 0.8 was applied to the ratio image to separate stem-end
from the fruit surface. On the other hand, the PCA based on two
images at 691 and 769 nm was also performed to generate two
PC images (PC-1 and PC-2). Subsequently, the addition operation
was performed between stem-end binary image and PC-2 image
to increase stem-end gray value (stem-end gray compensation).
Finally, the compensated PC-2 image was subjected to a simple
thresholding method with a global threshold value of 0.23 to separate the defective skins from the normal fruit surfaces. Note that
ow charts of the key steps for defects detection algorithm based
on PCA from six (630, 691, 769, 786, 810 and 875 nm) and two
(691 and 769 nm) optimal wavelengths are similar. The only difference is if the PC images are generated by six wavebands, the addition operation was performed using stem-end binary image and
resultant PC-3 image.
In some cases, there are some very minor defects/blemishes
(noises) on normal orange surfaces. Because these noises may carry
the same spectral signature as defects detected in this work, some
normal orange could be misidentied as defected oranges. In order
to overcome this problem, one step morphological opening
operation (Erosion and Dilation) and two steps dilation operation

45

for stem-end binary images and single one step morphological

opening operation for defect binary images was used to remove
these noises (Fig. 10). Note that two steps dilation operation for
stem-end binary image is used in order to compensate the stemend as completely as possible in addition operation. The resultant
binary image with segmented defects is shown at the left bottom
image in Fig. 10. Two global threshold values of 0.8 and 0.23 used
in the algorithm was chosen based on spatial prole plots for
stem-end and defects. Fig. 11 shows R875/R691 ratio image and
compensated PC-2 image in Fig. 10. The plots adjacent to these
two images illustrate spatial variations in intensity values (percent
reectance) along the white lines in the images, respectively. The
dotted horizontal lines in the prole plots represent threshold lines
for stem-end and thrips scarring segmentation, respectively. Note
that although only one representative orange with stem-end and
defect are shown in Fig. 11, selection of threshold values are generally based on the results observed in all training samples.
In terms of stem-ends identication, canker spots and phytotoxicity for R875/R691 images sometimes could be misclassied
as stem-ends if only intensity information was used. However,
the area of region infected by phytotoxicity is usually signicantly
bigger than stem-end area and the number of canker spots on the
orange surface is more than one. Thus, the size and number of
marked region for stem-end binary image, as two characteristics,
can be used to further identify stem-end. Firstly, the stem-end
binary image (Fig. 10) was obtained and marked. If only one region in the stem-end binary image was marked and its size was
less than 278 pixels (Note that a stem-end region vary in size
from 164 to 275 pixels in this research), this region was classied
as stem-end. Then, the addition operation for stem-end gray value compensation was performed before defects segmentation.
Otherwise, any other case, the addition operation was not performed and corresponding PC images (PC-3 for six bands and
PC-2 for two bands) directly were used to segment defects. By
doing this, although there was a very low percentage of false
detections for samples with canker spots and phytotoxicity (Note
that the false detections are acceptable due to only occurring in
defective fruit), the classication procedure above-mentioned
can be simplied.
An example of analysis performed by the proposed approach for
each peel type is shown in Fig. 12. The illustrated nine samples
were randomly selected from test set. In order to obtain more

Fig. 10. Flow chart of the key steps involved in orange peel defects detection algorithm.

46

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

Fig. 11. The two-band ratio image (R875/R691) (a) and compensated PC-2 image (b) in Fig. 10. The images are accompanied by spatial prole plots for the white horizontal
lines on the orange, respectively. Note that the white lines transect the stem-end and thrips scarring on the orange, respectively.

Fig. 12. Example of defects detection for (a) insect damage, (b) wind scarring, (c) thrips scarring, (d) scale infestation, (e) canker spot, (f) copper burn, (g) phytotoxicity, (h)
heterochromatic stripe and (i) stem-end.

47

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

distinct contrast, RGB images of fruit were showed in the rst row,
and the defective regions were marked manually using a graphical
tool. Second row shows the two-band ratio images (R875/R691).
Third and fourth rows correspond to PC-2 images obtained using
the two selected wavebands (691 and 769 nm) in the VIS region
and PC-3 images obtained using six selected wavebands (630,
691, 769, 786, 810 and 875 nm) in the full wavelength region,
respectively. Fifth row corresponds to binary images obtained
using developed algorithm based on two-band ratio (R875/R691)
images and PC-2 images. Sixth row corresponds to binary images
obtained using developed algorithm based on two-band ratio
(R875/R691) images and PC-3 images. It can be noticed that defects
positions in the RGB images are different with those in the PC and
ratio images because the fruit was imaged using the different cameras (hyperspectral imaging cameras and RGB cameras) in different
inspection chambers, so moved from one image to other but presenting the same defects images for each type. In the binary
images, the normal fruit surfaces and stem-end were converted
to zero, and the remaining white regions represented defects isolated from the normal fruit surfaces, showing the effectiveness of
the proposed classication algorithm for defects detection.
3.8. Identication results
The algorithms for multispectral image processing and classication described above were evaluated using 270 independent
samples. The test results for different types of defects are shown
in Table 1. A total of 270 fruit samples were divided into two classes: Defected class including 240 samples with eight types of skin
defects, 30 samples for each type, and Normal class including 15
samples with stem-end and 15 samples without stem-end. All
samples were evaluated using two detection methods: one was
two-band ratio (R875/R691) and PCA using two wavelengths
(691 and 769 nm), the other was two-band ratio (R875/R691)
and PCA using six wavelengths (630, 691, 769, 786, 810 and
875 nm). As shown in Table 1, the overall detection accuracy for
the tested samples was 93.7% and 91.5% with no false positives
(0 out of 30 normal oranges) based on two methods, respectively.
In terms of the rst detection method, all Scale infestation,
Copper burn and Normal samples were correctly identied under
T1 = 0.8, T2 = 0.23. Note that the T1 and T2 represent the threshold
values applied to two-band ratio images and PC images (see
Fig. 11), respectively. Seventeen samples were undetected, including two Insect damage samples, three Wind scarring samples,

two Thrips scarring samples, two Canker spot samples, one Phytotoxicity sample, and seven Heterochromatic stripe samples.
The reectance properties for heterochromatic stripe was close to
those of normal skin (see Fig. 3), especially for the lightly heterochromatic stripe. That probably is the reason for the lowest identication rate for this case. However, if identication accuracy is not
required to be very high, fruit with heterochromatic stripe is usually considered as sound fruit. For undetected wind scarring, canker spot, one insect damage, and phytotoxicity samples, the
lightly defects presented on orange surfaces. After PCA was performed, the contrast between defective areas and sound skins is
lower. In this study, we also attempted to increase threshold value
T2 from 0.23 to 0.39 in order to improve detection rate of defective
oranges. However, although the defects detection rate was improved from 92.9% to 95.4%, it was also found in this investigation
that false positive rate was increased from 0% to 3.3% along with
increasing the threshold value, showing the selection of threshold
values is very important in defects detection. The cases of incorrect
detection of thrips scarring and other insect damage are different:
what happened here is that these damages located on the border of
the fruit in the detected image, indicating that the identication
algorithm could be affected by the position of defected regions
on fruit surface in the detected image. Therefore, the accuracy for
the identication using the simple global thresholding method
may be decreased for detecting the orange samples when defects
located on the border of the fruit. In general, two strategies could
be potential to resolve this problem. One method proposed in
many literatures was performed by simply not inspecting the borders of the fruit in the images. Not inspecting the borders, however,
does not imply losing efcacy, since in the in-line automatic computer vision systems for fruit inspection, the fruits rotate, thus
allowing the system to inspect most of the fruit surfaces by acquiring different images while they pass below the camera (Blasco
et al., 2007a). The other method was that local threshold value
was applied instead of global threshold method. In order to use this
principle, the fruit detected was rst segmented into two parts,
border region (circular region) and remaining region (middle region). Then, different threshold values were applied to two regions.
4. Conclusions
In this investigation, hyperspectral reectance images were
evaluated for detecting various common defects on the orange
surface in the wavelength range between 550 and 900 nm. This

Table 1
Test results for developed algorithms based on 270 independent samples with different defects types and normal surfaces.
Class

Defected (n = 240)

Normal (n = 30)
Total
a

Peel types

Insect damage
Wind scarring
Thrips scarring
Scale infestation
Canker spot
Copper burn
Phytotoxicity
Heterochromatic stripe
Normal with stem
Normal without stem
9

Number

Misclassied

Accuracy (%)

BRa+PCA
(6 bandsb)

BR + PCA
(3 bandsb)

BR + PCA
(6 bands)

BR + PCA
(3 bands)

30
30
30
30
30
30
30
30
15
15

2(2)
3(1)
2(2)
0(0)
3(2)
0(0)
2(0)
11(9)
0(1)
0(0)

2(2)
3(1)
2(2)
0(0)
2(0)
0(0)
1(0)
7(6)
0(1)
0(0)

93.3c (93.3)d
90.0 (96.7)
93.3 (93.3)
100.0 (100.0)
90.0 (93.3)
100.0 (100.0)
93.3 (100.0)
63.3 (70.0)
100.0 (93.3)
100.0 (100.0)

93.3 (93.3)
90.0 (96.7)
93.3 (93.3)
100.0 (100.0)
93.3 (100.0)
100.0 (100.0)
96.7 (100.0)
76.7 (80.0)
100.0 (93.3)
100.0 (100.0)

270

23(17)

17(12)

91.5 (93.7)

90.4
(93.3)

100
(96.7)

92.9
(95.4)

100
(96.7)

93.7 (95.6)

The BR means the band ratio algorithm.

The 6 bands and 3 bands mean six wavelengths (685, 710, 769, 786, 810 and 875 nm) and the three wavelengths (691, 769 and 875 nm), respectively.
The accuracies without brackets were obtained under T1 = 0.8, T2 = 0.23. The T1 and T2 represent the threshold values applied to two-band ratio images and PC images,
respectively.
d
The accuracies inside the brackets were obtained under T1 = 0.8, T2 = 0.39. The T1 and T2 represent the threshold values applied to two-band ratio images and PC images,
respectively.
b

48

J. Li et al. / Computers and Electronics in Agriculture 78 (2011) 3848

investigation illustrated that high spectral dimensionality of reectance images data were reduced to few optimal wavelengths to
form multispectral images by using the PCA method. We identied
six wavelengths (630, 691, 769, 786, 810 and 875 nm) and, alternatively, three wavelengths (691 and 769 and 875 nm) could potentially be implemented in multispectral imaging systems for
detection of orange peel defects. The two-band ratio and PCA coupled with a simple threshold method achieved the best 93.7%
orange surface defects identication accuracy and no false positives. It should also be pointed out that simple two-band ratio
(R875/R691) algorithm could be more effective to identify
stem-ends from skin defects compared to pattern recognition
algorithms, which increase the computational complexity.
Although the results shown in this study were further manifestation that multispectral reectance techniques based on band
ratio and PCA method for detection of various common defects
on orange surfaces in practical applications may be feasible. However, because of the limited sample size for each type defect and
skin defect types, it could not indicate that wavelengths at 691,
769 and 875 nm were the best wavelengths to develop algorithm
for all defect types of citrus fruit. Therefore, further research will
focus on (1) increasing the sample size and collecting samples with
other types of skin defects, such as anthracnose and sooty mold, to
make the algorithm more robust; (2) applying this procedure to
other citrus fruit such as tangerine, pomelo and lemon; and (3)
developing an efcient multispectral imaging system, including
developing a better classication algorithm in conjunction with
morphological information of defects. In addition, multispectral
uorescence imaging techniques will be also tried as aid-system
for detection of surface defects in citrus.
Acknowledgements
The authors gratefully acknowledge the nancial support provided by National Natural Science Foundation of China (No.
30825027).
References
Aleixos, N., Blasco, J., Navarrn, F., Molt, E., 2002. Multispectral inspection of citrus
in real-time using machine vision and digital signal processors. Computers and
Electronics in Agriculture 33 (2), 121137.
Ariana, D., Lu, R., Guyer, D.E., 2006. Hyperspectral reectance imaging for detection
of bruises on pickling cucumbers. Computers and Electronics in Agriculture 53
(1), 6070.
Benneden, B.S., Peterson, D.L., 2005. Performance of a system for apple surface
defect identication in near-infrared images. Biosystems Engineering 90 (4),
419431.
Blasco, J., Aleixos, N., Molt, E., 2007a. Computer vision detection of peel defects in
citrus by means of a region oriented segmentation algorithm. Journal of Food
Engineering 81 (3), 535543.
Blasco, J., Aleixos, N., Gmez, J., Molt, E., 2007b. Citrus sorting by identication of
the most common defects using multispectral computer vision. Journal of Food
Engineering 83 (3), 384393.
Diaz, R., Gil, L., Serrano, C., Blasco, M., Molt, E., Blasco, J., 2004. Comparison of three
algorithms in the classication of table olives by means of computer vision.
Journal of Food Engineering 61 (1), 101107.
ElMasry, G., Wang, N., Vigneault, C., Qiao, J., ElSayed, A., 2008. Early detection of
apple bruises on different background colors using hyperspectral imaging. LWT
Food Science and Technology 41 (2), 337345.
ElMasry, G., Wang, N., Vigneault, C., 2009. Detecting chilling injury in Red Delicious
apple using hyperspectral imaging and neural networks. Postharvest Biology
and Technology 52 (1), 18.
Gmez-Sanchis, J., Molt, E., Gomez-Chova, L., Aleixos, N., Camps-Valls, G., Juste, F.,
2004. Hyperspectral computer vision system for the detection of Penicillium
digitatum in citrus packing lines. In: 2004 CIGR International Conference,
Beijing, China, 1114 October, 2004.
Gmez-Sanchis, J., Gmez-Chova, L., Aleixos, N., Camps-Valls, G., MontesinosHerrero, C., Molt, E., Blasco, J., 2008. Hyperspectral system for early detection

of rottenness caused by Penicillium digitatum in mandarins. Journal of Food

Engineering 89 (1), 8086.
Heia, K., Sivertsen, A., Stormo, S., Elvevoll, E., Wold, J., Nilsen, H., 2007. Detection of
nematodes in Cod (Gadus morhua) llets by imaging spectroscopy. Journal of
Food Engineering 72 (1), E011E015.
Hernndez-Andrs, J., Romero, J., Nieves, J.L., 2001. Color and spectral analysis of
daylight in southern Europe. Journal of the Optical Society of America a-Opticals
Image Science and Vision 18 (6), 13251335.
Kim, M.S., Lefcourt, A.M., Chao, K., Chen, Y.R., Kim, I., Chan, D.E., 2002. Multispectral
detection of fecal contamination on apples based on hyperspectral imagery:
part I. Application of visible and near-infrared reectance imaging. Transactions
of the ASAE 45 (6), 20272037.
Kim, M.S., Lefcourt, A.M., Chen, Y.R., Yang, T., 2005. Automated detection of fecal
contamination of apples based on multispectral uorescence image fusion.
Journal of Food Engineering 71 (1), 8591.
Kim, D.G., Burks, T.F., Qin, J.W., Bulanonm, D.M., 2009. Classication of grapefruit
peel diseases using color texture feature analysis. International Journal of
Agricultural and Biological Engineering 2 (3), 4150.
Lawrence, K.C., Windham, W.R., Park, B., Heitschmidt, G.W., Smith, D.P., Feldner, P.,
2006. Partial least squares regression of hyperspectral images for contaminant
detection on poultry carcasses. Journal of Near Infrared Spectroscopy 14 (4),
223230.
Lawrence, K.C., Yoon, S.C., Heitschmidt, G.W., Jones, D.R., Park, B., 2008.
Imaging system with modied-pressure chamber for crack detection in
shell eggs. Sensing and Instrumentation for Food Quality and Safety 2 (3),
116122.
Leemans, V., Magein, H., Destain, M.F., 1998. Defects segmentation on Golden
Delicious apples by using colour machine vision. Computers and Electronics in
Agriculture 20 (2), 117130.
Leemans, V., Destain, M.F., 2004. A real-time grading method of apples based
on features extracted from defects. Journal of Food Engineering 61 (1),
8389.
Li, J.B., Rao, X.Q., Guo, J.X., Ying, Y.B., 2010. Hyperspectral reectance imaging for
detecting citrus canker based on dual-band ratio image classication method.
Proceedings of SPIE 7656, 110.
Liu, Y.L., Chen, Y.R., Kim, M.S., Chan, D.E., Lefcourt, A.M., 2007. Development of
simple algorithms for the detection of fecal contaminants on apples from
visible/near infrared hyperspectral reectance imaging. Journal of Food
Engineering 81 (2), 412418.
Lpez-Garca, F., Andreu-Garca, G., Blasco, J., Aleixos, N., Valiente, J., 2010.
Automatic detection of skin defects in citrus fruits using a multivariate
image analysis approach. Computers and Electronics in Agriculture 71 (2),
189197.
Malinowski, E.R., Howery, D.G., 1980. Factor Analysis in Chemistry. John Wiley &
Sons, New York.
Mehl, P.M., Chao, K., Kim, M., Chen, Y.R., 2002. Detection of defects on selected apple
cultivars using hyperspectral and multispectral image analysis. Applied
Engineering in Agriculture 18 (2), 219226.
Mendoza, F., Aguilera, J.M., 2004. Application of image analysis for classication of
ripening bananas. Journal of Food Science 69 (9), E471E477.
Miller, B.K., Delwiche, M.J., 1991. Peach defect detection with machine vision.
Transactions of the ASAE 34 (6), 25882597.
Polder, G., Gerie, W.A.M., Van, D.H., 2003. Calibration and characterization of
imaging spectrographs. Near Infrared Spectroscopy 11 (3), 193210.
Qiao, J., Ngadi, M., Wang, N., Gariepy, C., Prasher, S., 2007. Pork quality and marbling
level assessment using a hyperspectral imaging system. Journal of Food
Engineering 83 (1), 1016.
Qin, J., Lu, R., 2005. Detection of pits in tart cherries by hyperspectral transmission
imaging. Transactions of the ASAE 48 (5), 19631970.
Qin, J.W., Burks, T.F., Kim, M.S., Chao, K.L., Ritenour, M.A., 2008. Detecting citrus
canker by hyperspectral reectance imaging and PCA-based image
classication method. Proceedings of SPIE 6983, 111.
Qin, J.W., Burks, T.F., Ritenour, M.A., Bonn, W.G., 2009. Detection of citrus canker
using hyperspectral reectance imaging with spectral information divergence.
Journal of Food Engineering 93 (2), 183191.
Recce, M., Plebe, A., Taylor, J., Tropiano, G., 1998. Video grading of oranges in real
time. Articial Intelligence Review 12 (1), 117136.
Schubert, T.S., Rizvi, S.A., Sun, X., Gottwald, T.R., Graham, J.H., Dixon, W.N., 2001.
Meeting the challenge of eradicating citrus canker in Florida-Again. Plant
Disease 85 (4), 340356.
Skoglund, A., Kessler, W., Kessler, R.W., 2004. On-line spectroscopic measurements
of wood chips before a continuous digester. Chemometrics and Intelligent
Laboratory Systems 70 (2), 129135.
Vargas, A.M., Kim, M.S., Tao, Y., Lefcourt, A.M., Chen, Y.R., Luo, Y., Song, Y., Buchanan,
R., 2005. Defection of fecal contamination on cantaloupes using hyperspectral
uorescence imagery. Journal of Food Science 70 (8), 471476.
Wallays, C., Missotten, B., Baerdemaeker, J.D., Saeys, W., 2009. Hyperspectral
waveband selection for on-line measurement of grain cleanness. Biosystems
Engineering 104 (1), 17.
Xing, J., Bravo, C., Jancsk, P., Ramon, H., De Baerdemaeker, J., 2005. Detecting
bruises on Golden Delicious apples using hyperspectral imaging with multiple
wavebands. Biosystems Engineering 90 (1), 2736.