High speed potato grading and quality inspection based on a color vision system

J.C. Noordam* , G.W. Otten, A.J.M. Timmermans, B.H. van Zwol Department Production & Control Systems, ATO, P.O. Box 17, 6700 AA, Wageningen, the Netherlands
ABSTRACT
A high-speed machine vision system for the quality inspection and grading of potatoes has been developed. The vision system grades potatoes on size, shape and external defects such as greening, mechanical damages, rhizoctonia, silver scab, common scab, cracks and growth cracks. A 3-CCD line-scan camera inspects the potatoes in flight as they pass under the camera. The use of mirrors to obtain a 360-degree view of the potato and the lack of product holders guarantee a full view of the potato. To achieve the required capacity of 12 tons/hour, 11 SHARC Digital Signal Processors perform the image processing and classification tasks. The total capacity of the system is about 50 potatoes/sec. The color segmentation procedure uses Linear Discriminant Analysis (LDA) in combination with a Mahalanobis distance classifier to classify the pixels. The procedure for the detection of misshapen potatoes uses a Fourier based shape classification technique. Features such as area, eccentricity and central moments are used to discriminate between similar colored defects. Experiments with red and yellow skin-colored potatoes have shown that the system is robust and consistent in its classification. Keywords: image processing, computer vision, color segmentation, potato grading, quality inspection

1. INTRODUCTION
Grading and sorting potatoes ensures that the products meet defined grade and quality requirements for sellers and provides expected quality for buyers. Usually, quality sorting is performed by trained human inspectors who assess the potato by seeing the potato for a particular quality attribute. However, there are some disadvantages to apply human inspectors such as inconsistency, extensive time to inspect huge volumes and expensive labor costs. Computer vision may improve inspection results and take over the visually intensive inspection work from the human inspector. Various studies related to machine vision inspection of potatoes have been reported in literature. An automated inspection station for machine vision grading of potatoes on size and shape has been reported1,2,3,4 . The color segmentation results of a multilayer feed forward Neural Network (MLF-NN) and a traditional classifier for the color inspection of potatoes have been compared5 . The throughput of the system as reported by Heinemann was three potatoes/min. and the classification results decreased significantly when the potatoes were moving1 . All systems described above can not fulfill the potato industry requirements for high throughput and real-time speed. Besides a low throughput, none of the described systems is capable to inspect for size, shape, and multiple color defects. To overcome this low throughput, a PC-based high-speed machine vision system for potato inspection with a throughput of 50 images/sec has been reported6 . The system is capable to classify potatoes for size, weight, cross-sectional diameter, shape, and color. The weakness of the system is that the color classification procedure discriminates between good potatoes and green potatoes only. Detection of multiple color defects is not possible with the system. Besides greening, other defects such as cracks, common scab and rhizoctonia are also important features which influence the consumer preferred quality. For a machine vision system to become successful in the potato packaging industry, defects as described above must be detected. The objective for this work was to develop a computer vision system to inspect and grade potatoes based on multiple external color defects, size and shape. The HIghspeed QUality Inspection of Potatoes (HIQUIP) system incorporates conveyor lanes to transport the potatoes to and from the vision unit. It is assumed dust and dirt are removed before inspection by washing. After inspection and grading, the potatoes are transported to a packaging device where potatoes are packed in little bags and sold on the consumer market. As the machine must operate in a potato packaging plant, some extra demands are being imposed to the concept. Apart from recognizing external defects and detecting misshapen potatoes, it must also have a high accuracy and a capacity of 12 tons/hour. The paper is organized as follows. In section 2 the mechanical part of the HIQUIP system is described, followed by a description about the characterization of potato defects. In section 3, the algorithms for the color segmentation, shape

Correspondence : Email : J.C.Noordam@ato.wag-ur.nl; WWW: http://www.ato.wageningen-ur.nl; Fax : +31.317.475347

classification and crack detection are described. The experiments are discussed in section 4. Finally, in section 5, the conclusions and further research are discussed.

2. THE HIQUIP SYSTEM

2.1. Machine vision system The complete potato inspection system consists of a conveyor unit, a vision unit and a rejection unit, all placed in a single line. The conveyor unit consists of two singulating conveyors (SC1 and SC2) to separate the potatoes and to create a single line of potatoes. The speed of conveyor SC2 is slightly higher than the speed of conveyor SC1 to separate the potatoes at the transaction from SC1 to SC2. Conveyor SC2 transports the potatoes towards the vision unit where the inspection takes place. Two conveyor belts (VC1 and VC2) of the vision unit, placed one after another, transport the potato under the camera for inspection. A digital 3-CCD color line-scan camera scans the narrow gap between the conveyors VC1 and VC2 to achieve in-flight inspection of the potato. To obtain a 360 degree view of the potato, mirrors are placed in the small gap (4 cm) between the conveyors VC1 and VC2. The lack of product holders and the use of mirrors guarantee a full view of the potato. Figure 1 shows the two conveyors VC1 and VC2 with the mirrors placed in the gap as the potatoes fly from the upper conveyor VC1 to the lower conveyor VC2 (left). The right image of figure 1 shows a top view as the potato passes the gap between the conveyors VC1 and VC2 . The surplus value of the mirrors is immediately shown in the right image, as the crack in the bottom of the potato is still visible in the mirror image.
TL-tubes mirrors conveyor VC1 conveyor VC2 crack in mirror image

Figure 1. Two conveyors with mirrors placed in the gap (angle view, left) and camera view( right)

Three conveyor units, each consisting of VC1 and VC2 with the accompanying lighting and mirrors, are inspected all by one camera to achieve the required capacity of 12 tons/hour. A single conveyor unit must have a capacity of 4000 kg / 3600 sec. = 1.1 kg/sec. For table potatoes, this implies that about 10 potatoes/sec must be processed. With an average length of 10 cm per potato and a spacing of 5 cm between two potatoes, a belt speed of 1.5 m/sec is required. To detect objects with a size of 1 mm, a resolution of 2 pixels/mm is required. To obtain this resolution of 2 pixels/mm, the camera must grab 2000 lines/sec. To achieve a similar resolution in the direction perpendicular to the movement direction, a camera with 2098 pixels is sufficient to cover the width of 1.1 meter for the three conveyor units. A grab frequency of 2000 lines/sec. requires powerful lighting equipment. Therefore, folded small-sized high-frequency TL tubes with parabolic reflectors are used to illuminate the potatoes. Four TL tubes illuminate the bottom part of the potato and six tubes illuminate the upper part of the potato. The illumination amplitude of each individual TL tubes is controllable to obtain a homogenous illumination on the potato area. The camera grabs continuously and the software detects when a potato passes the gap between the conveyors VC1 and VC2. Therefore, the camera requires no additional starting signal when a potato approaches the imaging area. After inspection, the potato is transported to the rejection unit. The rejection unit consists of individually controlled product holders. Each product holder is kept upwards by electro magnets. Once a potato arrives at the correct rejection lane, the magnets are released and the potato drops at the correct rejection station.

A high grab frequency requires dedicated hardware for the image processing and classification tasks. A Spectrum Signal PCI-card with 11 SHARCs (Analog Devices ADSP-21060) Digital Signal Processors (DSP) is responsible for the image acquisition and classification tasks. One DSP communicates with the Host PC and transports the measurement results to the screen for visualization. Three DSPs perform the image acquisition, combine the color planes to a single pixel and perform the color correction. Four DSPs perform the color segmentation, image compression and spurious pixel removal. The remaining three DSP are divided over the three conveyor lanes to perform the operations for color and shape classification. A MS-Windows based graphical user interface (GUI) contains various parameter settings and visualization tools for the operator to: select different color and shape models for different potato cultivars adjust the margins for the product grading classes learn the HIQUIP system new color defects or new cultivars log or view the history and segmented images of earlier classified potatoes The number of product classes is theoretically unlimited because the software is extendable to grade the potatoes in multiple classes. With the modular design of the rejection unit, an unlimited number of rejection stations is possible. In practice however, the number of rejection stations is limited to five or six (two size classes and four quality classes) due to increasing costs, as each station requires its own conveyor and storage capacity. 2.2. Characterization of potato defects Product experts characterize potato defects and diseases on color and shape. Factors such as size, shape, greening, cracks, scab etc. determine the final grade of a potato. The potatoes are graded into four different categories, dependent on the presence of a defect and the area of the defect (as shown in table 1). Kind of defect Qual i t y c l a sses Good potato Minor defect Medium defect Major defect

Outward roughness (scab, Scale 1 Scale 1.5 Scale 2.5 scale 3.5 skin spot, black scurf) ( 1 - 2 spots, < 3.125% ) ( 2 - 4 spots, < 6.25%) (5 - 10 spots, < 12.5%) (20-40 spots, < 25% ) Tuber greening Pressure spots Damaged potatoes Tuber cracks Misshapen potatoes 0 - 1% not present < 5 mm2 Small cracks not misshapen 0 - 2% 0 - 2% 0-3% ( 5-10 mm )
2

2 - 5% 2 - 5% 3-8% (10-15 mm )
2

> 5% > 5% > 8% (15-20 mm2 )

> 0.25cm deep, > 1/5 length tuber not allowed misshapen

Table 1. Grading classes, in percentages of total potato area

Similar diseases on potatoes of different cultivar (scab, skin spot and black scurf ) may have a distinct color due to the underlying skin color of the potato. This requires a different reference set of images for each potato cultivar. Besides the difference in skin color for different cultivars, differences in skin structure and shape are also important features.

Figure 2. Four different defects: Misshapen, damaged, rhizoctonia and cracks

For the characterization of defects and diseases, product experts examined 6 potato cultivars. From each cultivar an image collection of all possible defects was created. Each potato image is accompanied with a sensorial description and stored in a database. This reference database contains more than 1100 images of 6 cultivars with the most occurring defects and diseases. The database is used for the development and testing of the color and shape algorithms. Four defects on two different potato cultivars are shown in figure 2. Images of potatoes are made using the camera set-up as shown in figure 3. It consists of a lighting chamber developed by ATO. High frequency TL-lamps (color 84, 20 Watt) are mounted in such a way that a uniform light intensity is obtained at the potato tuber. Furthermore, constant light intensity is realized by measuring the light intensity and feeding this back to the power unit of the TL-lamps.

Color camera Computer Lighting chamber

Potato tuber Mirror Mirror

Figure 3. Schematic view of the camera set up

Color images of the potatoes are taken with a Sony 3-CCD color camera. Mirrors make it possible to inspect the potato from all sides.

3. ALGORITHM DESIGN
3.1. Color segmentation The majority of external defects and diseases are identified by its color, which makes the classification of pixels into homogeneous regions an important part of the algorithm. Multi-layer feed-forward Neural Networks (MLF-NN) and statistical discriminant functions have been used successfully for the segmentation of potato images 4,5,7 . In literature 5 , the results of potato image segmentation of MLF-NN and Linear Discriminant Analysis (LDA) were compared for a limited number of defects; only the color defects bruises and greening were considered. A MLF-NN was reported to slightly outperform a statistical discriminant function for the classification. However, MLF-NN are known to be sensitive for parameter settings (number of hidden nodes, learning rate etc.). Moreover, they require substantial training time. Although this makes LDA preferable above MLF-NN techniques, both MLF-NN and LDA techniques are compared for the segmentation of potato color defects in section 4. Six different color classes are identified: background, potato skin, greening, silver scab, outward roughness and rhizoctonia. All classes are divided into a number of sub classes, e.g. different background colors belong to the background class and the class good skin exists of dark skin and light skin. In total 6 different main classes are used, as shown in table 2. Main class Background Skin Greening Rizotonia Silver Scab Outward roughness Sub class White background, dark background, mirror edge Skin light, skin dark Green light, green dark Light gray, dark gray, black Silver Scab Brown light, brown dark

Table 2. Classes and subclasses for the image segmentation

Due to the difference in skin color it is not sufficient to use a single model for different potato cultivars. For each potato cultivar a new color model is created. As MLF-NN and LDA are supervised classification routines, a labeled training set is required to extract the color model for a potato cultivar. A labeled test set is used to test the performance of the classifier. The images in the database are used to extract the data for the training set and test set. The building of a training-set for the color segmentation of multiple potatoes is an elaborate task. All different colors have to be shown to the HIQUIP system and be trained manually. Instead of single pixel selection, Principal Component Analysis (PCA) selection has been used for the extraction of a training and test set. PCA is a data compression technique and produces a linear combination of the variables (red,green,blue) which form improved descriptors for structure or patterns in the data. Pixels with similar RGB values tend to group in the PCA space (clusters). The pixels in these clusters contain similar information and correspond to similar regions in the image. The pixels of a single cluster can be selected, labeled and stored to disk. With this procedure, the time consuming selection of pixels for training sets and test sets is overcome. In figure 4, a bildstar potato is shown with three different classes: skin, silver scab and rhizoctonia (black spots). The region surrounded by the white rectangle is used as an example for a PCA based pixel selection, as shown in figure 5a&b.

Figure 4. Bildstar potato with three classes: silver scab, rhizoctonia and good skin.

Figure 5a shows the corresponding score plots after a PCA on the part of the image surrounded by the white square. The score plots show three distinct clusters which correspond to the classes skin, silver scab and rhizoctonia. The selected clusters in the score plot are mapped back in the images (figure 5b) to show that the selected pixels correspond to similar regions in the image.

Figure 5a. Score plot with clusters of good skin (left), scab (middle) and rhizoctonia (right).

Figure 5b. Selected clusters of figure 5a mapped back into the original images: good skin (left), scab (middle) and rhizoctonia (right).

These selected pixels are extracted from the image, labeled and saved to disk. The procedure is repeated for all color defects. To create a training-set and test-set for a cultivar, labeled clusters of multiple images of the same cultivar are merged and stored. 3.2. Discrimination between similar colored objects There are a number of defects and diseases which have similar color. Defects such as cracks and rhizoctonia both have a black color (figure 2). Discrimination between these defects is important since cracks are a more serious defect. Rhizoctonia and cracks both appear as unappetizing defects, but cracks may become rotten and infect other potatoes. Therefore, cracks must be removed from the batch. Furthermore, growth cracks and common scab have similar color. Usually, a growth crack is no more than a deformed part of a potato and often has a similar color as the potato skin. However, due to the deformation, the skin color becomes darker and looks like common scab. Because a growth crack should be removed from the batch, discrimination between a growth crack and common scab is also required. For the discrimination between cracks and rhizoctonia and between growth cracks and common scab additional shape features are used. Shape features must be able to discriminate between the different defects, as cracks and growth cracks appear as more or less elongated in comparison with rhizoctonia spots and common scab. Eccentricity can be considered as a measure of length/width and is used to discriminate between cracks and rhizoctonia. It can vary from 1 to . A circular object gives an eccentricity of 1, a line shaped object has a higher value. Eccentricity is based on central moments of an object8,9 : the moments of order p+q of an object represented by the bitmap image b n,m with size n * m are :

m pq = x p y qbn, m
x =0 y =0

(1)

where b n,m is 1 for a foreground pixel and 0 for a background pixel, x and y are the position in the image. The zero order moment of an object is the area of the object. The first order moments M 01 and M10 are related to the center of mass of the object: (2)

x = M 1, 0 / M 0, 0

y = M 0,1 / M 0,0

In order to make the description independent for position, moments can be calculated with respect to the center of mass of the object. These moments are called central moments:

u pq = ( x x) p ( y y ) q bn, m
x =0 y =0

(3)

From the second order central moments a number of properties can be calculated that are comparable with the moments of inertia associated with rotating bodies in mechanics. The principal axis of a region are spanned by the eigenvectors of the matrix :

2 , 0 1,1

1,1 0, 2
2 2,0 2 0 ,2 2 2 ,0 02 + 4 12,1

(4)

and the corresponding eigenvalues of this matrix are called the principal moments :

max = 1 ( 2, 0 0 , 2 ) + 2

1 2

min = 1 ( 2, 0 0, 2 ) 1 2 2,0 2 0, 2 2 2 ,0 02 + 4 12,1 2 2

The eccentricity of a region can be defined as the ratio between the square roots of the two principal moments:

(5)

eccentrici ty =

max min

(6)

To limit processing time, only objects of the classes rhizoctonia and outward roughness are considered for the calculation of moments. Furthermore, only objects with an area greater than a threshold value are considered as there are many spurious small objects which are not cracks.

In the learning phase, from all available crack and growth crack examples in the database the eccentricities are calculated to determine a proper threshold. If the area of a object is above the area-threshold and the eccentricity of the object is above the eccentricity-threshold, the object is classified as a crack: (7) if ( area > threshold ) & &(eccentrici ty > Threshold ) object is a crack
object area object eccentricity

The procedure as describe above for the detection is sufficient if the objects of the classes rhizoctonia and outward roughness are not located at the edge of a potato image. A small object at the edge of a potato image may be misclassified as crack due the large view angle of the camera and the roundness of the potato. In this particular situation, the object appears as an elongated object in the image and is consequently misclassified as crack. To prevent these misclassifications, cracks detected at the border of an image are ignored if the principal axis of the objects runs parallel with the edge of the potato. This procedure does not effect actual cracks at the border of the potato because actual cracks at the edges are visible in either one of the remaining images. The similar procedure is followed for the detection of growth cracks in the class common scab, only with different values for area threshold and eccentricity threshold. An example is given in figure 6, where a bildstar potato is shown. The bildstar potato contains silver scab, a few small rhizoctonia spots and a crack. In the segmented image, multiple dark colored objects have been found. Only one of the objects is recognized as a crack by the crack detection procedure, indicated by a black surrounding rectangle.

Figure 6. Bildstar image with crack, segmented bildstar image with detected crack

3.3. Shape classification For the detection of misshapen potatoes and apples, various techniques have been presented1,7,10. A statistical discriminant function and 15 Fourier Descriptors (FDs) outperformed central moments for the shape classification of apples 10 . FDs can be made invariant to translation, rotation and scale, which is important in online sorting. A Fourier Descriptor based technique was developed to grade potatoes on shape7 . The FDs were combined to a single shape factor to discriminate between various shaped potatoes. A threshold value for the shape factor determines the final class of a potato and an accuracy of 89.2% for the shape separation was reported7 . In the HIQUIP system, FDs and LDA are used to discriminate between good and misshapen potatoes. From each segmented potato image, the boundary is extracted. The one-dimensional boundary is normalized to 256 equidistant points for equal level of comparison. For each boundary point, the distance to the centroid is calculated. This boundary signature is translated to the Fourier domain and the resulting FDs are the input variables for the LDA. It was reported7 that the first 10 harmonics were adequate for representing the shape information of a potato. In the shape classification experiments of section 4.2, different number of FDs were used to evaluate the influence of the number of FDs. A single shape model is not sufficient to segment all potato cultivars into the classed good and misshapen. Good shaped potatoes may vary from round, oval, to extreme oval. Therefore, different shape models are created for different potato cultivars. A shape training set and shape test set is created for each cultivar to discriminate between good potatoes and misshapen potatoes.

4. RESULTS AND DISCUSSIONS

In experiments 1-3, images of different potato cultivars from the image database were used to test the individual performance of the algorithms. In experiments 4-6, potatoes of the cultivar bintje and sant were used for the performance evaluation of the HIQUIP system. Unfortunately, only few potatoes with a limited number of defects were available for

testing. It is known that the number of potatoes used for testing is not significant to draw conclusions about the performance of the HIQUIP system. However, they do give an indication of the performance. In experiment 1, the performance of the color classification routine was evaluated where the segmentation results of MLFNN are compared with the results of LDA for five different potato cultivars. In experiment 2, the performance of the shape classification procedure is evaluated. The shape experiments were repeated for a different number of FDs to evaluate the influence of the amount of FDs. In experiment 3, the crack and growth crack detection procedure is evaluated. In experiment 4, the performance of the HIQUIP system was evaluated for two different defects; potatoes with the rhizoctonia defect and potatoes with the common scab defect were considered. In experiment 5, potatoes with cracks were inspected to evaluate the crack detection performance of the HIQUIP system. The results were compared with the classification results of a product expert, who classified the potatoes beforehand. In the last experiment, the robustness of the HIQUIP system is determined. 4.1. Results of LDA and MLF-NN color classification For six different potato cultivars, a training and test set was created as described in section 2. From the complete set of labeled pixels for each class, 500 pixels were randomly selected to create training and test sets. This resulted in 6*500 pixels for each labeled set. Multiple potato images from a single cultivar were used to create the training and test sets. For the MLF-NN, the input RGB values were auto-scaled. The MLF-NN consists of three input nodes, one hidden layer and an output layer. The output layer consists of 6 neurons, one for each color class. Variations were made in the number nodes in the hidden layer and the transfer function (sigmoid, tanh). The training sets were used to create the color model for the LDA and to train the MLF-NN, the test sets were used to evaluate the performance of both classifiers. The experiments were repeated 5 times. The average percentages of good classification for six classes are shown in table 3. Cultivar Bildstar Bintje Eigenheimer Irene Sante LDA (%) 95,1 96,2 86,8 89,3 98,6 MLF-NN (%) 99.2 96,9 88.1 91.6 98.7

Table 3. Comparison of MLF-NN and LDA color segmentation results (reference images)

For all cultivars, a MLF-NN with 12 nodes in the hidden layer and the tanh transfer function gave the best results. Dark colored defects and diseases may overlap in RGB space and are therefore hard to discriminate by LDA. For this reason the results of MLF-NN are slightly better than the results of LDA with the mahalanobis distance function. Although the overall performance of Neural Networks is slightly better than those of LDA with a mahalanobis distance classifier, the latter is implemented as segmentation technique in the HIQUIP system as LDA requires no parameter adjustment. 4.2. Results of shape classification In this experiment, the performance of the shape classification is considered. A set of 40 misshapen potato images from the image database is split in half to create a training set and test set for the misshapen potato class. A set of 130 potato images from the image database is split in half to create a training set and test set for the good potato class. LDA with a mahalanobis distance classifier classifies the potatoes into the classes good and misshapen potatoes. The experiment is carried out for 10, 20 and 30 FDs to evaluate the influence of the number of FDs. The results of the shape classifications are shown in table 4. Using 30 FDs for the boundary description of the potatoes gives the best results, all potatoes were classified correctly. This is to be expected, extra boundary information is added by using more FDs. As the number of FDs decreases, misshapen potatoes are classified as good potatoes. 30 FDs will be used in the shape classification procedure in the HIQUIP system. The results in table 4 may indicate that even with 10 FDs the results are acceptable. However, the misshapen potatoes in the reference database are truly misshapen; less misshapen potatoes may be more difficult to discriminate from good potatoes and may require extra FDs. Experiments with real potatoes on the HIQUIP system should indicate whether 10 FDs are sufficient. 30 FDs Good (labeled) Misshapen (labeled) Good potato (result) 65 0 Misshapen potato (result) 0 20

20 FDs Good (labeled) Misshapen (labeled) 10 FDs Good (labeled) Misshapen (labeled)

Good potato (result) 65 1 Good potato (result) 65 2

Misshapen potato (result) 0 19 Misshapen potato (result) 0 18

Table 4. Results of the shape classification for three different number of Fourier Descriptors

4.3. Results of the crack and growth crack detection experiment 4.3.1. Results of the crack detection experiment To verify the results of the crack detection procedure, 20 potato images with cracks and 75 potato images without cracks are selected from the image database. The 75 potato images without cracks are covered with similar colored defects of the classes rhizoctonia and outward roughness. After the color segmentation, eccentricity and area are calculated for all objects of the classes rhizoctonia and outward roughness. The results of the measurements are shown in table 5 and figure 7. Defect : Cracks Good potato (labeled) Potato with crack (labeled) Good potato (result) 75 0 Potato with crack (result) 0 20

Table 5. Results of the crack detection experiment

20 18 16 14 Eccentricity 12 10 8 6 4 2 0 1 10 19 28 37 46 55 64 73 82 Number of measurements 91 small object crack misclassified as crack threshold

Figure 7. Results of the crack detection experiment

The graph shows the calculated eccentricities for 95 potatoes. Objects with an area below the area threshold of 22 pixels are ignored by the crack detection procedure. Objects with eccentricities above the eccentricity threshold level of 2.9 are considered as cracks. Four measurements indicated by a triangle () are initially misclassified as crack. A closer look to the misclassified potato images shows that there is no crack on the potato, the object is a small object at the edge of the potato image and misclassified due the large view angle of the camera. As these objects are very close to the border, they can easily be recognized and ignored. As a result, all potatoes with a crack are classified correctly.

4.3.2. Results of the growth crack detection experiment To verify the results of the growth crack procedure, 64 potato images of the bintje cultivar are selected from the potato image database, from which 9 potato images were labeled as growth crack. The remaining 55 potato images are covered with the similar colored defect common scab. After the color segmentation, eccentricity and area are calculated for all objects of the class common scab. The results are shown in figure 8 and table 6.
12 10 small objects Eccentricity 8 growth crack 6 4 2 0 13 19 25 31 37 43 49 55 Number of measurements 61 1 7 misclassified as growth crack threshold

Figure 8. Results of the growth crack detection experiment

The graph shows the calculated eccentricities of 64 potatoes. The threshold level for eccentricity is set to 3. Objects with eccentricities above this level are considered as growth cracks. Two growth cracks were missed (below the horizontal threshold line), 4 potatoes are misclassified as growth crack if object area is not considered during classification. If the area threshold is set to 227 pixels, the four misclassified potatoes are correctly classified as good potatoes because the areas of the misclassified objects are below the area threshold. The results are shown in table 6. Defect : Growth cracks Good potato (labeled) Potato with growth crack (labeled) Good potato (result) 55 2 Potato with growth crack (result) 0 7

Table 6. Results of the growth crack detection experiment

4.4. HIQUIP system inspection results of potato defects 4.4.1. Inspection results of potatoes with the rhizoctonia defect In this experiment, the performance of the HIQUIP system is tested for the defect rhizoctonia. Potatoes of the cultivar sante were used. Two quality classes are considered, good potatoes and minor defects (table 1). A product expert inspected and classified each potato beforehand. The good potato class consists of 32 potatoes, the minor defect class consists of 27 potatoes. The results of the HIQUIP system are shown in table 7. All potatoes were classified correctly. Defect : Rhizoctonia Good potato (labeled) Minor defect (labeled) Good potato (result) 32 0 Minor defect (result) 0 27

Table 7. Results of the inspection by the HIQUIP system for the rhizoctonia defect

Although these results are promising, a few remarks are indispensable. The dark-gray colored defect rhizoctonia on a yellow skin potato results in a high contrast. Therefore, image segmentation is not as difficult as when the color defects are more similar. 4.4.2 Inspection results of potatoes with common scab defect In this experiment, potatoes of the cultivar bintje with common scab are inspected by the HIQUIP system. Two quality classes are considered, medium and major defects (table 1). A product expert inspected and classified each potato beforehand. The medium class consists of 23 potatoes; the major defect class consists of 27 potatoes. The results of the HIQUIP system are shown in table 8. Defect : common scab Medium defect (labeled) Major defect (labeled) Medium defect (result) 16 8 Major defect (result) 7 19

Table 8. Results of the inspection by the HIQUIP system for the common scab defect

The results indicate that common scab is difficult to discriminate from skin color, as almost 30% of each defect class is misclassified. A closer look to the segmented images showed that the misclassification mainly occurred in the two mirror images. The dark potato skin class in a mirror image was classified as common scab. This implies that the illumination intensity in the mirror images is less than the illumination intensity of the middle image. The common scab and dark skin pixel clusters in RGB space overlap due to this illumination difference. Therefore, pixel classification results of defects with more contrast are not affected by the illumination difference. To increase the common scab classification performance in the future, this illumination intensity difference must be corrected. 4.5. HIQUIP system inspection results of crack detection To verify the results of the crack detection procedure, potatoes with cracks of the bintje cultivar were inspected by the HIQUIP system. The product expert classified 15 potatoes as cracks beforehand. The results are shown in table 9. Two potatoes were missed, 13 potatoes with cracks were correctly classified. The cracks on the misclassified potatoes were small and could not be detected. Defect : Cracks Potato with crack (labeled) Good potato (result) 2 Potato with crack (result) 13

Table 9. Results of the crack detection by the HIQUIP system

4.6. Robustness of the HIQUIP system This experiment is carried out to evaluate the robustness of the HIQUIP system. Robustness is considered as how consistent a system measures the color defect areas. To test the robustness of the HIQUIP system, a potato with known defects is considered. The potato contains the defect rhizoctonia and the area of the defect is measured beforehand. The potato was inspected by the HIQUIP system. The experiment was repeated 20 times for different potato positions on the conveyor. The results are shown in the graph of figure 9. The average of the area measurements is 4.04 with a variance of 0.314.
6 5 4 3 2 1 0 1 3

Robustness test

area (%)

11

13

15

17

Number of measurements

Figure 9. Robustness test of the HIQUIP system. A potato with the rhizoctonia defect is measured 20 times.

19

5. CONCLUSION AND FUTURE WORK

A high speed color vision system for the inspection and grading of potatoes has been developed. The HIQUIP system grades potatoes on shape, size, cracks, growth cracks, and color defects such as greening, common scab, silver scab and rhizoctonia. A 3-CCD color line-scan camera inspects three independently driven conveyor lanes, moving at 1.5 m/sec, to achieve the required capacity of 12 tons/hour. Mirrors placed in the narrow gap between the two conveyors guarantee a full 360-degree view of the potato. Multiple folded small-sized high frequency TL tubes with parabolic reflectors are sufficient to get satisfactorily lighting. The PC-based PCI board with 11 Digital Signal Processors is fast enough to perform the image processing and classification tasks with a speed of 50 potatoes/sec. LDA and a mahalanobis distance classifier classify RGB pixels in 6 different color classes. Pixel classification experiments with 6 color classes show classification rates above 90% for 5 potato cultivars. Features such as area, eccentricity and central moments make it possible to discriminate between similar colored defects. With these features, cracks and growth cracks on potatoes can also be detected by the HIQUIP system with a high success rate. The Fourier-based shape classification procedure detects misshapen potatoes. The results of the experiments indicate that the HIQUIP system performs well on rhizoctonia and crack detection. The addition of a software based correction module to reduce the illumination differences in the mirror images will be a topic for further work. Although more test are necessary to evaluate the performance for other defects, the capacity of 12 tons/hour and the reported classification results indicate that the HIQUIP system can fulfill the demands of the potato packing industry.

6. ACKNOWLEDGEMENTS
Partial financial support from the Commission of European Communities (FAIR CT96 9505) is gratefully acknowledged. Thanks to Floor Verdenius for his comments and contribution to the paper.

7. REFERENCES
[1] [2] [3] [4] [5] P. H. Heinemann, N. P. Pathare, and C. T. Morrow, An automated inspection station for machine-vision grading of potatoes, Machine vision and Application, vol. 9, pp. 14-19, 1996. N. P. Pathare, P. H. Heinemann, C. T. Morrow, and S.Deck, Automated inspection station for grading of potatoes, presented at 1993 international summer meeting of the ASAE, Spokane, Washington, 1993. Y. Tao, C. T. Morrow, P. H. Heinemann, and H. J. Sommer, Automated machine vision inspection of potatoes, presented at 1990 international winter meeting of the ASAE, Hyatt regency Chicago, Illinois, 1990. Y. Tao, P. H. Heinemann, Z. Varghese, C. T. Morrow, and H. J. Sommer, Machine vision for color inspection of potatoes and apples, Transactions of the ASAE, vol. 38, pp. 1551-1561, 1995. S. H. Deck, C. T. Morrow, P. H. Heinemann, and H. J. Sommer, Comparison of a neural network and traditional classifier for machine vision inspection of potatoes, Applied engineering in agriculture, vol. 11, pp. 319-326, 1995. L. Zhou, V. Chalana, and Y. Kim, PC-based machine vision system for real-time computer-aided potato inspection, International journal of imaging systems and technology, vol. 9, pp. 423-433, 1998. Y. Tao, C. T. Morrow, P. H. Heinemann, and H. J. S. III, Fourier-based separation technique for shape grading of potatoes using machine vision, Transactions of the ASAE, vol. 38, pp. 949-957, 1995. M. Dai, P. Baylou, and M. Najim, An efficient algorithm for computation of shape moments from run-length codes or chain codes, Pattern Recognition, vol. 25, pp. 1119-1128, 1992. M.-K. Hu, Visual pattern recognition by moments invariants, IRE transactions on information theory, pp. 179187, 1962. V. Leemans, H.Magein, and M. F. Destain, Apple shape inspection with computer vision, presented at Sensors for non-destructive testing, Orlando, Florida, 1997.

[6] [7] [8] [9] [10]