Beruflich Dokumente
Kultur Dokumente
Abstract-Fruits and vegetables market is subject of choice. Fruit quality is defined by various factors such as shape,
Thus, it is important for the suppliers to label the quality of goods size, color and defect-free. There are various stages of ripeness
before dealing out. Some commercially available fruit sorting and
of fruit. Ripening of fruit proves to be a compound occurrence
grading system have been introduced, but they are almost
that affects fruits chemically as well as physiologically. Color
expensive for small and medium fruit processing industry.
of fruit describes the ripening stage of fruit. Therefore,
Presently, human experts grade the agriculture goods based on its
costumer's choice depends upon the color fruit which
vision based features that cause the inaccuracy, inconsistency and
inefficiency on defining the quality of agriculture goods. In this describes the ripening stage of fruit. Tomato plays a vital role
paper, we are inspecting the quality of tomato based on shape, in a meal of common man. Tomato is a very delicate fruit
size and degree of ripeness. An edge detection algorithm is used to which is used every day. Thus, it is need of time to establish a
estimate the shape and size of tomato and color detecting non-destructive automated technology to determine the quality
algorithm is used for the ripeness determination. All these of tomato and grade them accordingly without any damage.
algorithms are implemented on Raspberry Pi development board
which will become independent and cost effective system. Our Poor efficiency, high cost, slow speed of grading and
system includes Raspberry Pi development board, conveyor belt, complexity are drawbacks of the present fruit quality detecting
motors, Pi camera. All the interfacing of the above devices will be and grading system. Human experts sort the fruit on the basis
carried out and will make a cost effective embedded system for of color, shape and size and defects. But, human experts get
the determination of shape, size and degree of ripeness of tomato.
tired or bore or may be less attentive after a period of time
Same system can be utilized for other fruits and vegetables also.
which causes the inaccurate, inconsistent and ineffectual
Keywords-Image Processing, Ripeness, Grading, Sorting,
outputs. Therefore, it is the requirement of industry to establish
Conveyor System
high speed and low cost automated quality detection and
I. INTRODUCTION grading system. Then, the work will be faster and free from
error.
Agriculture plays a significance role and is the integral
part of overall economic development in India. Agriculture is Fuzzy Logic, Genetic Algorithm, Neural Network, Multi
the only way of living for the two-thirds of population in India. class SVM Based Classification etc are some of techniques
On comparing the development of electronics and automobiles used for determining the fruit quality. Listed techniques are bit
with the development of agriculture, it is observed that the tricky to establish for real time application. The proposed
development in agriculture is sluggish. Hence, it is essential to method is designed to determine the quality of tomato. The
move towards the innovative plan to raise the impact factor software part is very efficient for defining the quality of
agriculture development. tomato by detennining ripeness of tomato, estimating of shape
and size and detecting of defects in tomato. Here, the grading
Fruits and vegetables market is subject of choice. Thus, it
system will grade tomato in four different classes, namely,
is important for the suppliers to label the quality of goods
class 1: high ripeness class, class 2: medium ripeness class,
before dealing out. Presently, human experts grade the
class 3: low ripeness class and class 4: defected class.
agriculture goods based on its optical features. Nevertheless,
manual inspection causes the inaccuracy, inconsistency and The chip embedded in Raspberry Pi development board is
inefficiency on defming the quality of agriculture goods. It is Broadcom BCM2835 Soc. The core architecture of this board
human nature to get tired and bore or become inattentive after is ARMIl. It has 512 SDRAM. Raspbian is the operation
a period of time. The improvement in the automatic grading system booted in the micro SD card, which is similar to the
system of fruit on the basis of quality will results in accurate, LINUX operation system. Any HDMIIDVI monitor or TV
efficient and consistent outputs. As a result, it save time and should work as display for Raspberry Pi. Any USB keyboard
human labors and will give a handful of help to the economic or mouse can be attached for use. The development board
development. Raspberry Pi Model B+ is shown in fig 1.
Stage 1: Green: More than 90% of surface is green. V. Leemans, H. Mageinb, M.F. Destain presented in [6]
apple grading system based on European standards. Mean of
Stage 2: Breaking: 90% of surface is green and color
colour was calculated from 1000 images of apple and
other than green covers not more than lO% of the
estimated ground color area of golden delicious and
surface.
Jonagold[7]. For healthy fruit, correct classification rate is 90-
Stage 3: Turning: Range from 10 to 30% of the surface is 95%. It is time saving process based on individual defect
yellow, combination of light red and green. recognisition. The demerits of this method are need of better
segmentation technique for bruise detection and high
Stage 4: Pink: Range from 30 to 60% of the blending of
misclassification rate. Correct classification rates for golden
red and yellow or red color.
delicious and Jonagold are 78% and 72%.
Stage 5: Light red: Between 60 to 90% of the surface is
Heron and Zachariah (1974) [8] designed an automatic
red.
sorting machine for tomatoes. A wide belt high capacity
Stage 6: Red: Range from 90 to 100% surface is red. concept of color sorting was introduced. The unripe tomato
was rejected and no action was taken on ripen tomato. Also
Blasco et al. [3] build up techniques using image analysis
background was considered. When only half tomato is in
for online determination of the quality of fruits like peaches,
viewing area, the amount of background captured is more.
apples and oranges based on parameter such as color, size, and
Thus accuracy can be achieved by increasing the scanning
defect-free. In the segmentation procedure fruit can be
speed for increased intake of tomato to optimum level with no
perfectly notified and separated from the background by using
adverse effect on accuracy.
Bayesian discriminate analysis. 22 surface sectors of tomatoes
varying from greenish to red color are taken into account to Changyong Li [9] designed an automatic sorting machine
determine color of tomato accurately. Blasco's system decides for tomatoes. They proposed the algorithm based on dominant
the color of fruit by comparing many standard color indices. color histogram matching method. This method provides high
Hunter Lab co-ordinate values calculate these color indices. A accuracy and is suitable for real time application. The demerit
colorimeter in three random circular areas (8mm diameter) in of this algorithm is, it is less sensitive to lighting. The accuracy
each of the selected sectors gives Hunter Lab co-ordinate provided by this method is 97.2%.
values.
215
Hongshe Dang, Jinguo Song, Qin Guo [10] have projected The Raspberry Pi has a 26-pin General Purpose
system based on image processing for detecting the fruit size Input/Output (GPIO) connector and this carries a set of signals
and grade them accordingly. The hardware consists of ARM9 and buses. There are 8 general purpose digital 110 pins.
as main processor which control all other hardware. The WiringPi is a GPIO access library written in C for the
software part for detecting the fruit size and grading is done on BCM2835 used in the Raspberry Pi. WiringPi includes a
the QT/Embedded platform. command-line utility gpio which can be used to program and
setup the GPIO pins. You can use this to read and write the
Dah-Jye Lee [11] introduced as system based on direct
pins and even use it to control them from shell scripts. The
color mapping. In this system, the overall grading result based
following tables give the mapping of the Raspberry Pi GPIO
on both color index and consistency averages are consistent
Pins to the (PI) GPIO connector in relation to the pin numbers
with human grading. This method is simple, effective and
and the physical location on the connector.
gives high accuracy which is of 95%. But the problem with
this method is, it is not generalized for different types of fruits. Table 1 shows the 40 pin GPIO Connectors mounted on
Raspberry Pi Model B+.
John B. Njoroge [12] has projected an automated grading
system using image processing. The system detects the TABLE 1: THE GPIO CONNECTORS
external and internal blemish. The hardware includes six CCD
Wirine; Pi Pin Header Header Wirine;Pi Pin
cameras which are arranged such that two cameras on either
3.3 V 1 2 5V
side and other two cameras are on the top of the fruit. X-ray GPI02 3 4 5V
imaging is utilized for detecting biological changes internally. GPIO 3 5 6 GND
An automated system grades the fruits and sorts them by GPI04 7 8 GPIO 14
analyzing the features. GND 9 IO GPIO 15
GPI017 11 12 GPIO 18
Sowmya and Sheelarani [13] use soft computing GPI027 13 14 GND
techniques for segmenting RGB color image. Neural network GPIO 22 15 16 GPI023
3.3 V I7 18 GPI024
and C means algorithm are used by them. Fuzzy set and Fussy
GPIO 10 19 20 GND
logic techniques are used by some researchers for solving
GPI09 21 22 GPI025
issues in segmentation. Fuzzy homogeneity approach was used GPIO II 23 24 GPI08
by Cheng et al. for the segmentation of RGB color image [15]. GND 25 26 GPI07
Genetic algorithm and artificial neural network techniques are DNC 27 28 DNC
also used for the same purpose [16]. GPIO 5 29 30 GND
GPI06 31 32 GPIO 12
III. PROPOSED SYSTEM OVERVIEW GPIO 13 33 34 GND
GPIO 19 35 36 GPIO 16
As discussed in introduction, the complete system is GPI026 37 38 GPI020
flexible, modular conveyor based fruit sorting system consist GND 39 40 GPI021
of Raspberry Pi Model B+ development board, conveyor belt
The system arrangement is done as shown in fig 4. The
system, sorting storage bins and Pi camera. Raspberry Pi is
target of our proposed system is to determine the ripeness of
itself a processor that manages all the information about
tomato and grade the accordingly. The steps to be carried out
motors and Pi camera. The signals to the Pi camera and motor
in system are image acquisition, feature extraction, sorting and
drivers are controlled by Raspberry Pi. It manages the speed of
grading. The block diagram of the system is shown in fig 2.
conveyor belt and keep updated about the other modules
Here, we have proposed a mechanical assembly consisting of
during fruit sorting process like ripeness determination, shape
conveyor system and grading assembly is shown in fig 3.
and size estimation, defect detection etc. Table 1 shows the
40 pin GPIO Connectors mounted on Raspberry Pi Model B+.
216
determination of ripeness of tomato, detection of color is must.
According to color detected from an RGB image,
determination of ripeness of tomato is done. So, if red color is
detected from RGB image of tomato captured by camera, then
it is categorized in high ripeness class and if green color is
detected from RGB image of tomato, then it is categorized in
low ripeness class. Same principle is applicable for medium
ripen tomato. The ripeness of tomato is determined in
Raspberry Pi Model B+ by using the OpenCV libraries and
thus, color is detected.
Fig. 3. Hardware Model
0.
217
V. SHAPE AND SIZE ESTIMATION with respect to defmed distance. A usual clustering algorithm
used is K-means Clustering. K-means clustering is a fast
Size defines how big or small the object is, while the
process as well as straight forward process. Basically, the input
shape gives the outline of external boundaries. Size of fruit
dataset is partitioned in to clusters. K-means clustering
defines the dimensions of fruit, while shape of fruit defmes the
calculates the distance between the inputs and centers, and
sketch out of fruit. To classify a fruit, size and color of fruit is
assigns inputs with the nearest centers.
required. Once the ripeness of tomato is determined, the
pointer turns to the estimation of shape and size of tomato. The When the number of clusters is set to 2, on cluster
area of each color label defmed by a user is computed on the contains fruit part and other part contains the defected part and
fruit surface. background. If the cluster number is set to 3 then the defected
part can be separated from the background effectively. On
Algorithm for estimation of shape and size of tomato:
increasing the number of clusters, clusters formed on an image
1. Divide the image into contours and label index to goes on increasing. Further, if the number of cluster is
them. increased to 4 or 5 or more than that, then it is not possible to
segment all the defected portions of fruit. Hence, the nominal
2. Compare the area of each contour.
or typical value for the number of clusters is 3 or 4.
3. Find the biggest area among them.
The algorithm for defect detection is as follows:
4. Include the smallest area into the largest area and
1. Obtain fruit location using fruit blob function.
change the index label to the index of largest area.
2. Crop the image of fruit entangled in the rectangle.
5. Repeat the step 2, 3 and 4 until the full tomato is
scanned. 3. Convert the cropped RGB image of tomato into HSV
image.
6. Find the biggest area among them and name it as the
fruit size. 4. Apply K-means clustering on HSV image of tomato.
Depending upon the area of each color, an image is 5. Generate images that segment the input image by
divided into different contours. Each contour is labeled with a color which will result in different images based on
unique index. On determining the area of each contour, number of clusters.
comparison is done among the areas of each contour. The area
During the clustering operation, a vector spaced is formed
of small contour is included into the area of large contour.
based on data features and identifies the natural clustering in
Thus, it changes the index label of contour to the index of the
largest area. Repeat the above procedure until the whole image them. Clustering of objects is done around the centroid /-li'
of tomato is scanned. Therefore, at the output we obtain the where i = 1 ,2 , 3 , . . . k which is calculated by minimizing the
biggest fruit blob, fruit location and fruit size. The fruit blob
below equation.
function gives the index of biggest area of contour and display
it as the fruit size.
v = L�=, �::CX j - /-lJ2
jESi
VI. FRUIT LOCATION
As the image acquisition process is synchronised, every Where k=number of clusters i.e. Si = 1,2,3, . . . k and /-li
fruit is approximately place at the same place with respect to
is nothing but the centroid.
the image co-ordinates. The fruits have to be separated so as to
avoid the wrong estimation of size, because of the fruits The basic algorithm for k-means clustering is given as:
touching each other. To identify and separate the fruit from
1. Initialize the centres of clusters.
others, the location of fruit is determined by the factor 'Fruit
Location'. 2. Attribute the closest cluster to each data point.
VII. DEFECTS DETECTION 3. Set position of each cluster to the mean of all data
points belonging to that cluster.
The value of the fruit depends on the quality of fruit.
Thus, it is important to separate out the defected fruit. Data 4. Repeat the steps b and c until local minimum.
partitioning is a most common technique used for the analysis
VIII. RESULT ANALYSIS
of statistical data. The applications of data partitioning are
pattern recognition, data mining, machine learning, image This section aims at the determination of ripeness of
analysis, bioinformatics etc. Clustering is a process which tomato. The work is done on the live image captured by
differentiates the objects into many different groups and the camera mounted at proper distance. Here, an input RGB image
data in each cluster contains some common characteristics of tomato is expected to be with white background. Fig 5
218
shows the conversion of original RGB image to HSV image. From above, it is observed that the pixel count of red color
Fig 6(a) shows the original RGB image. Fig 6(b) shows the is maximum. We have provided a facility in our system to
respective HSV image. detect and display the name of color from the pixel counts and
accordingly, decide the class of ripeness as shown in fig 9 and
10. The time required for the determination of ripeness of
tomato, estimation of shape and size and defect detection of
the respective image along with red, green ane yellow count is
shown in fig 11.
Area R=40084
Area Y=65
Area_ G=380
(a) (b) Red detected
K �eans cl u ster'i.n g t'i.�e 0.06 sec
T otal P rocess'i.n t'i.�e 1 .24 sec
(c)
Fig. 7. (a) Redness Mask, (b) Greenness Mask, (c) Yellowness Mask
Area R=40084
Area Y=6 5
Area G= 380 (c)
Fig. 8. Pixel Counts of Each Color Fig. 12. (a) Fruit Size, (b) Edge Detected Image, (c) Fruit Location
219
To demonstrate the performance of K-mean clustering them in defected class. The result analysis is able to accurately
algorithm approach, we have taken defected tomato as case determine the ripeness of tomato. The modularity and
study. Fig 13(a) shows an image of defected tomato. When distributed nature of approach makes the system easy to be
number of cluster is set to 2, fruit part comprises one cluster upgraded in the future in order to increase quality standard.
while defected part and background comprises another one.
We have segmented the input image of defected tomato into REFERENCES
three clusters. Fig 13(b) shows HSV image of RGB image of [I] Jaramillo J., Rodriguez V., Guzman M., Zapata M. and Rengifo T.
defected tomato. If defected area is larger, cluster count will be (2007), "Technical Manual: Good Agricultural Practices in the
Production of Tomato under Protected Conditions". FAO 2007.
less while if defected area is smaller, cluster count will be
[2] Seng, Woo Chaw and Mirisaee, Seyed Hadi (2009), "A New Method for
more. Therefore, number of clusters for defect detection from Fruits Recognition System", Electrical Engineering and lriformatics,
the defected image is inversely proportional to the defected Vol. 01, pp. 130-134, August 2009.
area. Clustered image is shown in fig 13(c). Fig 14 shows the [3] Blasco 1., Aleixos N. and Mollo E. "Machine Vision System for
Automatic Quality Grading of Fruit", Biosystems Engineering, 2003.
time required for K-mean clustering.
Vol. 85: pp. 415-423.
[4] Nakano, Kazuhiro (1997), "Application of Neural Networks to the Color
Grading of Apples", Computers and Electronics in Agriculture, Elsevier,
pp. 105-116, 1997.
[5] Chen, Xuming and Yang, Simon X. (2013), "A Practical Solution for
Ripe Tomato Recognition and Localisation", Journal of Real-Time
Image Processing, Vol. 8, No. I, pp. 35-51, January 2013.
[6] Leemans, V., Mageinb, H. and Destain, M.F. (2002), "On-line Fruit
Grading According to their External Quality using Machine Vision",
Journal of Automation and Emerging Technologies, Belgium. Biosystems
(a) (b) Engineering, pp. 397-404, 2002
[7] Leemans, V. and Destain, M.F. (2004), "A Real-time Grading Method of
Apples based on Features Extracted from Defects", Journal of Food
Engineering, Vol. 61, No. 1, pp. 83-89, January 2004.
[8] Heron, J.R., and Zachariah, G.L. (1974), "Automatic Sorting of
Processing Tomatoes". Transaction of the ISAE, Vol. 17, No, (5):
pp. 987-992.
[9] Li, Changyong, Cao, Qixin and Guo, Feng (2009), "A Method for Color
Classification of Fruits Based on Machine Vision", WTOS 8, Vol. 2,
pp. 312-321, February, 2009.
[10] Dang, Hongshe, Song, Jinguo and Guo, Qin (2010), "A Fruit Size
(c) Detecting and Grading System Based on Image Processing," 2010
Second International Conference on Intelligent Human-Machine Systems
Fig 13. (a) Original Defected Image, (b) HSV Image, (c) Clustered Image and Cybernetics, pp. 83-86.
[11] Lee, Dah-Jye, Archibald, James K. and Xiong, Guangming (2011),
in Data preparing 0.23 sec "Rapid Color Grading for Fruit Quality Evaluation Using Direct Color
Mapping", IEEE Transactions on Automation Science and Engineering,
H mean processing 0.53 sec Vol. 8, No. 2, pp. 292-302, November 2011.
out Data Preparing processing 0.13 sec [12] Njoroge, John B., Kazunori Ninomiya. Naoshi Kondo and Toita, Hideki
(2002), "Automated Fruit Grading System using Image Processing", The
Total processing 0.89 sec Society of Instrument and Control Engineers(SICE2002), Osaka, Japan,
August 2002, pp 1346-1351.
Fig. 14. Time Required for K-mean Clustering [13] Ms. Jadhav, Rupali S. and Prof. Pati!, S.S. (2013), "A Fruit Quality
Management System Based on Image Processing" IOSR Journal of
IX. CONCLUSION Electronics and Communication Engineering (IOSR-JECE) e-ISSN:
2278-2834, p- ISSN: 2278-8735.Vol, 8, Issue 6 (Nov.-Dec. 2013), pp.
The proposed method is demonstration version of fruit 01-05.
sorting system and it is implemented on Raspberry Pi [14] David, J.F., Yau, K.Y. and Elmagarmid, A.K. (2001), "Automatic Image
development board. This paper proposed and appraised the Segmentation by Integrating Color-edge Extraction and Seeded Region
Growing", IEEE Transactions On Image Processing (TIP), Vol. 10,
frame for the determination of the ripeness of tomato,
No. 10, pp. 1454-1466, 200l.
estimation of size and shape and detection of defects of [15] Sowmya, B. and Sheelarani, B. (2009), "Colour Image Segmentation
tomato. This work presents the novel incorporated technique using Soft Computing Techniques", International Journal of Soft
for grading tomatoes based on their ripeness. The proposed Computing Applications, Vol. 4, pp. 69-80, 2009.
[16] Cheng, H.D., Chen, C.H., Chiu, H.H. and Xu, H. "Fuzzy Homogeneity
approach used the concept of color detection algorithm for
Approach to Multilevel Thresholding", IEEE Transactions On Image
classifying tomato in three classes and K-mean clustering Processing (TIP), Vol. 7, No. 7, pp. 1084-1086, 1998.
algorithm for detecting the defects in tomato and classifying
220