Presented By

Pratap Solapur

Under the guidance of
Prof. P. B. Patil

Introduction
System Description
Experimental Results
Live Demo
Conclusion



CBIR
- Extract low-level features for representing visual content
- Find similar images by measuring similarity on visual
features
A Content Based Image Retrieval system using
users assisted neural network inference is
described.
The main attention of the project was to design a
neural network decision tree driven by the users
relevance feedback as a tool for image retrieval
system.

user
Query Image
Image DB
Feature
Extraction
Feature
Extraction
Feature
Vector
Feature
Database
Similarity
Comparison
Indexing &
Retrieval
Retrieved
Results
Relevance
Feedback
Query
Formation
Neural Net
BDT class
Prediction
Reduced
Search space
O/p
Satisfied
with
results?
No
Create and
Train
Neural
NW-BDT
1
2
Extracting following features for each image in DB
and the query image also.
Color Histogram
input: Image, in RGB(MxNx3)
output:1x64 color histogram <== (HxSxV = 8x4x2)
Edge Histogram and Edge Direction
input: MxNx3 image data, in RGB or MxN
luminance channel
output: 1x5 edge histogram <== global 5
bins (0, 45, 90, 135, no-orientation) [1]


Lets take a set of samples
1

2
, ...,

each one
labeled by

{
1
,
2
, ...,

} where N is the
number of classes.
The tree is constructed with following steps

1. Divide the classes in two disjoint groups g1 and g2.
This is performed by calculating N gravity centers
for the N different classes.
2. The two classes that have the biggest Euclidean
distance from each other are assigned to each of
the two clustering groups.
Neural
NW
Neural
NW
Neural
NW
Neural
NW
Neural
NW
Neural
NW
Neural
NW
Neural
NW
4
6
5
2 9
1 7
8
3 10
g1[5,6,2,9,4]
g2[7,1,8,10,3]
g11[5,6,4]
g12[9,2]
g21[6,4]
g13[3,8,10
]
g14[7,1]
g22[6,4]
[1,2,3,4,5,6,7,8,9,10]
3. The class with the smallest Euclidean distance from
one of the clustering groups is found and assigned
to the corresponding group.
4. The gravity center of this group is then
recalculated to represent the addition of the
samples of the new class to the group.
5. The process continues until all unassigned classes
are assigned to one of the two possible groups.
6. The process continues recursively (dividing each of
the groups into two subgroups applying the
procedure explained above), until there is only one
class per group which defines a leaf in the decision
tree.
Training Neural NW BDT
Neural NW classifier at each non leaf node of the
Neural NW BDT is trained so as to predict the
label of any query image.
The retrieving procedure is based on Euclidean
distance measure between FVs of a query image and
images from database

-is the initial query FV

- is the FV associated with the i-th image in the

database.
The user is asked to specify the integer number K and
then the system retrieves and displays first K images
closest to the query based on

=d

= (

J
j=1

From K retrieved images user recognizes and
labels R images as relevant i.e. most similar to a
query.
Feature vectors of those images are used to modify
a query FV,

=mean(

)
i.e. mean values of FVs of relevant images.
The updated query is then fed to the Neural NW
based binary decision tree to determine the class,
to which it belongs.

The modified FV

is given to the binary classifier

at root level.
According to its result, it branches the query to
either left or right branch of the root, and the
process continues until it reaches the leaf level.
At leaf level the class label C of the

is obtained.
Calculate the distance between the query image FV
and the FVs of the images belonging to class C
Index the results, in the ascending order of
distance and display top K images.
Experiments are conducted on the COREL dataset
No of images in DB : 1000 images of size 384 x
256.
10 semantic categories consisting of 100 images
each.
Effectiveness of the system is measured by
precision and recall parameters.
=



=



Category 1
st
Iteration 2
nd
Iteration(With
Feedback)
3
rd
Iteration(With
Feedback)
Precision Recall Precision Recall Precision Recall
Africans
73% .126 100% .20 100% .20
Sea shore
Views
50% .10 100%

.20

100% .20
Buildings
45% .103 100% .20 100% .20
Buses
71.11% .142 100% .20 100% .20
Dinosaurs
91.66% .183 100% .20 100% .20
Elephants
53.5% .107 100% .20 100% .20
Roses
62.27% .133 100% .20 100% .20
Horses
80.71% .161 100% .20 100% .20
Mountains
30% .07 100% .20 100% .20
Dishes
66.5% .133 100% .20 100% .20
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4 5 6 7 8 9 10
P
r
e
c
i
s
i
o
n
/
R
e
c
a
l
l

Semantic Categories
Precision-Recall
Precision
Recall
Quick live demo to give you a full context before
digging into the code

1
st
Iteration Precision 85%
2
nd
Iteration(With Feedback) Precision 100%
A CBIR with RF framework is proposed which
performs efficient and quick retrieval using Neural
Network based Binary decision tree.
The Neural Net-BDT is used to predict the class label of
the updated query, formed from user relevance
feedback.
Thus, only images from this class are compared with
the query image, which reduces search space and
thereby retrieval time.
Future work : As the semantic categories increase the
complexity of system increases, we need to work on
this.


1. Manjunath, B.S.; Ohm, J.-R.; Vasudevan, V.V.; Yamada, A., "Color and
texture descriptors" IEEE Trans. CSVT, Volume: 11 Issue: 6 , Page(s):
703 -715, June 2001 (section III.B)
2. A Multi-class SVM Classifier Utilizing Binary Decision Tree Gjorgji
Madzarov, Dejan Gjorgjevikj and Ivan ChorbevDepartment of Computer
Science and Engineering Faculty of Electrical Engineering and
Information Technology Karpos 2 b.b., 1000 Skopje, Macedonia E-mail:
madzarovg@feit.ukim.edu.mk
3. S. MacArthur, C. Brodley, and C. Shyu, Relevance feedback decision
trees in content-based image retrieval, In IEEE Workshop on Content-based
Access of Image and Video Libraries(CBAIVL00), pp 68 72, 2000.