DATA RETRIEVAL USING HAND DRAWN QUERIES

A.D.M. Kushwaha1 A. Bhaskar2 M. Jain2 N. Gupta2 S. Datta2

1
Lecturer, Department of Electronics and Communication Engineering
Birla Institute of Technology, Jaipur (India)
2
Student, B.E. VIII Semester, Department of Electronics and Communication Engineering
Birla Institute of Technology, Jaipur (India)

materials from the hand-drawn sketch database by

sketching a query drawing.
ABSTRACT
2. SYSTEM DESCRIPTION

The storage and retrieval of drawings either technical or

free hand sketches, hand written notes, maps and texts is
pretty cumbersome as the database could be very large
and remembering the name of every file is close to
impossible. In this paper we propose a method to retrieve
sketches stored in the form of multiple strokes, by
extracting the shape information for each stroke and by
considering the spatial relation between the strokes. The
spatial relation between the strokes can be used to
determine the stroke correspondence between the query
sketch and the database sketch with matching score.
Important features are extracted by Edge detection
algorithm and hierarchically matched to the files in the
database.

1. INTRODUCTION

Pen-based devices such as Personal Digital Assistants Figure 1: System diagram

(PDA) and electronic whiteboards have become more
and more common to the general public. The captured
database is stored in the form of sketch which consists of
strokes that are sequences of coordinates of the points
sampled by the pen-based device. It is necessary to have
an efficient sketch retrieval scheme in order to allow
users to search for relevant information from the
database. Retrieval in the sketch database is equivalent to
finding a stroke or multiple strokes from the database
that are a good match to the query stroke(s). Sketch
retrieval can be applied to various applications such as
free-form hand-drawings, hand-written or printed text
and trademarks. Searching through a collection of free-
form hand-drawings [1] provides a good motivation for
sketch retrieval. The students can retrieve relevant lecture
2.1 Stroke Merging
A single stroke may sometimes be drawn as smaller
broken strokes. In order to account for these different
styles, stroke merging is performed to merge broken
strokes based on the proximity of the end points. A stroke
is first checked whether its two end points are close. If
they are not close, then that stroke is considered as open
stroke and it may be merged with another open stroke if
the end point of one stroke is close to the end point of the
other. For example, in Figure-2 (a), the two end points of
one of the strokes are very close thus it is not an open
stroke and the two strokes will not be merged. On the
other hand, in Figure-2 (b), the four strokes are open
strokes and the end point of one stroke is close to the end
point of another stroke, therefore these four strokes will
be merged to become one stroke.
Figure2: Stroke merging
2.2 Hierarchy Construction
This module is responsible for building a stroke
hierarchical representation for a sketch. The stroke
hierarchy describes the structural relationship between
the strokes in a sketch.
2.3 Sketch Simplification
The process of detecting and representing the shaded area
as a single unit is called sketch simplification [2,3]. A
region consisting of a stroke with all its descendent
strokes is more likely to be a shaded region if the ink
density in that area is high. A second order function is
chosen as the decision boundary because the training data
cannot be well separated by a linear function.
After a region is decided as a shaded region, all those
strokes forming this region will be replaced by a single
hyper-stroke keeping only features about that region in
the stroke hierarchy.
2.4 Feature Extraction
Three kinds of features are extracted from a sketch. The
purpose of matching in three different feature spaces is to
consider various aspects of similarity including overall
region similarity, stroke similarity and spatial relation
similarity [4]. Figure-3 shows the different features.
2.4.1 Hyper-Stroke Features
A hyper-stroke consists of a parent stroke with all its
descendent strokes.
2.4.2 Stroke Features
Stroke features are geometric features extracted from
each stroke. Different features are used to determine the
likelihood that each stroke falls in each basic shape: line,
circle and polygon.
2.4.3 Bi-Stroke Features
A bi-stroke feature consists of stroke features of a pair of
strokes within the same level together with the spatial
relation between them.
Figure 3: Example features
2.5Hierarchical Matching
The similarity is evaluated in a top to bottom hierarchical
manner i.e. the similarity scores are computed for the
hyper-stroke features, stroke features and the bi-stroke
features at each level of the query stroke hierarchy. The
final similarity score is computed as the weighted sum of
the scores from the three feature spaces and the similarity
score from the stroke hierarchical structures [2].
3. CONCLUSION
This paper discusses a method for simplifying and
retrieving hand-drawn sketches in a hierarchical manner.
Sketch simplification allows the shaded region to be
represented by a single hyper-stroke. Experiments show
that the hierarchical matching outperforms some existing
methods in terms of the retrieval performance. The
processing time can be improved by exploring other
possibilities of evaluating the similarity of the stroke
hierarchy structures and algorithms.
REFERENCES
[1] Wing Ho Leung and Tsuhan Chen. User-Independent
Retrieval of Free-Form Hand-Drawn Sketches. In
Proceedings of the IEEE International Conference on
Acoustics Speech and Signal Processing (ICASSP 2002),
volume 2, pages 2029–2032, Orlando, Florida, USA,
May 2002. IEEE Press.
[2] Wing Ho Leung and Tsuhan Chen. Retrieval of
Sketches Based on Spatial Relation Between Strokes. In
Proceedings of the IEEE International Conference on
Image Processing (ICIP 2002), volume 1, pages 908–
911, Rochester, New York, USA, September 2002. IEEE
Press.
[3] D. and Tomkins A. “Temporal domain matching of
hand-drawn pictorial queries”. In Proc. of the Seventh
Conf. of The Intl. Graphonomics Society, pp. 98-99.
August 1995.
[4] Leung W. H. and Chen T. “User-
Independent Retrieval of Free-Form Hand-
Drawn Sketches”. To appear in ICASSP2002,
May 2002.