Autonomous Systems Lab

Prof. Roland Siegwart

Semester-Thesis
Supervised by: Author:
Markus Achtelik Martin L. Wyss
Laurent Kneip
Robustness of dierent
feature extraction methods
against image compression
Spring Term 2011
Contents
Abstract iii
Symbols v
1 Introduction 1
2 JPEG-Compression 3
2.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 Feature detectors and descriptors 5
3.1 FAST corner detection . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 SURF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 BRIEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 Method 7
4.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2 Evaluation criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2.1 Repeatability . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2.2 Recall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
5 Framework 9
5.1 Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5.1.1 Generating raw data . . . . . . . . . . . . . . . . . . . . . . . 10
5.1.2 Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.1.3 Single evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 12
6 Results 13
6.1 Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6.1.1 Number of features . . . . . . . . . . . . . . . . . . . . . . . . 13
6.1.2 Region match over quality . . . . . . . . . . . . . . . . . . . . 13
6.1.3 Region match over translation . . . . . . . . . . . . . . . . . 14
6.2 Descriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.2.1 Number of matches over quality . . . . . . . . . . . . . . . . 15
6.2.2 Number of matches over translation . . . . . . . . . . . . . . 15
6.2.3 Recall over quality . . . . . . . . . . . . . . . . . . . . . . . . 15
6.2.4 Recall over translation . . . . . . . . . . . . . . . . . . . . . . 16
7 Conclusion 17
7.1 Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
7.2 Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
7.2.1 Detectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
7.2.2 Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
7.3 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
i
A Feature properties 19
B Framework 21
B.1 Variable list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Bibliography 23
ii
Abstract
This Semester-Thesis evaluates the robustness of dierent feature extraction meth-
ods against image compression. The robustness was tested for JPG compression, a
loss aicted compression method. The selected feature detectors were FAST and
SURF. As descriptors SURF, BRIEF and an image patch were chosen.
A Matlab framework was developed to do the evaluations. Results show why SURF
is not an appropriate feature for JPG images in motion. FAST and BRIEF turned
out to be robust against compression and motion.
iii
iv
Symbols
Acronyms and Abbreviations
ACU Accuracy of corner detection
ALU Arithmetic Logic Unit
CCN Consistency of Corner Numbers
DCT Discrete Cosine Transformation
ETH Eidgenossische Technische Hochschule
FAST Feature from Accelerated Segment Test
GTI Ground Truth Image
IDCT Inverse Discrete Cosine Transformation
JPEG Joint Photographic Experts Group
JPG Short for JPEG
NCC Normalized Cross Correlation
SIFT Scale-invariant feature transform
SSD Sum of Squared Dierences
SURF Speed Up Robust Features
v
vi
Chapter 1
Introduction
As part of the sFly project, the robustness of dierent feature detectors and descrip-
tors against image compression with the common JPG format had to be examined.
sFly aims for micro aerial vehicles exploring urban environments. In order to pro-
cess the captured information, a data stream is sent to a ground station. Image
compression is required in order to deal with the limited transmission bandwidth.
JPG has been chosen because it is one of the most common compression methods
for todays photo cameras.
Figure 1.1: sFly, micro aerial vehicles exploring cities.
1
Chapter 1. Introduction 2
Chapter 2
JPEG-Compression
Since the JPEG format is the basis for this thesis, it is essential to understand the
JEPG compression algorithm.
2.1 Description
JPEG is a loss aicted compression method for saving pictures. The quality can be
expressed in percent. 100% means no compression and 0% is the maximum which
can be reached. With increasing compression the Quality gets worse and the le
size smaller.
Important is the fact that 100% quality is already loss aected. To show the eect
the dierence between a BMP and a 100% quality JPG version of the same RGB
image has been taken pixelwise. In the resulting image the pixels greater equal to
one are set to 255. The result can be seen in gure 2.1. The result shows that there
is a structure related error even if the human eye is not able to see it.
Figure 2.1: Visualisation of losses between BMP and JPG.
Technically spoken the picture gets processed by the following steps:
1. RGB colours get mapped onto the YCbCr-space
1
2. Divided in 8x8 pixel matrices.
3. Every matrix get DCT transformed into a matrix G. The elements in the G
matrix depend on the frequencies.
4. Quantisation of the matrices. The matrices get divided elementwise by the
matrix Q and rounded, which is loss aected. The values in the Q matrix
1
Usually used with the standard CCIR 601
3
Chapter 2. JPEG-Compression 4
take into account that the human eye is not very good in distinguishing the
exact strength of a high frequency brightness variation. This means that for
high frequencies the divisor in the Q matrix gets bigger.
B
j,k
= round(
G
j,k
Q
j,k
); for j = 0, 1, 2, . . . , 7; k = 0, 1, 2, . . . , 7 (2.1)
5. Using Human coding, the B matrices nally get compressed. Human cod-
ing is a kind of entropy coding which is a lossless data compression. Before
applying Human the data gets rearranged in a zigzag pattern. As mentioned
Figure 2.2: zigzag pattern, picture from Wikipedia by Alex Khristov
the divisor gets bigger with increasing frequency so the further the rearrange-
ment gets carried out the smaller the number in the B matrix become. If the
rearranged B matrix shows a zero sequence at the end the zeros can be cut o
and the end of the matrix must be marked with the special code word EOB.
Human coding can be applied on the rest.
This algorithm can be inverted in order to restore the picture.
Chapter 3
Feature detectors and
descriptors
3.1 FAST corner detection
The FAST corner detector nds local invariant points. It nds corners in the pic-
tures by comparing pixel-gradients in a neighbourhood of pixels. Matlab implemen-
tations are available online [3]. The detector is presented in the paper of E. Rosten
and T. Drummond [4].
Figure 3.1: Visualisation of the FAST corner detection feature[3]
In the picture 3.1 a discrete circle with radius 3 and a reference point p is shown.
To extract corners a grey scaled picture is sucient and allows much faster extrac-
tion than a coloured one. In order to detect an existing corner, the grey scale of
the pixels lying on the discrete circle are compared with the centre pixel p. If a
certain consecutive number of dierences lies above or below a certain threshold t,
the considered pixel is marked as corner. t serves as parameter for controlling the
total numbers of extracted corners.
FAST detects sometimes more than one corner in a certain neighbourhood. In order
to reduce the number of found corners only the corner with the highest cornerness
is kept. This is as well called non maximal suppression which is described in [5].
Optimizing the calculation cost can be done by examining the pixels 1,9,5 and 13
because a feature can only exist when three of them are beyond the threshold. With
this algorithm only 2.8 pixels have to be examined in average. To achieve even a
lower examination rate a machine learning approach was done in [5] which reached
5
Chapter 3. Feature detectors and descriptors 6
an average of 2.26 dierence checks per pixel.
Due to low cost of comparing a small amount of pixels, the FAST feature is much
faster than the SUSAN or the Harris feature. The amount of potential frames per
second depends on the threshold. The higher the threshold, the less feature get
detected and the higher the speed of detection.
3.2 SURF
SURF comes as a combined detector and descriptor. The SURF detector is a
modied version of the SIFT algorithm developed by Herbert Bay and his colleagues
[9]. SURF is known to be faster and more robust against dierent transformation
than SIFT. A list of properties can be found in appendix A. This is achieved by:
Using integral images for image convolution
Using a Hessian matrix-based measure for the detection
The algorithm for SURF is much more computationally demanding than the FAST
algorithm. The description can be found in [8].
The SURF detector can be extended towards ane invariant regions using ane
normalisation of the ellipse. This increases the computation time.
3.3 BRIEF
BRIEF is just a descriptor. It can be combined with any feature detector i.e.
FAST. BRIEF compares the brightness of pixels in the neighbourhood of the feature.
Which pixels have to be compared is predened and such lists can be found online
[10]. It is very fast because the descriptor stores bitwise information. For 512 pairs
just 64 bytes are used and bitwise operations are very fast on the ALU. Because
there was no Matlab code online available, an own implementation of the BRIEF
feature had to be done.
The input arguments are a blurred image and a list of the feature coordinates.
brief.m gives back the descriptor of the format 512xn. The image can be blurred
by the code in listing 3.1.
Listing 3.1: Smoothing function from Matlab
1 imfilter(self.im_array{i,i2}.grey ,kernel ,conv );
Chapter 4
Method
The aim of the project was to test feature detectors and descriptors separately for
robustness against compression and translation.
4.1 Datasets
The source for the evaluations were three GTI of which six quality steps were taken
(100, 90, 70, 50, 20, 0%) for seven translation steps (0 ,1 ,2, 3, 4, 6, 8 pixels in
x-direction). So 42 images had to be analysed.
Boat Theme Castle Theme Grati Theme
Figure 4.1: The chosen GTI from [7]
4.2 Evaluation criteria
The experiments were inspired and derived from Krystian Mikolajczyks research[2].
4.2.1 Repeatability
The formula below stands for the repeatability. A correspondence is a feature at
the same spot in both images, which overlaps by 50% in the area, of course with
eventual translation taken into account. Features do have a pattern, a radius or
a scale, which allows to calculate the overlap geometrically. The found amount
of correspondences is then divided by the minimum of the features found in both
images (GTI and compressed image). This shows how well the geometry of the
feature can be found back. A score for the detector results between 0 and 1 (where
7
Chapter 4. Method 8
one is the best grade).
repeatability =
# correspondences
min(# gt features, # img feature)
(4.1)
4.2.2 Recall
The recall gives a possibility to evaluate the descriptor. Again a score between 0
and 1 results where 1 is the best grade.
A match is found by comparing all descriptors from the GTI to the ones of the
compressed image by taking the dierence between the descriptor vectors. For each
candidate match, the absolute values of the dierence vector can then be summed
up and the smallest resulting number is considered to be the best match. However,
the smallest dierence still needs to lie below a certain threshold value in order to
be considered as a match. A correct match is dened as a match which also is a
geometrical correspondence (as explained in subsection 4.2.1).
recall =
# correct matches
# matches
(4.2)
Chapter 5
Framework
Figure 5.1: Framework datastructure and dataow.
A high number of tests for a lot of images had to be done, therefore a framework
was developed to generate the evaluation automatically. The program structure
and data ow can be seen in the gure 5.1. The program consists of four Matlab
les:
1. A super class im data.m,
2. a child class eval data.m,
3. an evaluation script generic script.m,
4. and a test mask multiplot.m.
generic script.m runs all evaluations for the themes and for the features separately.
For example for three themes and three features the script runs nine evaluations
and stores nine datasets. An evaluation is independent of other features or themes,
at this point the multiplot.m gets important. In order to plot dierent evaluation
results into one Matlab gure, a test mask is needed which holds the access to all
9
Chapter 5. Framework 10
evaluations. The separate evaluations carried out by the class eval data.m are based
on the raw image and feature data, which are generated from the class im data.m.
Therefore, the class eval data.m inherits the class im eval.m.
While the three rst items in the list are generic the last one is not. The test mask
has to be adjusted for every change in the input arguments. To emphasize on this
point the structure can be kept, but the indices have to be changed in for-loops.
In this chapter, the program is not explained in every detail. In case some adjust-
ments will be necessary in the future, the list of all variables can be found in the
appendix B.1.
5.1 Usage
5.1.1 Generating raw data
The framework for the whole evaluation is implemented in the le generic script.m.
For the reason of future changes, the input arguments are explained here.
Because im data is inherited by eval data the constructor of eval data automatically
executes the constructor of im data. Therefore the input arguments for both classes
have to be handed over for creating an instance. Moreover, the raw data is generated
instantly by the constructor of the super class.
Listing 5.1: Creating an instance with the shown arguments.
1 comp_method = jpg ;
2 quality_list = [100 ,90 ,70 ,50 ,20 ,0];
3 trans_list = [0 1 2 3 4 6 8; 0 0 0 0 0 0 0];
4 rot_list = 0; % not in use
5 path_origin {1}=C:\ boatSc100.jpg ;
6 %path_origin {2}=C:\ grafiti100.jpg ;
7 path_save = P:\ semArbeitSave \;
8 feature {1} = brief ; % fast , brief , surf
9 %feature {2} = fast ;
10 im_obj = eval_data(comp_method ,quality_list ,trans_list ,rot_list ,
path_origin{index},path_save ,feature{index});
The brackets used in the Matlab code in listing 5.2, can be set for path origin and
feature in order to run the evaluation automatically for multiple images and feature
types. Below is a short explanation of the listed code:
1. The compression method is set to be JPEG. No other compression method is
supported in this framework version.
2. The quality list stands for the quality steps in percent.
3. The translation list states how much the compressed image is shifted to the
ground truth image. It can be shifted in x and y direction. The example above
just addresses a translation in x direction from 0 to 8 pixels (the translation
should not be bigger than 8 pixels). In order to simulate the translation, a
window in the GTI is selected that gets shifted over the image.
4. The rotation list is not in use. It was implemented for future tests.
5. The program needs one ground truth image for every theme. Note that the
image has to be named with 6 letters for the theme, three letters for the
quality, and with the ending jpg (not jpeg). For example castle100.jpg.
6. In the script generic script.m, multiple themes can be evaluated by setting
multiple path originn.
11 5.1. Usage
7. path save tells where the successfully generated evaluations and raw data are
saved.
8. The feature has to be set to fast, surf or brief, no other feature is imple-
mented so far. When fast is selected, the patch descriptor is chosen, and
when brief is selected, the FAST feature will cover the detector part.
9. As explained above, setting multiple features in generic script.m, will run
multiple evaluations at once.
All input arguments above have to be set as explained. The generated image list
can be found in gure 5.2. Please note that only one feature or image theme can be
handed over to the constructor of the class eval data.m. Multiple evaluations are
executed by generic script.m.
In the gure 5.2 the generated image list by the above explained input arguments
can be seen.
Figure 5.2: Generated image list shown by Matlab at execution time.
In the title, boatSc stands for the image theme. Each line indicates that the com-
pression method is jpg and the feature is BRIEF. In the rst line the translation is
0 in x and 0 in y direction. The numbers stand for the quality. There are two 100%
entries because one is the GTI which is unshifted, and the other is the shifted GTI.
The quality steps are taken from the shifted 100% image.
5.1.2 Evaluations
Figure 5.3: Evaluation list.
All evaluations are methods of the class eval data. As can be seen in gure 5.3,
the evaluations get listed up after they are successfully nished. New evaluations
are saved and attached at the end of the list. For every theme and feature an
own dataset is created. The number of the evaluation is called evaluation id. The
expression feature nr is related to the kind of evaluation. The expression gt(0 0)
stands for the ground truth image, and the rest for the evaluated image pairs. The
Chapter 5. Framework 12
numbers (in the red circle) are indices of the image list. Because Matlab does not
know the index zero, the expression gt(0 0) actually means that there is no ground
truth image.
Below is a list of the evaluations in use:
feature nr: Refers to number of found features.
location match: Refers to region match or repeatability.
pair match: Refers to pair match or recall.
All of these analyses have an optional parameter. 1 stands for evaluation over quality
and 2 for evaluation over translation. This parameter has to be set for getting the
appropriate x-axis and scaling in the gures. A list of GTIs
1
gt trans or gt comp,
and a list of compressed images over trans or over comp are needed. Those lists
are generated automatically in generic scirpt.m when the input parameters are set
properly. All lists are of the form nx2, where for every entry in the gt list a new
cell array entry has to be created (see listing 5.2).
Listing 5.2: Creating an instance with the shown arguments.
1 gt_trans = [2 ,2;3 ,2];% two entries
2 over_trans {1} = [2 2 2 2; 2 3 4 5];%gt entry one
3 over_trans {2} = [3 3 3 3; 2 3 4 5];%gt entry two
Those numbers for the lists can be picked from the raw image data array. In gure
5.4, the red ovals mark the indices over translation, and the green ones those over
quality. These ovals could be obviously drawn for every row and column.
Figure 5.4: Red: Over translation, Green: Over quality.
5.1.3 Single evaluation
Due to the fact that it can be very time intensive to run always a whole set of
evaluations, a stored dataset can be restored and single evaluations can be carried
out on it. Therefore the variable s in generic script.m has to be set to the desired
case of the switch case structure. In order to have the program evaluate a whole
set of data s has to be set back to 0 (zero).
In the single case mode, a few new cases can be used which are not meant to be
used by generating a whole dataset:
-2: Delete evaluations: A dialoguebox appears where the evaluation id can be
handed over or multiple evaluation ids with the colon operator, e.g. 3:10.
-3: Show evaluation: After an evaluation is made, it can be displayed with
this case. Again the colon works for showing multiple evaluations.
1
except feature nr
Chapter 6
Results
For the evaluation, an evaluation mask was programmed to give access to dierent
features. This allowed to plot dierent feature evaluations into the same gure.
The evaluation can be split up into descriptor and detector evaluation.
6.1 Detector
In this part of the evaluation the two features FAST and SURF are examined.
6.1.1 Number of features
According gure 6.1 for every image theme the amount of found features is about
800 for the GTI. This is the case because the detector threshold was set accordingly
in order to have a fair evaluation.
FAST seems to be more unstable than SURF, but there is no rule of thumb. The
numbers of found features does not change strongly over translation. The scene
texture, however, seems to have a signicant inuence.
Boat Theme Castle Theme Grati Theme
Figure 6.1: The chosen GTI from [7]
6.1.2 Region match over quality
As can be observed in gure 6.2 the SURF feature shows slightly better results
above a quality of 20%. Below 20%, both features drop quickly.
13
Chapter 6. Results 14
Boat Theme Castle Theme Grati Theme
Figure 6.2: Repeatability over quality
6.1.3 Region match over translation
For this tests the gures look pretty similar for all three themes, therefore only the
castle theme is shown.
The results for SURF happen to be pretty chaotic for a quality of 100%. This hap-
pens because the quantisation takes already place at 100% quality, even when the
human eye is not able to notice the JPG compression pattern. The image transla-
tion leads to the fact that the JPG compression pattern group dierent pixels, and
hence dierent blob features are being cut depending on the translation. It can be
stated that the SURF detector is very sensitive to the pixel regrouping, while the
FAST feature is much more stable. A proof for the relation between repeatability
and the JPG compression pattern is the fact that after 8 pixel of translation the
score goes back to 1 again. The reason that it does not reach one, is due to the fact
that a few features are moving out of the image and a few new ones are moving
into the image. Those features can not be found (anymore) in the GTI.
It can be observed that the FAST feature detector gets worse with decreasing qual-
ity. SURF does not decline as fast and so for a high compression FAST has a worse
result for repeatability than SURF.
Castle Theme 100% Castle Theme 70% Castle Theme 0%
Figure 6.3: Repeatability over translation.
15 6.2. Descriptor
6.2 Descriptor
6.2.1 Number of matches over quality
In this evaluation pairs stand for matches. This evaluation analyses the image
information consistency by matching the 800 GTI features. All three curves behave
similar and return reasonable results above 20%. This originates from the fact that
the second threshold is being applied. In the grati theme about 70% of quality,
all three curves had to pass through 90% of inliers after the threshold was applied
(explained in section 4.2.2). Of course this threshold can be tuned and optimized,
but for this evaluations it was xed.
Boat Theme Castle Theme Grati Theme
Figure 6.4: Number of matches over quality.
6.2.2 Number of matches over translation
Over translation it can be seen again that the SURF curve looks chaotic for a
quality of 100%. SURF is loosing about 50% of the image information content
for a translation of 3 pixels. BRIEF and the image patch are doing quite well
for 100% quality and they get successively worse over compression. By comparing
the gures to the repeatability of the region match, it can be seen that the curves
look very similar. This comes from the fact that the descriptors are not completely
independent from the detectors, but SURF still looses more features during the
matching.
Castle Theme 100% Castle Theme 70% Castle Theme 0%
Figure 6.5: Numbers of matches over translation.
6.2.3 Recall over quality
Figure 6.6 focuses on the recall to grade the descriptors over quality. The gure
looks similar for every theme, so only the castle Theme is shown.
Chapter 6. Results 16
SURF reaches a very stable recall around 1. Brief is doing nearly as well and patch
is decreasing quicker. After 20% the recall is getting rapidly worse for BRIEF and
the image patch.
Figure 6.6: Recall over quality for the castle theme.
6.2.4 Recall over translation
Recall over translation has again been analysed using the castle theme. SURF and
BRIEF turn out to have the best results for the three quality steps. The pixel patch
is getting considerably poor results. This is due to the fact that the patch descriptor
is not very distinctive and suers strongly from little changes in the image, espe-
cially from the JPG compression pattern as observable in the gure for 0% quality.
But nevertheless, up to a quality of 70% the patch reaches not too bad results with
a recall of 88%. How important the threshold is can be observed in the gures for
SURF and BRIEF. Even for a quality of 0% the recall is between 0.9 and 1, which
gives reason to state that a threshold is a appropriate way to distinguish correct
and incorrect matches. The BRIEF and the SURF descriptor are very distinctive.
Again the pattern eect can be observed by the curves returning to 1 after a trans-
lation of 8 pixels.
Castle Theme 100% Castle Theme 70% Castle Theme 0%
Figure 6.7: Recall over translation for the castle theme.
Chapter 7
Conclusion
The conclusion can be split up into a program and an evaluation part.
7.1 Program
A considerable amount of time was invested into the evaluation program to make
it generic. The program works awless and the code-structure supports further
implementations:
The program can be extended by new feature extractors or descriptors.
A dierent compression method can be implemented.
More image themes can be included.
Further evaluations can be implemented into the existing structure.
Since the program is split up into two classes, the data generation can be used
without the evaluation.
Since the BRIEF implementation could not be found online, it was self implemented
and it is now free to be used for any further projects.
7.2 Evaluations
7.2.1 Detectors
SURF has a bad repeatability against translation. It suers more from translation
than from compression. Therefore the image information content is getting really
poor for realistic scenes with images or with the camera in motion.
FAST gets good results over translation and compression. It can be stated that
FAST is robust against translation and compression above a certain compression
ratio (20% in terms of quality).
7.2.2 Descriptors
Even if the SURF detector did not have a good repeatability over translation, its
descriptor worked well for the recall. The loss of features over translation is still
bad. With the loss in features the information content of the image gets low and
therefore SURF is not an appropriate feature for JPG compression. This is in agree-
ment with the experience from working with SURF in former projects.
BRIEF showed good results through repeatability and recall. It is quite robust
17
Chapter 7. Conclusion 18
against the JPG compression pattern in translation as long as the quality stays
above 20%.
The pixel patch is not a highly sophisticated feature, its descriptor is not distinctive
enough to catch up with BRIEF and SURF, which was the expected result.
7.3 Outlook
The framework is ready to be used and it can be extended by new feature detectors
and descriptors. It can also be used as a base for another thesis which goes for
example deeper into image compression.
The observed behaviour of SURF against image compression and translation can
be taken into account in future projects.
Appendix A
Feature properties
Table of feature detectors properties from [1].
19
7.2 Summary on the Detectors 257
T
a
b
l
e
7
.
1
O
v
e
r
v
i
e
w
o
f
f
e
a
t
u
r
e
d
e
t
e
c
t
o
r
s
.
R
o
t
a
t
i
o
n
S
c
a
l
e
A

n
e
L
o
c
a
l
i
z
a
t
i
o
n
F
e
a
t
u
r
e
D
e
t
e
c
t
o
r
C
o
r
n
e
r
B
l
o
b
R
e
g
i
o
n
i
n
v
a
r
i
a
n
t
i
n
v
a
r
i
a
n
t
i
n
v
a
r
i
a
n
t
R
e
p
e
a
t
a
b
i
l
i
t
y
a
c
c
u
r
a
c
y
R
o
b
u
s
t
n
e
s
s
E

c
i
e
n
c
y
H
a
r
r
i
s

+
+
+
+
+
+
+
+
+
+
+
H
e
s
s
i
a
n

+
+
+
+
+
+
+
S
U
S
A
N

+
+
+
+
+
+
+
+
+
H
a
r
r
i
s
-
L
a
p
l
a
c
e

+
+
+
+
+
+
+
+
+
H
e
s
s
i
a
n
-
L
a
p
l
a
c
e
(

+
+
+
+
+
+
+
+
+
+
D
o
G
(

+
+
+
+
+
+
+
+
S
U
R
F
(

+
+
+
+
+
+
+
+
+
H
a
r
r
i
s
-
A

n
e

+
+
+
+
+
+
+
+
+
+
H
e
s
s
i
a
n
-
A

n
e
(

+
+
+
+
+
+
+
+
+
+
+
S
a
l
i
e
n
t
R
e
g
i
o
n
s
(

)
+
+
+
+
+
E
d
g
e
-
b
a
s
e
d

+
+
+
+
+
+
+
+
M
S
E
R

+
+
+
+
+
+
+
+
+
+
+
I
n
t
e
n
s
i
t
y
-
b
a
s
e
d

+
+
+
+
+
+
+
+
S
u
p
e
r
p
i
x
e
l
s

)
(

)
+
+
+
+
Appendix B
Framework
B.1 Variable list
21
Generic - Datastructure
class im_data
Properties: Data Type: Description:
im_array{run,quality}. cell-array run: translation steps, quality: quality steps
name 6 char theme name
quality num quality between 0-100%
comp 3 char compressen method, i.e. "jpg"
data nxmx3 num rgb image data
grey nxm num grey image data
trans 1x2 num translation in [x,y]
feature. structure
name char Featurename
data Diverse What the implemented feature func. returns
coor nx2 num [x,y]xn, list of coordinates for the features
descriptor nxm num Collection of descriptors (nx1) for the #features m
descriptor2 nxz num like descriptor, but for special pattern
coor2 zx2 num [x,y] coordinates for features in descriptor2
radius num Feature extractor radius -> used for overlap 50%
class eval_data
Properties: Data Type: Description:
eval_array{eval_id}. cell-array eval-id: addresses a successful evaluation
name char evaluation name
groundtruth 1x2 num run and quality for gt-im in im_array
x 1xn num x-axis for evaluation plot
y 1xn num y-axis for evaluation plot
option num 1: over quality, 2: over translation
im(im_nr). structure of the involved images
coor num pairs: difference of coordinates, location match:
coordinate [x,y]xn for every feature
radius 1xn num translation information
phase 1xn num translation information
im_array 1x2 num run and quality for im(im_nr) in im_array
ind_gt 1xv num inliers after pair match in compressed image.
ind_im 1xv num inliers after pair match in gt image
y_p 1xv num y-axis for history plot of pair phase
y_r 1xv num y-axis for history plot of pair radius
y_std 1xv num y-axis for standard deviation
err 1xn num distance error for threshold compairision
threshold num the set threshold
Bibliography
[1] Tinne Tuylelaars, Krystian Mikolajczyk: Local Invariant Feature De-
tectors: A Survey. Foundations and Trends in Computer Graphics and Vision,
Vol. 3, No. 3 (2007) 177-280
[2] Krystian Mikolajczyk: A Comparison of Ane Region Detectors. Univer-
sity of Oxford, OX1 3PJ, Oxford, United Kingdom, 2006.
[3] Edward Rosten: FAST Corner Detection Homepage: http://svr-
www.eng.cam.ac.uk/ er258/work/fast.html, Version of 23.02.2011
[4] E. Rosten and T. Drummond: Fusing points and lines for high performance
tracking. In Proceedings of the International Conference on Computer Vision,
pp. 1508-1511, 2005.
[5] E. Rosten and T. Drummond: Machine learning for high-speed corner de-
tection. In Proceedings of the European Conference on Computer Vision, pp.
430-443, 2006.
[6] Coots, T.F., Taylor, C., eds: Performenace Evaluation of Corner De-
tection Algorithms under Ane and Similarity Transforms. In Cootes, T.F.,
Taylor, C., eds.: 12th British Machine Vision Conference, Manchester, British
Machine Vision Assosciation (2001).
[7] Visual Geometry Group, Research Ane Covariant Features.
Homepage: http://www.robots.ox.ac.uk/vgg/research/ane/, Version of
13.03.2011
[8] Dirk-Jan Kroon OpenSURF(including Image Wrap). SURF algorithm from
homepage: http://www.mathworks.com/matlabcentral/leexchange/28300,
Version of 18.03.2011
[9] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool SURF:
Speeded Up Robust Features. Computer Vision and Image Understanding
(CVIU), Vol. 110, No. 3, pp. 346359, 2008
[10] OpenCV: test pairs.txt revision 3931. Homepage:
https://code.ros.org/trac/opencv/browser/trunk/opencv/modules/features2d/
src/test pairs.txt?rev=3931
23