Feature Detection

March 28th, 2016

Jonathan Samuel Lumentut

What is Local Feature?

2
 What is Local Feature?
*Image pattern which differs from its immediate
neighborhood

*Tinne Tuytelaars and Krystian Mikolajczyk. "Local invariant feature detectors: a survey." Foundations and Trends in
Computer Graphics and Vision , vol. 3, no. 3, pp. 177-280, January 2008.

3
Intuition : Object Matching

4
Challenge : Diverse Scene Variation

Photometric Transform Rotation & Scaling

Original Image

Projective Effect
Skew
Anisotropic Scaling
5
Level of Invariance

Feature detection should be invariant to class of

transformation :
Rotation

Scale

Affine

6
Feature Detection Properties

Repeatable
Same features from different images shows the same object

Distinctive
Rich variation in intensity pattern so that different image
structures can be told apart from each other

Locality
Detected features should be local, which is less sensitive to
occlusion and viewpoint changes
7
Feature Detection Properties

Quantity
Numbers of detected features should be sufficiently large

Accuracy
Features should be accurately localized, both in location,
scale and shape

Efficiency
Detection time should be reasonable, preferably allowing
time-critical application
8
Example of Local Features

Edge

Corner

Blob
a) Edge feature b) Corner feature
Affine covariant
region

c) Blob feature d) Region feature

http://kr.mathworks.com/products/computer-vision/features.html

9
 Rotation Invariant Descriptor

Find dominant orientation of gradient of feature

Build histogram of gradient angle, choose the peak

Rotate patch according to the dominant angle

Support area for computing descriptor will be rotation

invariant

Credits : Lazebnik, M.Brown

10
Affine Covariant Feature Description

Affine region normalization

Map all region to a patch with constant radius (for scale
invariance)
Rotate the ellipses main axis to horizontal
Scale horizontally so the area becomes a circle

Credits : T. Tuytelaars

11
Distribution Based Descriptor

Captures substantial amount of information about

spatial intensity pattern

Robust to small deformation & localization errors

Histogram may built upon :

Gradient orientation

Pixel intensities

Edge points, etc

Histogram of Integral Image response, SURF descriptor

12
Binary Descriptor

Inspired by Census Transform, robust to illumination

changes

Drawbacks of distribution based descriptor:

High dimensionality
Time complexity
Memory consumption

General Equation

13
Binary Descriptor
Binary descriptors varying in terms of
Orientation compensation
Sampling pattern
Sampling pairs
Matching:
Hamming distance (SUM of XOR) instead of Euclidean
Pros
Very Fast and low memory consumption
Robust for illumination changes
Cons
Less distinctive
14
Local Feature Descriptors

1. SIFT
2. SURF
3. FAST
4. BRIEF
5. LIOP

15
SIFT Scale Space Detection
Simulating scale changes

Build image pyramid

Use scale space function

Incrementally convolve image with Gaussian

separated by constant factor k

16
SIFT Scale Space Detection

Obtain LoG approximation efficiently using DoG

Substract neighboring layer to obtain DoG

17
SIFT Scale Space Detection

Find extrema by 3D nonmax surpression

Compare a pixel with 26 pixels in current and

adjacent scales

Select as as keypoint (extrema) if larger or

smaller than all 26 pixels

More extrema, more computationally

expensive

18
SIFT - Orientation Check

Using gradient magnitude and direction of each

keypoints.

Create weighted histogram in a neighborhood of

keypoint

Find the highest peak

19
SIFT
Result of Detected SIFT Features

20
SURF

Use determinant of Hessian matrix to detect blob like

structure (called Fast Hessian)

Approximate directional LoG with box filter for efficiency

Fast computation with Integral Images

21
SURF

Increasing filter size instead of downsampling the image

Made possible and faster by integral images and box filter

structure

22
SURF

An octave respresent series of

filter response maps by
convolving with filter of
increasing size.

Filter responses are normalized

according to the size (scale
normalization).

Faster and no aliasing

23
SURF

Orientation assignment using wavelet responses in

horizontal and vertical direction for neighborhood of
size 6s. Dominant orientation estimated by sum of all
responses within a sliding orientation window.

24
SURF

Feature descriptions: using wavelet responses in

horizontal and vertical direction, each neighbourhood
of size 20x20 taken around keypoint divided into 4x4
subregions. Vector from each subregions is formed like:
= ( , , || , ||)

Left: homogeneous, region, values are low

Middle: frequencies in x direction make value of || high

Right: Intensity that is gradually increasing in x direction makes

and || are high

25
SURF
Result of Detected SURF Features

26
FAST

Mostly used for corner detector.

Looking for corners by considering a circle of 16 pixels around

the corner candidate p.

p is a corner if there exists a set of n contiguous pixels in the

circle of 16 pixels which are all brighter than (Ip + th) or (Ip th)
*th=threshold

27
FAST

Examine four pixels at 1,5,9,13 locations to exclude a large

number of non-corners. If 1(north) and 9(south) are too
brighter or darker, then checks 5(east) and 13(west).

Each pixel in 16 pixels neighborhoods can have the following

three states:

d,
Sp->x = , < < +
, +

p is considered to be a corner if at least three of these must

higher than (Ip + th) or lower than (Ip th).

28
FAST Non Maxima Suppression
Detected points may be adjacent one to another

There is no corner response value, therefore

compute score function V instead

Some possibility of V :
1. Maximum value of n for which p is still corner

2. Maximum value of t for which p is still corner

3. The sum of the absolute difference between the pixels in

the contiguous arc and the center pixel
29
FAST Non Maxima Suppression

Compare two detected adjacent corners, compare its

V values

Discard the one with lower V value

For computation efficiency the third V score is used :

30
FAST

31
BRIEF
Binary distributions are used as feature point descriptor since
their dimensionality can be reduced.
BRIEF method requires a pre-smoothed image as initiation.
Select a set (Nd(x,y)) of locations pairs randomly and store it in x
and y vectors from smoothed image patch.
Binary comparison using pixel intensity is done on these set of
locations.
Ex. A pair of match pixels in Nd, p and q are compared ((p,q) Nd).
If I(p) < I(q), result is 1, otherwise 0.
Dimension of locations pairs can be 128, 256, 512.

32
BRIEF Sampling Pairs

Different approaches to
choosing test locations:
GI: Locations of p and q are distributed over the
patch

GII: p and q are chosen from an isotropic Gaussian

distribution.

GIII: p is chosen from a Gaussian centered around the

origin and q from Gaussian centered around xi.

GIV: p and q are randomly sampled from discrete

locations of coarse polar grid

GV: p and q takes all possible values on coarse polar

grid.

33
LIOP

Encode local ordinal information of each pixel

Overall intensity order is used to divide the local patch

into subregions called ordinal bins.

LIOP of each point is defined based on the relationships

among the intensities of its neighboring sample points.

34
LIOP
Firstly, smooth image with Gaussian
filter to reduce noise.

Normalized to circular regions with

fixed parameter.

Quantize each local regions in

ascending order into ordinal bins.

Using affine covariant region,

localize feature position and
estimate the affine shape including
its neighborhood.

35
LIOP
Let P be the vectors of all possible permutation in N dimension.
= {(, , , )}

The mapping is defined to map PN to a permutation .

The mapping sorts N elements of P in a non-descending order

The vectors in the same partition have the same order relationship.

Map each partition into index table for all possible permutation

LIOP (x) = = (0,,0, 1 ,0,0)

where P(x) = (I(x1),I(x2),,I(xN))

To obtain rotation invariant sampling, first point is sampled along radial direction and
the rest N-1 is sampled in anti-clockwise direction.

Each transformed pixels classified in same bins by the intensity order and rotation
invariant sampling.
36
LIOP

The overview of the LIOP workflow can be seen

in this figure:

37
Conclusion
Detector descriptor combination obtain different result

Features from different detectors complement each other

Generally, different descriptors use different image properties (gradient,

intensity, edge, etc)

The choice of detector and descriptor should be adjusted with the scene content
(structured, textured, etc)

Combination of multiple detectors could be applied to reduce the dependency of

scene content

Distribution based descriptor usually obtain better performance

Binary descriptor for fast computation.

38
References
Tuytelaars, T., & Mikolajczyk, K. (2008). Local invariant feature detectors: a survey. Foundations and
Trends in Computer Graphics and Vision, 3(3), 177-280.
Lowe, D. G. (1999). Object recognition from local scale-invariant features. InComputer vision, 1999. The
proceedings of the seventh IEEE international conference on (Vol. 2, pp. 1150-1157). Ieee.
Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In Computer visionECCV
2006 (pp. 404-417). Springer Berlin Heidelberg.
Rosten, E., & Drummond, T. (2006). Machine learning for high-speed corner detection. In Computer
VisionECCV 2006 (pp. 430-443). Springer Berlin Heidelberg.
Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). Brief: Binary robust independent elementary
features. Computer VisionECCV 2010, 778-792.
Wang, Z., Fan, B., & Wu, F. (2011, November). Local intensity order pattern for feature description.
In Computer Vision (ICCV), 2011 IEEE International Conference on (pp. 603-610). IEEE.
http://docs.opencv.org/3.0-
beta/doc/py_tutorials/py_feature2d/py_features_meaning/py_features_meaning.html#features-
meaning

39