Robust Audio Fingerprint Extraction Algorithm Based on 2-D Chroma
Hongxue Wang, Xiaoqing Yu*, Wanggen Wan, Ram Swaminathan
Shool of Communication and Information Engineering
Institute of Smart City, Shanghai University
Shanghai HanPan Information S&T Ltd.
Shanghai 200072, China
Abstract
Audio fingerprinting, like human fingerprint,
identifies audio clips from a large number of databases
successfully, even when the audio signals are slightly
or seriously distorted. In the paper, based on 2-D, we
propose an improved audio fingerprinting extraction
algorithm which was proposed by Shazam company.
The algorithm uses a combinatorial hashed time-
frequency analysis of the audio, yielding unusual
properties in which multiple tracks mixed together may
each be identified. The results of experiment verify the
improvement in the retrieval speed and accuracy.
1. Introduction
Fingerprint systems are over one hundred years old.
With the development of digital media technology,
people are routinely exposed to music in everyday
environments but are frustrated by not being able to
learn more about what they hear. They may, for
instance, be interested in a particular piece of music
and want to know its title and the name of the artist
who created it. These problems promoted the
development of a query-by-example (QBE) music
search service that enables users to learn the identity of
audible prerecorded music by sampling a few seconds
of audio using a recording device [1].
Audio fingerprinting system has the following
characteristics [2]:
a) Robustness: the audio signals which suffer serious
noise or frequency domain distortion can be
accurately identified. To a certain extent, audio
fingerprinting carries out the invariance of the
signal processing in order to get the robustness of
audio.
*Corresponding Author: Xiaoqing Yu, Shanghai University
Email Address: yxq@staff.shu.edu.cn
b) Granularity: It means the length of the query.
Obviously the shorter the granularity, the better
the system performance. But too short granularity
978-1-4673-0174-9/12/$31.00 ©2012 IEEE
will impact the retrieval accuracy.
c) Retrieval speed: With the quick growing of audio
database, an audio query to be retrieved can take a
long time. In particular, commercial audio
retrieval systems require a relatively high retrieval
speed. So the design of audio fingerprint and
database can reduce the retrieval time, accordingly
achieve the actual requirements.
Above these parameters have certain relevance; a
parameter change will affect the performance of
another parameter. Therefore, the value of the
parameters should be set according to the actual
application.
Audio fingerprinting system consists of two main
parts: audio fingerprint extraction algorithm and audio
search algorithm. Audio fingerprint which can stand
for the characteristics of original audio is extracted
from audio signal, then the calculation of perception
hash to generate a highly robust audio fingerprint;
In the paper, Section 2 introduces some previous
research works on audio fingerprint. Section 3
introduces our proposed audio fingerprinting
algorithms. Some preliminary experimental results are
presented in Section 5. Finally in Section 6,
conclusions and suggestions for future work are
collected.
2. Review previous audio fingerprint
algorithm
Up to now, many researchers have started
researching the field of audio and have developed the
audio fingerprint algorithm which some algorithm has
been commercialized. In 2004, the Gracenote Inc. and
the Royal Philips Research institute developed the
music recognition software“Gracenote Mobile” [3]
which can be used through mobile phone.
The famous Philips’ AF algorithm [4] proposed by
Haitsma and Kalker has widely applied. The audio
signal sampled by 44100HZ is segmented into
Hanning windowed overlapping framed at an overlap
factor of 31/32.After FFT of per frame; the audio
763 ICALIP2012
spectrum is divided into 33frequency bands which are
used to obtain 32 hash bits. Actually it’s obvious that
the internal energy difference structure of the audio
signal is relatively invariant to many types of noises
and distortions.
Another important algorithm is proposed by
Shazam Entertainment,Ltd[5] in UK. The audio signal
is transformed into a 2-D image[6] by a series of
transformation which can find the maximum energy of
the peak point in the temporal-frequency domain, then
is generated the audio fingerprint by composing a pair
of features. This algorithm does not need to retain the
global information of entire spectrum; it only extracts
part of the significant spectral component, so the
algorithm has the advantages of anti-noise
performance, fingerprint with high robustness.

3. Proposed audio fingerprinting algorithm

Figure 1. Flowchart of proposed extraction
Audio retrieval system contains two parts: scheme
fingerprint extraction and fingerprint matching
the signal-to-noise radio (SNR) of 5dB (b). It can be
algorithm.
seen that after adding noise the original signal figure is
Now audio fingerprint technology is quite mature,
not smooth, so the original signal should be processing
but in the strong noise high robustness of the
before extracting audio feature in order to make the
fingerprint extraction is still a concern. The proposed
original signal smoother and extract high robust
algorithm converts original signal to 2-D image
fingerprint. So the paper takes the 8 kHz sampling rate
analysis using local maximum chroma energy (LMCE)
to down-sampling, then window function with time
as feature points, then generate a high robust audio
length 64ms is used, finally carry through framing with
fingerprint through fingerprint modeling. We’ll
the overlapping 60% which is defined in Equation (2).
demonstrate this in details later in the next section. The
extracted fingerprint uses the hash algorithm to build S ' (k , j ) = s ( K * Ls + j )
index table, then find the most similar reference (2)
fingerprints by matching algorithm, finally the system
k = 0,1,2" N ,1 ≤ j ≤ 512
will output corresponding audio metadata.
It is well known that the Human Auditory System
(HAS) [7]is sensitive to the value of the spectrum. So
using FE which is defined as in Equation (1) to
measure the perceptual similarity of two audio signals.

w0
FE = log(∫ X ( w) dw)
2
(1) (a)
0
Where X(w) is the Fourier coefficients, is half
sampling rate.
The flowchart of extracting audio fingerprint in
Fig.1.

3.1. Frequency energy

First stage of the algorithm is to segment the audio
signal into overlapping frames; after FFT and High-
pass filter, calculate the chroma energy. Figure2 shows
the waveform of original audio (a) and the signal with
(b)
Figure 2. Waveform of audio signal (original
and SNR=5)
Figure3 shows the image of Tempo-Frequency,
which are derived from the upper Figure2. From the
figure we can see that chroma maximum has higher

764
energy which cannot be varied under noise and other
forms of distortion. Using FFT to convert the signal to
frequency domain S(k,j)=FFT(s(k,j)), then can get
local energy aggregation properties after weighted-
processing, referred in Equation (3).
i+M i+ N
S (i, j ) = ∑∑ C (m, n) * S (m, n)
i−M i− N
(3)
Where C(m,n) for 2-D Gaussian window function,
M and N represent the length of neighborhood, the
paper takes M=N=2.
(c)
(d)
Figure 3. The image of Tempo-Frequency
(original and SNR=5)
3.2. Audio fingerprint’s generation
This paper uses LMCE to extract the perception
features of Tempo-Frequency domain. In the
experiment less than five peaks are limited to each
frame.Fig.4 is the feature distribution of pure signal;
Fig.5 is the feature distribution of audio signal with
765
SNR=5; from the figure, redundancy can be generated
which interfered by noise. So this paper introduces the
threshold comparison method to find peaks in order to
make sure the sparse uniform characteristics of the
peaks. The update of threshold in Equation (4).
Thres(i, j ) = Thres(i, j ),Thres(i, j ) > S (k , n) *W ( j )
Thres(i, j ) = S ( k , n) * W ( j ), Thres(i, j ) < S ( k , n) * W ( j ) (4)
Where W ( j) is Gaussian window function,
− ( j − n )2
W ( j) = e 2
Figure 4. feature distribution of pure signal
Figure 5. feature distribution of audio signal
with SNR=5
Figure6 is feature distribution which is preserved
with high robustness under strong noise; from the
entire spectrum range, the distribution of feature points
is more even and distribute the entire spectrum
sparsely, so the extracted feature own high robust and
compactness.
 Table 1. Results of comparing Shazam with
Proposed
matching Start Value
approach ID
Number frame i=1
Shazam
481 11 3357 0
approach
Proposed
481 48 3358 -1
approach

Figure 6. feature distribution with high 5. Experimental results

robustness
The simulations are made using a music database of
After peaks are extracted, audio fingerprint is 1000songs. Including rock, pop, classical, jazz and
generated by the combination of characteristic country music, 20 music clips with the length of 5
parameters in order to improve the accuracy and seconds are randomly chosen from the database. To
retrieval speed. Select each feature as anchor then assess the robustness of the algorithm the following
paired to generate the fingerprint. distortions are applied to queries.
Set 1: Additive white noise with the different SNR at
4. Audio search algorithm 10dB, 5dB, 0dB.
Set 2: Recoding in lab, office, outdoor environment.
It is possible to perform similarity searches from Set 3: the query clip with 5s, 10s, and 15s.
large database, but two similar sound events,
containing similar or identical spectral peaks at the Table 2. the accuracy with different
same relative timing, will generate many common experiments
hashes, only enough common hashes are generated to length pure 10dB 5 dB 0 dB
differentiate with chance hash collisions. A statistical
15s 99% 97% 95% 92.5%
threshold can be set to identify the peak and detect a
match. In the case that the change of linear speed, the 10s 99% 95.5% 92% 89%
frequency components in the tempo-frequency is
essentially invariable, only the time offset changes; 5s 98.6% 93% 88% 80%
Original algorithm may lead to low accuracy rate, so in
order to reduce distortion, a tolerable range is set to the From the table2, noise intensity and time length of
time offset which referred in Equation(5): query will have different fluencies on the accuracy rate
of the retrieval system. The longer query clips, the
Hash = [ f 1, Δf , Δt + i ] = [ f 1, Δf , Δt ] (5) better accuracy rate.
Where f1 is start frequency; is the frequency
6. Conclusion
change of between two features; is the time change.
Under different noise environments, i can be amended In this letter, we introduce a robust audio
to increase the recognition rate, we take i=-1, 0, 1. fingerprinting algorithm. The audio fingerprints are
From the table1, in the case of distortion, the produced based on 2-Dimagel, which is expected to be
proposed algorithm has the better distinguish and highly robust to noises and distortions. In the retrieval
higher accuracy. process, inverted file index technique combined with a
special-designed hashing function is adopted to reduce
the retrieval time. Preliminary experimental results
suggest that the proposed audio fingerprinting
algorithm can work well in broadcast monitoring
application with good discrimination and recognition
accuracy.

766
 For the present the proposed scheme has not yet
be evaluated in a variety of noises and degradations. It
is possible that the minimal bit error rate will be raised
significantly and the recognition accuracy will decline
correspondingly. In the future we’ll focus on testing
using degraded audios samples and improve the
proposed algorithm accordingly.

7. Acknowledgements
This work was supported in part by Shanghai’s Key
Discipline Development Program under Grant No.
J50104. It was also supported in part by Datasentric
Inc, 38660 Lexington Street No.440 Fremont, CA,
USA.

References
[1] Avery Wang. “The Shazam Music Recognition Service”.
Communications of the ACM, Vol.49, No.8, August 2006.
[2] J.T.Foote, “An Overview of Audio Information
Retrieval”,ACM-SpringerMultimediaSystem,Vol.7,no.1,pp.2-
11,ACMPress/Springer-Verlag,Jan.1999.
[3] Yuan-Yuan Shi, Xuan Zhu, Hyoung-Gook Kim, Ki-
Wan Eom. A Robust Music Retrieval System.AES 120th
Convention, Paris, France, pp.20-30, 2006.
[4] J. Haitsma and T. Kalker. “A Highly Robust Audio
Fingerpriting System”, Proc. 3rd ISMIR, pp. 144-148,
2002.
[5] A.Wang. “An industrial strength audio search algorithm”,
Proc. 4th ISMIR, pp. 7–13, 2003.
[6] B.Zhu,W Li, “A Novel Audio Fingerprinting Method
Robust to Time Scale Modification and Pitch Shifting” Proc.
10thMM, pp. 25–29, 2010.
[7] Lotter, T., Vary, P. “Speech enhancement by MAP
spectral amplitude estimation using a super-Gaussian speech
model”. EURASIPJ. Appl. Signal Proc. 7, pp.1110–1126,
2005.

767