MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 36
ACCEPTED FROM OPEN CALL
DESIGN OF TRAFFIC SIGN DETECTION,
RECOGNITION, AND TRANSMISSION SYSTEMS FOR
SMART VEHICLES
ABDELHAMID MAMMERI AND AZZEDINE BOUKERCHE, UNIVERSITY OF OTTAWA
MOHAMMED ALMULLA, KUWAIT UNIVERSITY
ABSTRACT
Traffic sign detection and recognition
(TSDR) is an essential component of advanced
driver assistance systems (ADAS). It is mainly
designed to enhance driver safety through the
fast acquisition and interpretation of traffic
signs. However, such systems still suffer from the
inability to accurately recognize signs. Moreover,
the sharing of wirelessly recognized signs among
vehicles is not yet supported by current systems.
In some safety scenarios, vehicle-to-vehicle com-
munication of traffic sign information is
required. In this article, we first address chal-
lenges and undesirable factors facing TSDR sys-
tems. After that, we show how to design a TSDR
system by addressing some useful techniques
used in each stage of the system. For each stage,
these techniques are regrouped into different
categories. Then, for each category, a short
description is given followed by some concluding
remarks. Finally, the transmission of the recog-
nized signs is briefly investigated.
INTRODUCTION
Advanced driver assistance systems (ADAS)
are intelligent systems designed mainly to
improve driving comfort and enhance road
safety by assisting drivers in their tasks. Some
examples of such systems include the pedestri-
an protection system, adaptive cruise control
system, and vehicular communication system.
One major component of ADAS is the traffic
sign detection and recognition (TSDR) system.
It is intended to increase driver and pedestrian
safety through the quick detection and interpre-
tation of signs. However, even with large num-
bers of published works and platforms, current
systems are still far from perfect. This is mainly
due to some challenges and issues. These chal-
lenges are various in number; we classify them
in different levels: low-to-medium camera reso-
lution; unsatisfactory road conditions; imperfect
sign state; varied surrounding conditions; and a
moving vibrating car. To design a robust TSDR
36 1536-1284/13/$25.00 © 2013 IEEE
system, the aforementioned issues have to be
accurately addressed.
In most published TSDR systems, a two-stage
sequential strategy has been adopted: the detec-
tion stage is followed by a recognition stage. In
the detection stage, a search for the regions of
interest (ROIs), which contain the traffic sign
within the image, is performed. These ROIs are
then examined by the recognition stage to cor-
rectly identify signs. We believe that a well
designed system involves, rather, three to four
stages. Each stage requires special consideration.
These stages are the following: preprocessing,
detection, recognition, and tracking.
The purpose of the pre-processing stage is to
enhance the input image in order to increase the
likelihood of the successful delivery of ROIs to
subsequent stages. It includes mainly image pro-
cessing and camera calibration [1]. To avoid
dense scanning of the whole image in this first
stage, a localization technique may be used to
select the areas that possibly contain signs. The
extraction of such an ROI is performed in the
second stage (i.e., detection). It is accomplished
by extracting certain features from the input
image like color, shape, or gradient information.
After that, the recognition stage receives a list of
ROIs that possibly contain one or more signs. In
this stage, the ROIs in question will either be
recognized as traffic signs or they will not; and
they will be furthermore classified according to
their categories (stop signs, speed limit signs,
etc.). This will be carried out using the knowl-
edge obtained from a set of training samples.
Finally, to enable the following of signs over
time, a tracking stage was included in our strate-
gy. Compared to the individual frame-based sys-
tems, this stage has several advantages including
the ability to decrease false detections over
times, and to predict future sign positioning in
the image; thus, this stage feeds the detection
stage with updated candidates. Consequently,
the most common trackers used in TSDR sys-
tems are the Kalman filter (KF), discussed in [2],
and the particle filter (PF), which is based on
Bayesian inference, adopted in [3].
IEEE Wireless Communications • December 2013
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 37
In almost all published TSDR systems, the
recognized sign is used by one driver without
being shared between neighboring vehicles. In
some safety scenarios, the intervehicle com-
munication of sign information is required to
prevent approaching drivers from a possible
crash. To the best of our knowledge, such sys-
tems for the detection, recognition, and trans-
mission of sign information do not exist. This
is due to the fact that the existing technology
has not yet reached the level of maturity need-
ed to perform inter vehicle communication
perfectly. We expect in the near future that a
new generation of vehicles will be equipped
with onboard units based on the standard
IEEE 802.11 interfaces and antennas. In fact,
the development of IEEE 802.11p and the
frequency band allocations in the 5.8–5.9 GHz
bands for public safety services illustrate the
next step forward in the deployment of net-
works of vehicles.
In this article, we show how a TSDR system
should be designed. We point out some appro-
priate techniques used by the four stages of this
system. For each stage, these techniques are
classified into different categories, as shown in
Table 1. Furthermore, for each category, a short
description is given followed by a brief example
from the literature in this field and some useful
remarks. Finally, the transmission of recognized
signs is briefly investigated. This is a survey type
of article on an important topic that is currently
receiving a great deal of attention. This article is
meant to provide design guidelines, which shall
supply readers with useful techniques at each
stage for the building of a well structured TSDR
system.
CHALLENGES
Several undesirable factors may influence the
design of an ideal TSDR. We classify them in
five different categories: a low-acquisition vision
system; an unsatisfactory road condition; a
changeable surrounding condition; a moving
vibrating car; and an imperfect sign state.
ACQUISITION VISION SYSTEM
When a traffic sign is far away from a camera,
its ideogram appears small in the image. The
recognition of such signs with a low-resolution
camera is a challenging problem for moving
vehicles. This is due to the fact that signs not
only appear small, but noisy and blurry in the
image. One way to handle this problem is to use
a high resolution camera. Many researchers use
a standard camera with a resolution that is usu-
ally at most 640 ¥ 480 pixels. Unfortunately, this
resolution seems insufficient, especially when
signs are far away from vehicles.
ROAD CONDITIONS
Camera vibrations and oscillations are an issue
for moving cars. In fact, the mounted TSDR sys-
tem suffers from car movements caused by road
roughness. The performance of TSDR systems
can be improved if the vibrations introduced by
a camera are corrected by an image stabilization
system. One way to decrease camera vibrations
is to exploit the movement of lenses to achieve
IEEE Wireless Communications • December 2013
Stages Techniques Examples
Preprocessing Camera calibration [1]
Color thresholding
Color [4–6]
Color learning
Edge extraction
Detection Template matching
Shape [4, 7–12, 15]
Machine learning
Gradient HOG
Hybrid methods [2]
Particle-filter-based technique [3]
Tracking
Kalman-filter-based technique [2]
Template matching techniques [11]
Recognition
Machine learning techniques [5]
Table 1. Some techniques used by each stage of TSDR systems.
an adequate level of stabilization, or through the
use of an electromechanical stabilization plat-
form. Another possibility is through image stabi-
lization algorithms. To the best of our
knowledge, almost no TSDR systems include a
stabilization module.
SURROUNDING CONDITIONS
Surrounding conditions concern mainly the fol-
lowing factors: lighting, occlusions, and objects
similar to signs. The degree of light intensity
influences detection quality. This quality depends
mainly on weather conditions and those found
through the day where lighting conditions differ
from night to day. Surrounding objects, such as
buildings or trees, may directly affect (by occlu-
sion) or indirectly (by their shadow) the quality
of detection. Moreover, occlusion could be par-
tial or total. In the case of total occlusion, all
detection algorithms fail.
MOVING CAR
As images of signs are acquired from moving
vehicles, sign detection suffers from car vibra-
tions and motion blur. Motion blur is defined as
the visible streaking of rapidly moving signs in
frames. It results from a change in the recorded
image during the recording process of a single
frame due to fast movement. Moreover, car
vibrations are mainly due to unsatisfactory road
conditions.
TRAFFIC SIGN STATE
The physical degradation of signs can be classi-
fied into two categories: nature-made and man-
made. Nature-made degradation is due to
ultra-violet radiation and retro-reflective materi-
als, which mainly influence the color sign. Man-
made degradation can affect shapes, colors, and
parts of the sign or the entire sign.
37
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 38
STOP
STOP
Tracking
Preprocessing Detection Recognition Transmission
STOP STOP STOP
Figure 1. Traffic sign detection, recognition, and transmission system stages.
TSDR STAGES
A well designed TSDR system involves four
stages. These stages are pre-processing, detec-
tion, recognition and tracking. In the following
sections, each stage of the STDR is properly
addressed, and the most useful techniques are
presented, as demonstrated in Fig. 1 and Table 1.
PREPROCESSING TECHNIQUES
To design a robust TSDR system, a preprocess-
ing stage is required. The main advantage of this
stage is that it decreases the false detection ratio
and speeds up the detection process.
The fundamental function of the preprocess-
ing stage is to enhance the input image and
avoid dense scanning of it. This increases the
chance of a fast and successful delivery of ROIs
to the subsequent stages, which in turn mini-
mizes the number of false positive detections.
We believe that the preprocessing stage consists
mainly of operations related to the vision system
(camera) and the scene taken by the camera
(i.e., the output image/video). In fact, the camera
should be calibrated and some low-level tuning
such as dynamic range adjustments have to be
performed.
Camera calibration refers to the process of
finding extrinsic and intrinsic parameters that
produce a given image or video [1]. Only a few
works have been devoted to the calibration of
cameras. Most of them begin by computing the
intrinsic parameters, which are assumed to be
constant, while the extrinsic parameters are con-
tinuously updated [1].
Dynamic range adjustment (DRA) refers to the
ratio between the largest and the smallest measur-
able light intensities. Images usually necessitate
some adaptive dynamic ranges to increase their
contrast. Recently, high dynamic range (HDR)
image-based systems have begun to expand into
the field of driver assistance technologies since
they have commenced to provide high contrast
images even at night. An HDR camera should be
tested for sign detection application due to the
good quality of image it delivers.
Besides camera calibration, reduction of the
search area inside the image helps in the process
of building a fast detection system. This is due to
the fact that dense scanning of the whole image
wastes time. To avoid dense scanning of the
image, a localization technique can be used to
38
select areas that possibly contain signs. For
instance, known positions of the road in the
respective image help reduce the search space
for signs, and horizontal edges of buildings with-
in the target image guide us to focus our search
on some regions instead of the entire image.
To the best of our knowledge, the majority of
existing TSDR systems do not explicitly include
such a camera calibration step. Research work in
this area is required.
DETECTION TECHNIQUES
The main task of this stage is the generation of
candidates. This stage extracts ROIs from the
image and sends them to the recognition stage,
avoiding as many background regions as possi-
ble. The key to this stage is to avoid missing
signs as much as possible. Otherwise, the recog-
nition stage will not be able to correct errors. To
design a robust detection system, many
approaches exist. Most of these approaches
achieve detection through color, shape, and gra-
dient, or by also basing detection systems on
hybrid algorithms, as demonstrated in Table 1.
DETECTION THROUGH COLOR
Color-based detection methods aim to segment
the colors of signs in order to provide ROIs for
the recognition stage. Color segmentation con-
sists of partitioning an image into subsets of con-
nected pixels that share similar color properties.
Its aim is to intensify certain colors or edges,
and to suppress inappropriate areas. Different
color spaces are used to separate signs from the
background. The selection of the appropriate
color space is of high importance, since it affects
the detection result. Different color spaces are
used in the literature, such as RGB, normalized-
RGB, HSI/HSV, and YCbCr.
In a more recent study [4], the authors not
only made a comparison between color spaces
for the purpose of selecting those that are supe-
rior, but they also presented a quantitative com-
parison of several segmentation methods that
serve in sign detection. The color spaces identi-
fied as superior in [4] are normalized-RGB, fol-
lowed by HSI. Unfortunately, this study does not
consider other color spaces such as CIECAM97,
CIELab, YCbCr, and Luv.
After determining a suitable color space, a
search for signs within the input image was start-
ed using segmentation. Basically, two types of
color segmentation were reported, segmentation
based on color thresholding and segmentation
based on color learning.
Segmentation Based on Color Thresholding — Color
thresholding is the most intuitive segmentation
technique available when dealing with colors. In
fact, every pixel with a chrominance value above
a certain threshold is marked with an appropri-
ate label. After that, an ROI containing signs is
identified. Various authors have experimented
with color thresholding, especially in the 1990s.
The authors in [5] present some segmentation
algorithms based on color thresholding. Unfortu-
nately, when segmentation based on color
thresholding (SCT) methods were used, a num-
ber of thresholds had to be adjusted, and their
IEEE Wireless Communications • December 2013
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 39
adjustment depends only on the image undergo-
ing testing. Moreover, SCT techniques are sensi-
tive to lighting conditions, especially those
occurring on sunny days.
Segmentation Based on Color Learning (SCL) — The
disadvantages of SCT techniques can be alleviat-
ed by using a well-known machine learning tech-
niques such as the support vector machine
(SVM); this provides some improvements over
SCT methods [4]. An algorithm based on SVMs
was presented in [4] to classify pixels using color
information. Samples of the targeted color
required for detection and some other colors
from training images were labeled and used to
train the SVM. A lookup table (LUT), in which
the less significant bits were reduced, was used
to improve speed; this nevertheless also main-
tained quality. The authors show that segmenta-
tion based on SVM yields comparatively better
results than some SCT segmentation methods.
Neural networks (NNs) are also used to seg-
ment images based on their colors. In fact, in [6]
NNs are used in color training to recognize
signs. The basic idea is to segment the input
image by using an NN in eight colors. The ROIs
are then analyzed by a second NN to determine
if their shape corresponds to a TS. Unfortunate-
ly, NNs are slow, especially in the training phase.
Of the SCT and SCL methods used for detec-
tion, SVM outperforms SCT methods. Com-
pared to NNs, which have multiple solutions
associated with the local minima, the advantage
of SVM is that it provides a unique solution
since the optimality problem is convex [4]. How-
ever, the main problem of this technique is its
speed, which can be alleviated using the LUT
proposed in [4].
DETECTION THROUGH SHAPE
Besides color, shape can be effectively used to
detect signs, especially when color-based tech-
niques fail (e.g., changes in illumination). After
image preprocessing, shape detection is per-
formed. This is begun by extracting feature rep-
resentations of images for the generation of
maps with features, after which ROIs that possi-
bly contain signs are processed in the next stages.
From the literature reviewed, image features are
divided into two categories: local features such
as edges, and global features such as histograms
of oriented gradients (HOGs) [10]. The choice
of features is often coupled with the detection
method. Edges are found using some classical
edge detection techniques, such as the Canny fil-
ter [4]; these techniques are usually followed by
edge linking techniques (e.g., Hough Transform,
HT, matching) or distance transform matching
for the detection of TS. HOGs are found by cre-
ating histograms of gradients on images.
Shape is usually extracted by using perceiv-
able features (e.g., edges), gradient features
(e.g., HOG), and template matching (TM); it
may also extracted using machine learning based
techniques.
Shape Extraction through Edge Features — Most of the
shape detectors are based on two edge detec-
tion/linking techniques: the HT and Fast Radial
Symmetry (FRS) algorithms. These techniques
IEEE Wireless Communications • December 2013
are briefly explained in the following paragraphs. After image preprocess-
Earlier detection methods rely on edge segmen-
tation followed by linking to obtain a contour ing, shape detection is
representation, from which various parameters
such as perimeter and center are extracted. For
performed. This is begun
instance, the authors in [7] apply the Laplacian by extracting feature
filter to the input image to extract outlines and
edges. Although the application of these tech- representations of
niques is simple, they produce several ROIs, images for the genera-
which burden the subsequent stages.
The purpose of HT, the second technique, is tion of maps with fea-
to isolate features of a particular shape by a vot-
ing procedure. The classic HT technique is con-
tures, after which ROIs
cerned with the identification of regular curves that possibly contain
such as ellipses. The HT is later extended into a
generalized HT technique to deal with more signs are processed in
complex shapes. The HT technique is commonly the next stages.
used to detect the shape of signs. For instance,
the authors in [8] combine HT with an iterative
process of median filtering and dilation to refine
the sign candidate set. However, the HT tech-
nique remains computationally expensive, espe-
cially for large images.
The third technique, FRS, is a variant of the
circular HT technique. It was extended to traffic
sign application for the yielding of a regular
polygon detector in [9]. This method is based on
the symmetric nature of sign shapes and the pat-
tern of edge orientations exhibited by equiangu-
lar polygons. Compared to HT, this technique is
fast enough to produce robust systems. As it
detects shapes based on edges, the algorithm is
robust to illumination changes. However, this
method may fail when many edges are present in
images. For the non-frontal viewing of signs, the
correspondent image is usually transformed
through an accurate transformation (e.g., rota-
tion) before detection if the orientation of the
sign is a priori known; otherwise, the detection
fails.
A comparative study between different seg-
mentation techniques is presented in [4]; in this
work, it is pointed out that edge detection meth-
ods are not the most appropriate, except when
applied to achromatic decomposition (white
color).
Shape Extraction through Gradient Features — Colors,
edges, or corners are perceivable features usual-
ly extracted via image filtering or using learning
techniques. Recently, gradient information
(directions and magnitudes) have been success-
fully used to extract signs. This includes, among
other features, HOG [10]. Unlike the HT tech-
nique, this approach does not require prior
knowledge of the target.
The HOG detector, proposed initially for
pedestrian detection, starts by dividing the image
into a set of overlapping blocks. Each block is
then divided into non-overlapping cells, and for
each cell a histogram of the gradient orienta-
tions is performed. HOG has been successfully
applied to detect signs in several works (e.g.,
[16]). Different sizes of blocks, cells, and strides
of the HOG descriptor are used in [15] to evalu-
ate the performance of the classification stage. It
is reported that cells and blocks of small sizes
achieve higher recall and precision rates. This is
due to the finer spatial binning of the gradient
orientations. HOG has proven its efficiency over
39
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 40
HOG detector, proposed other algorithms because of the following desir-
able properties: invariance to scale, local con-
initially for pedestrian trast normalization, coarse spatial sampling, and
fine weighted gradient orientations [15].
detection, starts by
dividing the image into Shape Extraction through Template Matching — Shape
can also be extracted by using TM techniques.
a set of overlapping TM concerns the process of matching a template
blocks. Each block is to an image in the database. TM algorithms are
used to search through training samples and
then divided into non- determine the nearest neighbor of the test sam-
ple. However, the application of the TM
overlapping cells, and approach is unsuitable for real-time systems; the
for each cell a histogram reason for this is that the technique in question
requires many cross-correlation computations
of the gradient orienta- between the template and the ROIs for sign
tions is performed. HOG detection.
TM using distance transform (DT) is widely
has been successfully employed for shape detection. DT represents the
distance of each pixel to the nearest edge in the
applied to detect signs corresponding image edge. After edge detection,
in several works. a DT image is built. To find the shape of inter-
est, a template is matched (e.g., a regular circle)
against the DT image. Color-based DT was
recently introduced in [11], where a DT was sep-
arately computed for every color channel. It is
shown that shape detection through TM offers a
more suitable description of signs and entails
easy training. However, it is sensitive to occlu-
sions and rotations. The main advantage of using
TM is that it can be easily modified to recognize
other objects. This approach is applicable to
both detection and recognition stages.
Shape Extraction through Machine Learning Techniques —
Machine learning techniques are used success-
fully to extract shapes. NNs and SVMs are usu-
ally used for sign detection due to their efficiency
in detecting shapes. For instance, two NNs are
trained to extract the shape and color features of
signs in [2]. This approach is accurate but com-
putationally very expensive. However, the addi-
tion of signs to the NN implies a new training of
the network and the manual selection of the
training samples. The investigation of SVM for
shape detection is illustrated in [12], where the
system contains three phases: color segmenta-
tion, shape extraction using linear SVMs, and
recognition based on Gaussian-kernel SVMs.
The suggested method is invariant to rotation,
scale, and translation, and to partial occlusions.
HYBRID METHODS
Many detection approaches follow a two-step
sequential strategy. That is, color or shape
thresholding segmentation is first performed on
the input image to yield some ROIs. This step is
immediately followed by the sign detection stage
applied on the generated ROIs. Unfortunately,
ROIs that have been inaccurately rejected dur-
ing segmentation cannot be recovered for fur-
ther processing in the detection and recognition
stages. In this context, a parallel (joint) process-
ing of color and shape has thus been introduced
to overcome this drawback. For instance, the
authors in [2] have built a feature map of the
input image based on the color and model shape
of signs. It is observed that the false rejection
ratio is significantly lowered in comparison to
non-hybrid methods.
40
After ROI detection, object localization is
performed. This is accomplished by using some
methods like simple circle fitting operation on
circular signs or the bounded area space tech-
nique. With the first technique, once the center
of the ROI is localized, the sign is cropped and
cross-correlation template matching is applied to
find the best match in the sign database. With
the bounded area space technique, small objects
(e.g., red traffic lights) or large objects (e.g., red
cars) are excluded. For instance, the authors in
[5] use a limited area with a size that varies
between 200 and 5000 pixels.
Given the large number of papers found in the
literature on detection systems, it is difficult to
select one method among many as the best
option. Nevertheless, gradient feature-based algo-
rithms are obviously gaining comparative popular-
ity over other approaches, due to some desirable
properties such as scale invariance, local contrast
normalization, coarse spatial sampling, and fine
weighted gradient orientations [15]. Further
research is needed to improve the quality of
detection by reducing the number of features and
hence speeding up the whole process.
TRACKING TECHNIQUES
The most evolved TSDR systems use a tracking
stage to monitor signs over time. Compared to
individual frame-based systems, this stage has
several advantages, which include the following
attributes: the reduction of false detections over
time, increasing the robustness of the whole sys-
tem, and the prediction of future sign positions
in the image; this enables the reduction of the
search space and supplying of the detection
stage with updated candidates. The most com-
mon trackers used in TSDR systems are the
Kalman filter (KF), which is discussed, for
instance, in [2], and the particle filter (PF) elab-
orated on in [3]. The KF is often used for linear
systems, while the PF is more robust in its appli-
cation for multimodal and nonlinear systems.
The KF is used to produce a statistically opti-
mal estimate of the position and scale of the sign
in consecutive frames of an input video. Conse-
quently, sign trackers based on the KF usually
assume a uniform and constant direction motion
for the vehicle with a priori known velocity and
traffic sign size. An example of the adoption of
the KF tracker in the TSDR system is found in
[2]. The same assumptions are used by the afore-
mentioned authors; that is, the vehicle moves in
a straight line and exposed to a constant slope
while maintaining an approximately constant
speed. Based on these assumptions, they esti-
mated sign position and velocity in 3D space.
The nonlinearity of the resulting measurement
model was eliminated by adopting the extended
KF [2]. The following three limitations of the
system in [2] were identified: unrealistic assump-
tions related to the real size of signs; the con-
stant speed of vehicles, which have to be known
a priori; and the assumption that trajectories are
straight.
In general, the PF is also widely used in
object tracking. Very few research works using
the PF to detect signs have been found. Recent-
ly, the authors of [3] used the sequential impor-
IEEE Wireless Communications • December 2013
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 41
tance sampling PF to track signs. As an observa-
tion model, the authors used shape features
extracted through a Sobel filter. The output
edge map was approximated using discrete points
at the inner and outer edges of the sign board.
The likelihood was then updated using the
Mahalanobis distance between the measured
and predicted positions of signs. The algorithms
have shown high detection rates based on the F-
measure evaluations.
Although tracking represents an important
stage in TSDR systems, this stage has not
received as much attention as other stages. Most
existing works rely mainly on KFs or PFs. The
KF is the more heavily utilized algorithm com-
pared to PF. Its application is unfortunately lim-
ited to linear systems. On the other hand, PF
proves to be more robust when applied to multi-
modal and nonlinear problems. In our opinion,
much work remains to be done on tracking signs.
RECOGNITION TECHNIQUES
The recognition stage receives a list of ROIs
that possibly contains one or more signs. In this
stage, ROIs are first recognized as to whether
they are signs. They are then classified, with
regard to their content, into the appropriate cat-
egories (stop signs, speed limit signs, etc.); this is
done by using the knowledge obtained from a
set of training samples. As determined from our
assessment of the reviewed literature, we would
like to emphasize that not all research works on
TSDR use a separate detection/recognition step
as described in this article. To design a robust
recognition stage, three steps are used: prepro-
cessing, training, and testing. As the output of
the detection sign is probably unaligned with an
unknown size of candidates, some basic image
preprocessing, such as rescaling or rotation, is
required in the first step. The second step, which
is training, determines the set of basic elements
needed to represent signs. Each sign is repre-
sented by a coefficient vector, which is generated
by applying a set of basic elements on the sign.
The third step is the testing process, in which the
best match for a sign is carried out.
Mainly, signs are recognized using template
matching or machine learning techniques. The
choice of the appropriate recognition algorithm
depends essentially on the training sample statis-
tics (e.g., class distribution and size), the features
used, and the output of the detection algorithm.
Template-matching-based algorithms look
for the most similar training samples that exist,
which are stored in a database of a respective
and unknown ROI. Recall that TM is performed
in the extracted ROIs for the purpose of finding
signs. An interesting method for simultaneous
detection and recognition, based on TM, is the
one previously discussed: [11]. The input image
was matched to DT images of differently shaped
sign templates, grouped in terms of similarity
within a tree structure.
Machine-learning-based algorithms: As with
detection, learning-based methods can be effi-
ciently used for sign recognition. They are
trained to find an optimal separation between
classes. These include SVM, NN, Adaboost [14],
and others. Multi-layer perceptron NNs (MLP-
IEEE Wireless Communications • December 2013
VS
50
Figure 2. Transmission scenario of traffic signs.
NNs) are used in [5] to implement the recogni-
tion stage. Each pattern is recognized through
the use of one network, since it makes the classi-
fier expandable. The authors use six MLP-NNs
to identify ROIs as belonging to one of the six
categories of signs used in their experiments.
Two algorithms are used to train the classifiers:
the Scaled-Conjugate-Gradient algorithm and
Resilient-Back-Propagation. The authors show
that Scaled-Conjugate-Gradient yields the best
results compared to Back-Propagation and SVM.
Recently, a comparative study was introduced in
[15] to evaluate the performance of k-d trees,
random forests, and SVMs for traffic-sign recog-
nition. A random forest with 100 trees yielded
higher classification accuracy on subsets with
more than 500 features. The random forests
used in this work outperform SVMs and k-d
trees due to their randomness and the robust-
ness to outliers and unbalanced data sets. Unfor-
tunately, no comparison with NNs was made.
Tree classifiers are useful techniques used to
recognize signs. A random forest classifier, a
kind of decision tree, is a very promising tech-
nique used in sign recognition. A random forest
consists of constructing many decision trees and
yields the class that is the mode of class output
of individual trees [15]. As stated in [15], it out-
performs SVM and k-d trees in its identification
of signs.
Despite the enormous number of papers, the
recognition of signs is still not perfect. Among
classifiers found in the literature, the random
forest remains a powerful classifier [15]. Ulti-
mately, accurate recognition techniques are
needed to increase the recognition rate.
TRANSMISSION
After the detection and recognition of a sign, the
latter is used only by one driver in his or her
own vehicle. Sharing recognized sign information
between vehicles is not yet supported by modern
TSDR systems. To show the impact of the trans-
mission stage, let us consider a particular safety
scenario: Fig. 2. Once a vehicle source Vs detects
and recognizes signs through its TSDR system, it
warns the driver to take appropriate actions. In
some situations, sharing signs between V s and
neighbouring vehicles is required. Hence, the Vs
41
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 42
To the best of our driver notifies other drivers located within its
transmission range. The communication system
knowledge, recognition of vehicles ensures that data packets are reliably
sent to all vehicles, and hence allows drivers to
and transmission sys- make appropriate decisions.
tems of signs do not Regarding the communication aspect of
TSDR, inter-vehicle communication is particu-
exist. We expect that larly challenging due to the high rate of vehicle
the new generation of mobility and signal-weakening obstructions, such
as buildings. To the best of our knowledge,
vehicles will be recognition and transmission systems of signs do
not exist. We expect that the new generation of
equipped with on-board vehicles will be equipped with onboard units
units based on the stan- based on the standard IEEE 802.11p interfaces
and antennas that work on a frequency band
dard IEEE 802.11p ranging from 5.8 to 5.9 GHz [13].
interfaces and antennas
that work on a frequen-
CONCLUSION
In this article, TSDR systems are studied. Chal-
cy band ranging from lenges facing TSDR systems are regrouped into
5.8 to 5.9 GHz. five different categories: camera acquisition, sur-
rounding conditions, moving and vibrating car,
sign state, and road conditions. A brief descrip-
tion of the problems encountered at each stage
is given. We divide the TSDR into four different
stages: preprocessing, detection, recognition, and
tracking. We then describe each of these stages,
showing some techniques that can be used effi-
ciently. For each stage, the suggested techniques
are further classified. For example, in the detec-
tion stage, techniques are classified following the
shape, color, and gradient information. For each
stage, the best techniques used in the literature
are highlighted, followed by some concluding
interpretations. The transmission stage is also
examined in this study. This is due to its impor-
tance in inter-vehicular communication.
ACKNOWLEDGMENT
This work was partially supported by Canada
Research Chair Programs, DIVA Strategic
Research Center, Natural Sciences and Engi-
neering Research Council of Canada (NSERC),
and Fonds de Recherche du Québec, Nature et
Technologie (FRQNT).
REFERENCES
[1] S. Marsi et al., “Video Enhancement and Dynamic
Range Control of HDR Sequences for Automotive Appli-
cations,” EURASIP J. Advances in Signal Processing, vol.
2007, 2007, p. 9.
[2] C. Y. Fang, S. W. Chen, and C. S. Fuh, “Road Sign
Detection and Tracking,” IEEE Trans. Vehic. Tech., vol.
52, no. 5, 2003, pp. 1329–41.
[3] M. Meuter, A. Kummert, and S. Muller-Schneiders, “3D
Traffic Sign Tracking Using a Particle Filter,” Proc. 11th
Int’l. IEEE Conf. Intelligent Transportation Sys., 2008,
pp. 168–73.
[4] H. Gómez-Moreno et al., “Goal Evaluation of Segmen-
tation Algorithms for Traffic Sign Recognition,” IEEE
Trans. Intelligent Transportation Systems, vol. 11, no. 4,
2010, pp. 917–30.
[5] Y. Y. Nguwi and A. Z. Kouzani, “Detection and Classifi-
cation of Road Signs in Natural Environments,” Neural
Computing & Applications, 2008, pp. 265–89.
[6] D. L. Kellmeyer and H. T. Zwahlen, “Detection of Highway
Warning Signs in Natural Video Images Using Color Mage
Processing and Neural Networks,” Proc. IEEE Int’l. Conf.
Neural Networks, vol. 7, 1994, pp. 4226–31.
[7] Y. Aoyagi and T. Asakura, “A Study on Traffic Sign
Recognition in Scene Image Using Genetic Algorithms
and Neural Networks,” Proc. 22nd IEEE Int’l. Conf. Ind.
Electron., Control Instrum., Taipei, Taiwan, Aug. 1996,
vol. 3, pp. 1838–43.
42
[8] W.-J. Kuo and C.-C. Lin, “Two-Stage Road Sign Detec-
tion and Recognition,” 2007 IEEE Int’l. Conf. Multime-
dia and Expo, 2007, pp. 1427–30.
[9] G. Loy, “Fast Shape-Based Road Sign Detection for a
Driver Assistance System,” IEEE/RSJ Int’l. Conf. Intelli-
gent Robots and Systems (IROS, 2004), pp. 70–75.
[10] N. Dalal and B. Triggs, “Histograms of Oriented Gradients
for Human Detection,” CVPR, vol. 1, 2005, pp. 886–89.
[11] A. Ruta, Y. Li, and X. Liu, “Real-Time Traffic Sign
Recognition from Video by Class-Specific Discriminative
Features,” Pattern Recognition, vol. 43, no. 1, 2010,
pp. 416–30.
[12] P. Gil-Jiménez, H. Gómez-Moreno, and F. López-Fer-
reras, “Road-Sign Detection and Recognition based on
Support Vector Machines,” S. Maldonado-Bascón, S.
Lafuente-Arroyo, IEEE Trans. Intelligent Transportation
Systems, 2007.
[13] T. Mangel, O. Klemp, and H. Hartenstein, “5.9 GHz
Inter-Vehicle Communication at Intersections: A Vali-
dated Non-Line-of-Sight Path-Loss and Fading Model,”
EURASIP J. Wireless Comm. and Networking 2011,
2011, p. 182.
[14] X. Baró et al., “Traffic Sign Recognition Using Evolu-
tionary Adaboost Detection and Forest-ECOC Classifica-
tion,” IEEE Trans. Intelligent Transport Systems, vol. 10,
issue 1, 2009, pp. 113–26.
[15] F. Zaklouta and B. Stanciulescu, “Real-Time Traffic-Sign
Recognition Using Tree Classifiers,” IEEE Trans. Intelligent
Transportation Systems, vol. 13, no. 4, 2012, pp. 1507–14.
[16] G. Overett et al., “Creating Robust High-Throughput
Traffic Sign Detectors Using Centre-Surround HOG
Statistics,” Machine Vision and Applications, Special
Issue, pp. 1–14.
BIOGRAPHIES
ABDELHAMID MAMMERI (amammeri@uottawa.ca) is a senior
research and postdoc fellow at Diva research strategic cen-
ter, University of Ottawa, Canada. He obtained his Master’s
degree from Universite Catholique de Louvain, Belgium,
and his Ph.D. in electrical and computer engineering from
the University of Sherbrooke, Canada. He is the recipient of
the prestigious FQRNT Quebec fellowship award at the
postdoc level in 2012. His current research interests include
multimedia and image processing applied on smart vehi-
cles, multimedia streaming over wireless ad hoc and sensor
networks, multimedia green applications, and multimedia
streaming over VANET. He has extensively published in
highly ranked international conferences and journals in his
area. He served as a Technical Program Chair for the 3rd
ACM Symposium on Development and Analysis of Intelli-
gent Vehicular Networks and Applications (DIVANet 2013),
a Program Co-Chair of the 8th ACM Workshop on Perfor-
mance Monitoring and Measurement of Heterogeneous
Wireless and Wired Networks (PM²HW²N 2013) and Techni-
cal Program Co-Chair of the 9th IEEE International Work-
shop on Performance and Management of Wireless and
Mobile Networks (P2MNET 2013). He also served as a Tech-
nical Program Committee member for several conferences,
including MIC-CAI2012, IEEE VTC 2013, and MIC-C
AZZEDINE BOUKERCHE held a faculty position with the Univer-
sity of North Texas, Denton, and was a senior scientist with
the Simulation Sciences Division, Metron Corporation, San
Diego, CA. He is a full professor and holds a Canada
Research Chair position at the University of Ottawa,
Ontario, Canada. He is a Fellow of the Canadian Academy
of Engineering and the founding director of the PARADISE
Research Laboratory, School of Information Technology
and Engineering, Ottawa. Prior to this, he held a faculty
position at the University of North Texas, and he was a
senior scientist at the Simulation Sciences Division, Metron
Corporation, San Diego, California. He was also employed
as a faculty member in the School of Computer Science,
McGill University, Montreal, Quebec, Canada, and taught
at the Polytechnic of Montreal. He spent a year at the Jet
Propulsion Laboratories (JPL)/National Aeronautics and
Space Administration (NASA)-California Institute of Tech-
nology, Pasadena, where he contributed to a project cen-
tered on the specification and verification of the software
used to control interplanetary spacecraft operated by
JPL/NASA Laboratory. His current research interests include
wireless ad hoc and sensor networks, wireless networks,
mobile and pervasive computing, wireless multimedia, QoS
service provisioning, performance evaluation and modeling
of large-scale distributed systems, distributed computing,
large-scale distributed interactive simulation, and parallel
discrete-event simulation. He has published several research
IEEE Wireless Communications • December 2013
MAMMERI_LAYOUT_Layout 12/20/13 12:21 PM Page 43
papers in these areas. He served as a Guest Editor for the
Journal of Parallel and Distributed Computing Special
Issues on Routing for Mobile Ad Hoc, Wireless Communica-
tion and Mobile Computing, and Mobile Ad Hoc Network-
ing and Computing, ACM/Kluwer Wireless Networks,
ACM/Kluwer Mobile Networks Applications, and Journal of
Wireless Communication and Mobile Computing. He serves
as an Associate Editor of IEEE Transactions on Parallel and
Distributed Systems, IEEE Transactions on Vehicular Tech-
nology, Elsevier Ad Hoc Networks, Wiley International Jour-
nal of Wireless Communication and Mobile Computing,
Wiley’s Security and Communication Network Journal, Else-
vier Pervasive and Mobile Computing Journal, IEEE Wireless
Communications, Elsevier’s Journal of Parallel and Dis-
tributed Computing, and SCS Transactions on Simulation.
He was the recipient of the Best Research Paper Award at
IEEE/ACM PADS 1997, ACM MobiWac 2006, ICC 2008, ICC
2009, and IWCMC 2009, and the recipient of the Third
National Award for Telecommunication Software in 1999
for his work on a distributed security systems on mobile
phone operations. He has been nominated for the Best
Paper Award at the IEEE/ACM PADS 1999 and ACM MSWiM
2001. He is a recipient of an Ontario Early Research Excel-
lence Award (previously known as Premier of Ontario
Research Excellence Award), Ontario Distinguished
Researcher Award, and Glinski Research Excellence Award.
He is a co-founder of the QShine International Conference
on Quality of Service for Wireless/Wired Heterogeneous
Networks (QShine 2004). He served as General Chair of the
Eighth ACM/IEEE Symposium on Modeling, Analysis, and
Simulation of Wireless and Mobile Systems, and the Ninth
ACM/IEEE Symposium on Distributed Simulation and Real-
IEEE Wireless Communications • December 2013
Time Application (DS-RT), Program Chair for the ACM
Workshop on QoS and Security for Wireless and Mobile
Networks, ACM/IFIPS Europar 2002 Conference, IEEE/SCS
Annual Simulation Symposium (ANNS 2002), ACM WWW
2002, IEEE MWCN 2002, IEEE/ACM MASCOTS 2002, IEEE
Wireless Local Networks WLN 03-04, IEEE WMAN 04-05,
and ACM MSWiM 98-99, and a TPC member of numerous
IEEE and ACM-sponsored conferences. He served as Vice
General Chair for the Third IEEE Distributed Computing for
Sensor Networks Conference in 2007, as Program Cochair
for IEEE GLOBECOM 2007–2008 Symposium on Wireless
Ad Hoc and Sensor Networks, and for the 14th IEEE ISCC
2009 Symposium on Computer and Commmunication, and
as Finance Chair for ACM Multimedia 2008. He also serves
as a Steering Committee Chair for the ACM Modeling,
Analysis, and Simulation for Wireless and Mobile Systems
Conference, the ACM Symposium on Performance Evalua-
tion of Wireless Ad Hoc, Sensor, and Ubiquitous Networks,
and IEEE/ACM DS-RT.
MOHAMMED ALMULLA received his B.Sc. and Ph.D. degrees in
computer science from McGill University, Montreal, Cana-
da, in 1986 and 1995, respectively. He is an associate pro-
fessor in the Computer Science Department, College of
Computer Sciences and Engineering, Kuwait University.
Currently, he is the director of the Graduate Program at
the Computer Science Department at Kuwait University and
a consultant at Kuwait Central Agency for Information
Technology. His research interest is in the areas of dis-
tributed simulation, distributed computing, modeling and
simulation of parallel methodologies, and artificial intelli-
gence.
43