Tracking and counting of vehicles for flow analysis
from urban traffic videos
Igor S. Farias, Bruno J. T. Fernandes, Edison Q. Albuquerque and Byron L. D. Leite
Polytechnic School of Pernambuco - POLI
University of Pernambuco - UPE
Recife, Brazil
{isf, bjtf, edison, byronleite}@ecomp.poli.br
Abstract—Among the major problems faced by urban centers
there is traffic congestion. This problem comes from the growing
number of vehicles on the streets and has already become the
subject of several researches seeking for solutions to it. Among
the mechanisms that allow congestion reduction is traffic control,
which requires metrics that enable traffic analysis in real time.
To determine the flow of vehicles the widely used mechanism
is the counting of occurrence of vehicles on a street, which
is usually performed from sensors (e.g. magnetic or thermal).
However, these approaches have a rather high installation and
maintenance difficulty. Thus, the objective of this paper is to
present a mechanism capable of counting from video images.
To accomplish this task it is used image processing resources
that do not require large computational power, thus allowing the
mechanism to be easily coupled to common transit systems. The
result obtained has an accuracy of more than 90 % in videos of
urban traffic cameras.
Key words—Vehicle detection, vehicle count, image processing.
I. I NTRODUCTION
Transportation has always been an extremely relevant aspect
for society, the great advance of humanity is due to the
mobility, in addition to being treated as synonymous with
progress. Mobility problems, however, have increased con-
siderably, where problems related to traffic congestion have
become a major concern for urban centers [1].
Due to the huge fleet of vehicles we currently have, there is
an unavoidable interest in reducing the effects of congestion.
In the major urban centers, it is notorious and easily concluded
that the demand for services, commerce, and personal travel
grow year by year, placing unprecedented challenges in the
area of transportation.
The mechanism that allows a mitigation of this massive
problem is traffic control. This process makes the vehicle
flow to be maximized in the roads allowing a shorter travel
time, which implies the reduction of congestion. Vehicle flow
refers to the number of vehicles passing a particular route over
a period [2]. According to DENATRAN (Brazilian National
Department of Transit), the leading cause of congestion is the
intersection of streets. The traffic light is the main mechanism
used to control traffic at these intersections [3].
Urban traffic control has been evolving over the years, so
we are in a generation where real-time traffic data collection
978-1-5386-3734-0/17/$31.00
2017
c IEEE
is essential [4]. Based on the density of vehicles in the street,
it becomes possible to develop a mechanism that effectively
reduces congestion. From the number of vehicles that travel
on a particular route, it becomes possible to obtain predictions
of traffic jams, which makes it possible to take some actions,
such as a better adjustment of a traffic light’s time.
To obtain information about the flow of vehicles in real
time, it is necessary to use substantial computational power
together with images obtained by cameras. The use of simpler
sensors, such as magnetic sensors, has become unfeasible these
days mainly due to the difficulty inherent in their installation
and maintenance [5]. When using video cameras that have a
greater ease of handling, the amount of information that can
be obtained increases significantly, but for this, it becomes
necessary to use efficient systems to make the processing
useful when executed in real time [6].
The essential information for the process of traffic control
is the flow of vehicles [7]. In this way, this paper presents an
approach to be executed in real time that makes it possible to
obtain the counting of occurrences of the vehicles. This infor-
mation, when processed, has great value for traffic control.
II. R ELATED WORK
When attempting to mitigate traffic jam problems, the first
point to be addressed is regarding the density of vehicles; a
mechanism is presented in [7] that through the use of magnetic
sensors it becomes possible to count the quantity of cars that
pass in astreet. In addition to counting, based on the magnetic
field obtained by the sensors, the vehicles are classified by
bus, car, motorcycle or bicycle.
Based on the image obtained through a UAV (Unmanned
Aerial Vehicle), [4] counts vehicles in a scene. The process
begins with a screening of the paved areas, to restrict the
search space. The next step is the feature extraction process
based on scalar transformations. The information obtained is
passed to an SVM (Support Vector Machine) that can inform
the number of vehicles in the scene.
A video-based approach is made in [8] where the slicing of
video in frames is performed, and a mask is applied to extract
parts of the frame. Each slice obtained is used as input to a
CNN (Convolutional Neural Network) so that it can identify
and classify the vehicles. Such an approach is interesting
because it has proved quite robust for identifying vehicles in
a scene. Also using videos, [5] presents a method of detecting
and counting vehicles where the process is based on particle
clustering, so that each set of particles in a video represents a
vehicle. Despite achieving good results, this method requires a
great deal of computational effort, a scarce resource for traffic
monitoring devices.
In [9] a real-time vehicle counting technique using GMM
(Gaussian Mixture Model) is approached to perform a better
adaptation of the algorithm to the brightness of the scene.

III. V EHICLE C OUNTING S YSTEM

Vehicle flow analysis is only a starting point for traffic
control, and this initial process must be carried out efficiently
and effectively due to the need to be a basis for traffic control.
In the development of the present work, it was necessary to
implement an algorithm that can be easily used in embedded
systems, since the system responsible for flow control should
be the least invasive in traffic, i.e., the use of sensors or other
mechanisms that prevent flow should be avoided. On the other
hand, the use of computational systems that can obtain the
necessary information without intervening in the flow should
be stimulated. Another requirement that must be evaluated is
the computational cost required by this mechanism since it
is the insertion of the computational process into embedded
systems that have a processing limitation.
Sophisticated systems with very high precision usually have
a high computational cost, according to [10]. Thus, its use in
traffic control assistance in large cities is hampered because the
cost associated with implementation is very high. The process
discussed below is aimed at counting vehicles from a video,
but focusing on mitigating the need for great processing power.
The process has been divided into two stages, the detection
of a vehicle from a video is performed initially, and then the
counting of the vehicles that are present in the recording is
performed. The flowchart in the figure 1 presents the steps of
the algorithm that repeat until the video completes.

A. Detection Figure 1. Flowchart of the presented algorithm.

In this first stage of the process, the video is cut into several
frames. This cut must be done in such a way that a sequence
of images can be obtained and that allows the motion detection
of the objects contained therein.
Initially, the region of interest is selected, and information
such as trees and sidewalks is ignored. The sequence of steps
starts with the selection of a frame to be chosen as background,
by default the first frame obtained in the video was used,
however, for a greater accuracy, this image can be selected
manually. Background elimination takes place in three stages.
In the first step, the difference between two frames, f and f+1
is calculated, then the difference is compared, and finally, the
pixels that have the same value in the difference calculation
are eliminated. Then the image resulting from the background
removal is converted to grayscale. At that moment, the vehicle
is represented by a set of pixels that move coherently, that is,
a region that has a different color from the darkest background
or vice versa. The figure 2 presents a vehicle where the
background removal operation was performed.
In the following steps, only the foreground is used, since the
relevant information at that time is the objects obtained after
the background removal. The next step is the application of
morphological operations in the object segmentation process.
The first applied function is Closing so that small breaks in
the contour of the vehicles are corrected, then the Opening is
applied, so that the contours of the objects are smoothed, and
finally the Dilatation operation is carried out to Smooth curves
and correct imperfections [11].
After the segmentation process of the objects, the contours
are identified in the image, and for each object contained
in the scene, a minimum rectangle is inserted, in this way,
each rectangle obtained is the representation of a vehicle. The
equation 1 checks whether the selection of a given object

Figure 2. Frame after morphological operations.
can be interpreted as a vehicle. If the result is equal to 1,
the object is considered as a vehicle, if the result is 0, the
object is disregarded. A minimum rectangle size is defined so
that it is possible to identify valid objects in the image, that
is, even if it is inserted into a rectangle, the object is only
considered a vehicle if the rectangle containing it has height
and width greater than or equal to the threshold. The value
set for the threshold must be carefully checked and tested
because the distance from the camera to the object implied
directly in its height and width parameters, often causing
vehicle identification error.

objectSize ≥ minimumRectangle, 1
objectSize < minimumRectangle, 0
(1)
If the rectangle is set to a minimal value, there is a
possibility a single vehicle will be identified in several parts
and having such parts inserted each into a rectangle, causing
the error of a single vehicle to be identified as many. On the
other hand, if set to a very large size, the identification of
vehicles such as motorcycles can be ignored, also causing the
counting error. The size of the minimum rectangle has such
relevance that it can be used to classify the vehicle, by bus,
car or motorcycle, for example. However, this procedure will
not be addressed in this paper. The figure 3 shows the result
obtained with the mentioned steps for detection.
B. Counting
The counting process has as input the objects found in the
detection phase. Valid rectangles are taken as vehicles on the
scene, and from that point, the centroids of each object are
calculated. This mechanism will represent a vehicle in the
scene.
In the first scene, each of the centroids found is assumed
as a vehicle and are stored in a list. From the second scene
onwards, it is necessary to compare the vehicles stored in
the list, and with those found in the current frame, such
comparison is necessary so that there is a link between
the same vehicle in different frames. Because it is a video,
Figure 3. Result of the detection process.
the same vehicle appears in several scenes, being of great
importance the treatment of that fact, lest it allows the same
vehicles can be counted several times.
The comparison of vehicles stored in the list with those
identified in the new scene is performed based on the po-
sition of the centroids in the frames f and f-1. The com-
parison takes into account the distance between the two
points and the angle formed between them. To obtain the
distance is used the canonical form of the Euclidean distance
p
(x1 − x2 )2 + (y1 − y2 )2 and the angle is set between -
180o and 180o increasing clockwise. The inclusion of the
two parameters mentioned is of paramount importance so that
vehicles occasionally change tracks or pass at high speed are
not lost.
γ = −0.008 ∗ angle2 + 0.4 ∗ angle + 25 (2)
After obtaining the distance between the centroids and the
angle formed therebetween, it is necessary to use a threshold
to check whether the two centroids compared represent the
same vehicle or are different vehicles. The value of this
threshold should be cautiously defined because the distance of
the camera in the scene and the angle that the image is obtained
can influence the parameters. Based on tests, the threshold γ
used for this work is according to the equation 2. The values
used presented good results with the videos that compose the
database tested, which in turn, is composed of videos where
the camera positioning is diagonal concerning the street.
After setting the γ, it is evaluated whether the calculated
distance between the points is less than the value obtained. If
the distance at time t is less than the limit obtained, this new
point refers to the same vehicle since it has the centroid of the
previous scene, so that the position of the vehicle is updated
with the new values. Otherwise, the new centroid is defined
as a new vehicle and stored in the list.
To perform counting of vehicles, a counting point mark is
inserted into the frame. This point is the determination of a
linear position in the frame and each vehicle that crosses that
demarcated region, it must be counted. The count is given
by the position of the centroid on frame f and the defined
point, with the position of the centroid being greater or less,
 streets with a lot of tracks, the flow of vehicles that can pass
without being imply in congestion is greater than a street with
few tracks. That is, a street with more tracks will have a larger
number of vehicles, but this value does not necessarily imply
congestion.
The algorithm developed to perform the count by track,
simply change the region of interest by centralizing only one
track and add a loop that counts all of them.

Figure 4. Counting point marking.

depending on the direction of the vehicle. When the point

crosses the mark, addition the car on the accounting. The
vertical line in the figure 4 displays the counting point for
the scene.
The region defined as a counting point must be perpendic- Figure 5. Source image without region of interest selection.
ular to the flow of vehicles so that it is possible to recognize
all vehicles crossing the region. Marking should preferably be The region of interest defined was focusing on the central
done at the end of the image (direction of vehicle flow) so part of the route, so as to remove unnecessary information as
that all vehicles have already been identified at the point of part of the traffic light structure, such as sky, sidewalks, and
counting. After crossing this point, the vehicles are ignored trees. The area must be selected very careful to avoid losing
because its count has already been performed. information where the vehicles travel. The figure 6 represents
the selection made in one of the experiments related to the
IV. E XPERIMENTS AND R ESULTS whole scene captured in the figure 5.
The images used to perform the experiments were obtained
through the Camerite 1 website where it is possible to have
access to several cameras around the world. The videos used
were obtained in cameras that are in the city of Recife in
Brazil. All videos selected are with good light intensity and
during the day. The positioning of the selected cameras is
diagonal, a factor that hinders the detection process since it
can occlude objects. The best positioning is a vertical view of
the street, so occlusion becomes more difficult to occur.
Figure 6. Background according to selection of the region of interest.
The process described in the previous section was imple-
mented using the Python language and the OpenCV (Open
Source Computer Vision Library) library.
Initially, the video was sliced into several frames. This cut
is done with a capture every 200 milliseconds approximately.
This value may vary according to the experiment, but it is
important to note that this parameter will directly influence
the total execution time of the counting process and the
computational cost necessary to the process.
For the experiment presented, the algorithm counts all tracks
of the street, as can be seen in the figure 5. However, for an Figure 7. Frame after morphological operations.
intelligent traffic control process that uses the flow of vehicles
as input data, it becomes interesting to perform the count In the detection process, the functions used to remove
taking into account the number of tracks on the street. This the background and perform the morphological operations
normalization by tracks is important because on very wide mentioned in the previous session were those contained in the
OpenCV library for Python. The result is shown in the figure
1 www.camerite.com 7.
 Figure 8. Identification of vehicles after detection and counting phase.
The figure 8 presents the final result of the detection and
counting, where each vehicle is contained in a rectangle, and
all its centroids in the course of the frames are demarcated.
This factor implies not counting the same object more than
once. However, due to occlusion, some objects may be mis-
labeled. The error is caused by the undue grouping of two
or more objects or the opposite case which is the separation
of objects that are unique. The experimental results obtained
were compared with the results obtained by [12] and [13],
these works have similar objectives.
Table I
E XPERIMENTS DONE WITH VIDEO OF THE E NGENHEIRO D OMINGOS
F ERREIRA AVENUE
Experiment Actual Amount Amount detected
1 06 06
2 09 11
3 12 10
4 17 18
5 18 16
6 22 22
7 24 21
8 28 31
9 28 32
10 12 13
The process developed in [12] presents a good accuracy,
according to the author. However, the approach to the vehicle
tracking mechanism may cause greater difficulty in counting
objects. This problem was mitigated by the centroid mecha-
nism discussed in this paper. In turn, the algorithm developed
in [13] has the approach quite similar to that presented in this
work, since it uses centroids to track the vehicles, but it has
little efficiency. Even when using videos with good lighting
and in the morning the accuracy in some experiments reaches
less than 50%. With such a low hit rate, it is unfeasible to use
such a procedure to aid in the flow control of vehicles.
To demonstrate the operation of the algorithm, ten experi-
ments were selected using a camera at Engenheiro Domingos
Ferreira Avenue with videos lasting between thirty and sixty
seconds, all in AVI format. The results obtained are presented
in the table I.
After the executions presented, the Wilcoxon Signal Test
was applied with significance level equal to 95% to validate if
the count performed by the algorithm is statistically equal to
the present number of vehicles present in the video, the result
was positive. The number detected is statistically equals real
number.
Figure 9. Comparison of the quantity obtained with the process described
and the actual quantity.
The graph in the figure 9 presents the result by making
a direct comparison between the values found in the count
performed by the algorithm and the actual number of vehicles
in the video.
Table II
R ESULT FOR COUNTING WITH DIFFERENT FLOWS
Location Actual Amount Amount detected
Gov. Agamenon Magalhães Av. 09 10
Gov. Agamenon Magalhães Av. 14 14
Gov. Agamenon Magalhães Av. 27 24
Moreira e Silva Av. 15 15
Moreira e Silva Av. 21 21
Moreira e Silva Av. 25 21
To demonstrate the functioning of the algorithm, more
experiments were also carried out with cameras from other
parts of the same city (Governador Agamenon Magalhães and
Moreira e Silva avenues). All experiments were run three
times to validate the count performed. The results obtained are
presented in the table II. Based on the experiments performed
on videos from various locations, the algorithm obtained the
accuracy of well over 90%. Videos with good illumination
and without occlusion of vehicles, the accuracy was equal to
100%.
In real-time applications, a very important factor is the com-
putational cost. Since it is of the utmost importance that the
application can return promptly, a response without affecting
or delaying the performance of the system/application as a
whole is necessary. To present the computational cost, the
time in seconds for the experiments was listed in the table I.
The experiments were run on a laptop with Ubuntu operating
system version 16.04, Intel Core i5 processor, 8GB of RAM
and NVIDIA GeForce 820M graphics card with 2GB.
The table III presents the average computational cost ob-
tained for the processing of each frame. Observing the table,
it is possible to verify that the highest computational cost is
for the morphological operations. The processor usage aver-
aged 12.9% with a standard deviation of 1.41. One relevant
information is that the total execution time required to process
 Table III
C OMPUTATIONAL C OST
Stage Cost
Morphological Operations 8.41x10−4 seg
Vehicle Detection 4.38x10−4 seg
Vehicle Count 2.74x10−4 seg
Frame Processing 1.59x10−2 seg
a 60-second video was on average 17.1 seconds when the 200-
millisecond interval between frames capture was not used.
In this way, it is possible to realize that the algorithm is
computationally efficient to be used in real time.
Figure 10. Vehicle occlusion.
The task of counting vehicles is not trivial since several
factors hamper image processing, among them, are noise,
luminosity variation, climate and, mainly, occlusion. The latter
is quite complicated to deal with, the figure 10 exemplifies
the problem. In this case, the presence of a truck prevents
the visualization of other cars that are in the street, so that
the algorithm will not be able to identify them. Some papers
propose alternatives to mitigate this problem, in [14] is shown
a mechanism that uses mutually the windshield detection and
the grouping of points, in [15] is presented the geometric
process based on the vertices that form the vehicle. However,
the computational cost to use these techniques mentioned is
greater than presented in this paper.
The mentioned problems generate the counting errors pre-
sented in the tables I and II, for example, in the last experiment
presented for Moreira e Silva Avenue the detection of four
vehicles was not performed.
V. C ONCLUSION
In this paper, it was presented a mechanism to provide the
input data for a traffic control system, the density of vehicles in
one street. Once in possession of this information, it becomes
possible to evaluate the flow and to make an effective control
in the opening time of the traffic lights, avoiding the traffic
jam. The implemented algorithm achieves, in a simplified way
and without requiring much computational power to perform,
the counting of vehicles.
During the implementation process, some limitations were
found in the presented methodology. Among them, the ones
with the greatest impacts were related to the light density
and the stabilization of the camera. These two characteristics
directly compromise the detection of the vehicle, as they affect
the operations performed in the video processing. An example
of this is the removal of the background in each frame.
As an improvement in the presented process, it is necessary
to perform counting with light variation and camera instability,
since it becomes important to count vehicles regardless of
the brightness of the scene and climate variation. This paper
refers to the initial step for the development of effective traffic
control systems that use image processing mechanisms to
obtain traffic density.
R EFERENCES
[1] M. Heinen, C. Sá, F. Silveira, C. Cesconetto, and G. Sohn, “Controle
inteligente de semáforos utilizando redes neurais artificiais com funções
de base radial,” Encontro Anual de Tecnologia da Informação e Semana
Acadêmica de Tecnologia da Informação. Frederico Westphalen/RS, pp.
38–45, 2013.
[2] C. N. de Trânsito, Manual Brasileiro de Sinalização de Trânsito, 2007.
[3] D. N. de Trânsito, Manual de Semáforos, 2nd ed., 1984.
[4] T. Moranduzzo and F. Melgani, “Automatic car counting method for
unmanned aerial vehicle images,” IEEE Transactions on Geoscience and
Remote Sensing, vol. 52, no. 3, pp. 1635–1647, 2014.
[5] P. R. M. Barcellos, “Detecção e contagem de veı́culos em vı́deos de
tráfego urbano,” 2014.
[6] E. Mitsakis, J. M. Salanova, and G. Giannopoulos, “Combined dynamic
traffic assignment and urban traffic control models,” Procedia-Social and
Behavioral Sciences, vol. 20, pp. 427–436, 2011.
[7] J. M. Rizwan, P. N. Krishnan, R. Karthikeyan, and S. R. Kumar, “Multi
layer perception type artificial neural network based traffic control,”
Indian Journal of Science and Technology, vol. 9, no. 5, 2016.
[8] C. M. Bautista, C. A. Dy, M. I. Mañalac, R. A. Orbe, and M. Cordel,
“Convolutional neural network for vehicle detection in low resolution
traffic videos,” in Region 10 Symposium (TENSYMP), 2016 IEEE.
IEEE, 2016, pp. 277–281.
[9] B. Setiyono, D. S. Ratna, I. Mukhlash, and R. J. Augusta, “A new
approach algorithm for counting of vehicles moving based on image
processing,” International Journal of Computer Science and Information
Security, vol. 14, no. 10, p. 366, 2016.
[10] G. Buttazzo, Hard real-time computing systems: predictable scheduling
algorithms and applications. Springer Science & Business Media, 2011,
vol. 24.
[11] J. Y. Gil and R. Kimmel, “Efficient dilation, erosion, opening, and clos-
ing algorithms,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 24, no. 12, pp. 1606–1617, 2002.
[12] M. Daigavane and P. Bajaj, “Real time vehicle detection and counting
method for real time vehicle detection and counting method for unsuper-
vised traffic video on highways unsupervised traffic video on highways,”
IJCSNS, vol. 10, no. 8, p. 112, 2010.
[13] M. B. Subaweh and E. P. Wibowo, “Implementation of pixel based
adaptive segmenter method for tracking and counting vehicles in vi-
sual surveillance,” in Informatics and Computing (ICIC), International
Conference on. IEEE, 2016, pp. 1–5.
[14] J. Yang, Y. Wang, A. Sowmya, Z. Li, B. Zhang, and J. Xu, “Feature
fusion for vehicle detection and tracking with low-angle cameras,” in
Applications of Computer Vision (WACV), 2011 IEEE Workshop on.
IEEE, 2011, pp. 382–388.
[15] C. C. C. Pang, W. W. L. Lam, and N. H. C. Yung, “A method for vehicle
count in the presence of multiple-vehicle occlusions in traffic images,”
IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 3,
pp. 441–459, 2007.