See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/309461986

Clustering Techniques in Data Mining: A Comparison

Conference Paper · March 2015

CITATIONS READS
7 170

1 author:

Pradeep Kumar Singh

Amity University
80 PUBLICATIONS   202 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Software Quality Model and Assessment View project

All content following this page was uploaded by Pradeep Kumar Singh on 01 March 2019.

The user has requested enhancement of the downloaded file.

 Clustering Techniques in Data Mining: A
Comparison
Garima Hina Gulati
Amity University, Uttar Pradesh Amity University, Uttar Pradesh
Noida, U.P., INDIA Noida, U.P., INDIA
Email Id:gars.arora@gmail.com Email Id:hinagu821990@gmail.co
Abstract – Clusteringis a technique in which a given data set
is divided into groups called clusters in such a manner that
the data points that are similar lie together in one cluster.
Clustering plays an important role in the field of data mining
due to the large amount of data sets.This paper reviews the
various clustering algorithms available for data mining and
provides a comparative analysis of the various clustering
algorithms like DBSCAN, CLARA, CURE, CLARANS, K-
Means etc.
Keywords –FuzzyC Means (FCM),Balanced Iterative
Reducing and Clustering using Hierarchies (BIRCH),
Clustering using Representatives (CURE), Density based
Clustering (DBSCAN), Ordering Point to Identify Clustering
Structure (OPTICS), Distributed Density- Based Clustering
(DDC)
I. INTRODUCTION
Clustering is defined as the unsupervised classification of the
data items or the observations i.e. the data sets have not been
classified into any group and so they do not have any class
attribute associated with them. Clustering is widely used as one
of the important steps in the exploratory data
analysis[12].Clustering algorithms are used to find the useful
and unidentified classes of patterns. Clustering is used to divide
the data into groups of similar objects. The objects that are
dissimilar are placed in separate clusters. Depending upon the
metric chosen, a data object may belong to a single cluster or it
may belong to more than one cluster. For example, consider a
retail database that contains information about the items
purchased by the consumers. Clustering will group the
consumers according to their buying patterns. When we group
the objects into clusters then simplification is achieved at the
cost of losing some information. The choice of selecting the
clustering algorithm is a critical step.
Our focus in this paper is to provide a comparative analysis of
various clustering algorithms used in the data mining based
upon the similarity criterion and the complexity. This paper is
organized as follows: Section II provides the literature review.
Section III covers the discussion on Clustering algorithm
978-9-3805-4416-8/15/$31.00 2015
c IEEE 410
P.K.Singh
Amity University,
Noida, U.P., INDIA
Email Id:pksingh16@amity.edu
techniques. Section IV gives the comparison among the
existing clustering algorithms followed by conclusion in
Section V.
II. LITERATURE REVIEW
In [12], Dalal et al. discussed Clustering techniques and divide
them into three major categories: Partitional Clustering,
Density based Clustering and Hierarchical Clustering which
are further subdivided and shown in Fig. 2.1. According to [8]
Lu et al., When the data is large and continuous in nature then
the traditional approaches of mining are not applicable
because the real time data is quickly changing and requires
fast response as well. Random access to data stream is very
expensive so, a single access to streaming data is provided.
Also the storage needed is very large. Therefore, clustering
and mining techniques for stream data are required. In Fahad
et al., The effectiveness of the candidate clustering algorithms
is measured through a number of internal and external validity
metrics, stability, run time and scalability tests[7]. Big volume
of data or big data has its own deficiencies as it needs big
storages and this volume makes operations such as analytical
operations, process operations, retrieval operations, very
difficult and hugely time consuming. To overcome these
difficult problems big data is clustered in a compact format
that is still a informative version of the entire data.
DENCLUE, OptiGrid and BIRCH are suitable clustering
algorithms for dealing with large datasets, No clustering
algorithm performs well for all the evaluation criteria, and
future work should be dedicated to accordingly address the
drawbacks of each clustering algorithm for handling big data.
According to Douglass et al., [16]Document clustering is
usually not received well due to some constraints. Firstly the
clustering technique is relatively slow when handling large
document collection and moreover it is thought that clustering
technique donot help in improving retrieval. Douglass
proposed a document browsing technique that uses some fast
clustering algorithms as primary operations. They have
formulated an interactive browsing technique. This technique
is known as Scatter/Gather technique. Necessity of this
technique is fast clustering technique. Its working can be
understood by considering an example of textbook. In a
textbook if we need to search for some direct term we directly
go to index whereas if we are looking for answer to some
general question we search the table of content. The proposed
system i.e. scatter/gather uses cluster-based navigation method
in order to navigate among the documents. These techniques
group one or more similar documents for future reference.
Initially system scatter process cluster documents in groups and
then gather process selects group to form sub-collections.This
technique exists on two facilities i.e. clustering and
reclustering. For these we use some buckshot and fractionation
algorithm. According to [9] Deepika et al., Distributed
clustering is defined as the process of extraction of
.
The classifications of clustering algorithms for distributed data
are shown in Fig. 2.2. Distributed clustering works on two
types of architectures:
a) Homogeneous datasets: each local site has same attributes
b) Heterogeneous datasets: each local site has different
attributes but linking between the clusters depends on a
common key attribute.
According to Zhao et al., [22]Fast and high quality clustering
algorithms are important for browsing large amount of
information. We can use hierarchal clustering approaches like
partitioned and agglomerative clustering. By experimental
evaluation it was seen that partitional algorithm is always better
than agglomerative algorithms due to relative low
computational requirements and clustering performance. In
[17]authors have presented a new class of clustering algorithm
which is named as constrained agglomerative algorithms which
is combination of both partitional and agglomerative approach.
In this paper a new clustering algorithm is proposed that is
robust to outliners and deal with clusters non-spherical in
shape.It is a hierarchal clustering algorithm that gives a middle
ground between centroid-based and all point methods.It deals
with all short-comings of centroid based and all point method.
Authors have provided a comparison of BIRCH algorithm with
CURE.To handle large databases this algorithm uses random
partitioning and sampling. Clustering algorithms are used in
many applications like speech and character recognition, image
segmentation, vector quantization, etc[21].
The main drawback of the existing clustering algorithms is the
random selection of initial centroids and their ability to deal
with continuous arrival of data. This paper presents a novel
2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom)
classification data from the distributed nodes.If the traditional
clustering algorithms are applied in distributed databases then
it requires the transfer of all the datasets to the central site but
it is impractical due to the presence of large dataset at each
local site as well as due to the limited bandwidth and privacy
concerns. Distributed Clustering is divided into two:
a) Hard Clustering: In hard clusters (crisp datasets) each data
object can exist only in one cluster i.e. the clusters are disjoint.
Example: PAM, K means Clustering.b) Soft Clustering: In
soft clusters (fuzzy datasets) every data object belongs to
every cluster. Example: Neural Networks and Fuzzy
Clustering
Particle Swarm Optimization (PSO) based unsupervised
clustering algorithm for clustering of students for continuously
arriving data. It is a population based stochastic optimization
technique, which has been modeled based on the swarm of
particles. Authors have described the PSO algorithm, this
algorithm is used to cluster students based on 3 sample factors
– efficiency, accuracy and error count.
III. CLUSTERING ALGORITHM TECHNIQUES
Clustering algorithms are classified[1] according to:
• The type of input
• Clustering criterion defining the similarity between the
objects
• Concepts on which clustering analysis techniques are
based (e.g. fuzzy theory, numerical data, categorical data
etc).Several clustering algorithms along with their
application areas are shown in Fig. 3.1.
A.Partitional Clustering
In Partitional clustering, clusters are represented by the
prototype and we use the iterative controstrategy to optimise
the clustering. The partitioning algorithms divide the data sets
into various subsets called partitions and each partition
represents a cluster. The clusters formed have the following
characteristics:
• Eachclustermusthaveatleastoneobject
• Each object must be a part of exactly one clusters i.e. no
overlapping
411
Partitioning techniques are divided into two types:
• Medoid Algorithms: Each cluster contains the instances
that are closest to the gravity centre
• Centroid Algorithms:gravity centre of the instances is used
to represent each cluster. Example: K –Means clustering
technique, where the data set is partitioned into k subsets
in such a manner that all points in a given subset are
closest to the same gravity centre. The effectiveness of k-
means technique depends on the objective function that is
being used to calculate the distance between the instances.
K- Means technique has a requirement that all the data
must be available prior.
Time Complexity:O(nkl)where n is the number of patterns, k
is the number of clusters and l is the number of iterations taken
to converge. Generally, the value of k and l is fixed prior to
attain the linear time complexity.
Space Complexity: O(k+n) and addition storage is required for
storing the data.
B. Hierarchical Clustering
A tree of clusters is called dendogram is constructed depending
on the medium of proximity in Hierarchical technique. Each
cluster node contains other nodes called child nodes and the
nodes from same parent are called sibling nodes. Hierarchical
techniques have a property of quick termination. Example of
Hierarchical clustering include: CURE, BIRCH,
CHAMELEON etc. Hierarchical techniques are divided into
two categories:
• Agglomerative Method:It works in bottom- up fashion i.e.
groups all data instances until they belong to same cluster.
Closest pair of clusters is merged together. Closeness is
defined as single- link, complete- link and average- link.
• Single- link closeness: It is defined as the similarity
between the two instances, both of which are in separate
clusters. Clusters ofelliptical shapes are handled efficiently
but it is sensitive to noise and errors.
Fig.1. Clustering algorithms and their applications
412 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom)
• Complete- link closeness: It is the similarity among
the most dissimilar instances each belonging to separate
cluster. It is not suitable for convex shaped clusters, but is
less sensitive to noise and errors or outliers.
a.) Divisive Method: It works in top-down fashion in
which data sets are split into small clusters until each
cluster consists of only one data instance.
Time ComplexityO(n2logn)where n is the number of patterns
Space Complexity: O(n2)
B. Density Based Clustering (DBSCAN)
In DBSCAN, different distance metrics are used and number
of clusters is determined automatically by the algorithm. Data
objects are separated depending on the connectivity, boundary
or their region. In DBSCAN the data points either belong to
any cluster or are classified as noise.
The data points are either the core points, border points or
noise points.
• Core points: Points that lie inside the cluster are
called as core points. A point is considered to be
inside the cluster if the number of data points in the
neighbourhood exceeds a certain threshold value.
• Border points: Points that are not the core points
but they lie in the neighborhood of core points.
• Noise points: Point that is neither a core point and
nor a border point.
Time Complexity:O(m* time taken to find points in the
neighbourhood)
Space Complexity: O(m)
C. Neural Networks
Neural networks are non- linear data modelling tools that
simulate the working of brain. They are used to identify the
relationship between the patterns depending on the input and
output values.
 D. Fuzzy Clustering Grid based techniques are mostly used in spatial data
mining. They divide the space into finite number of cells
Fuzzy clustering is a reasoning based technique in which
and all the operations are then performed on the quantised
associations among the patterns and the clusters is done on
space.
the basis of membership functions. Fuzzy clustering
generates overlapping clusters.
IV.COMPARISON OF CLUSTERING ALGORITHMS
E. Grid Based Clustering
In this section we have analyzed various clustering algorithms
and represented in Table I.
TABLE I. ANALYSIS ON CLUSTERING ALGORITHMS

Authors Algorithm Clustering Dataset Advantages Disadvantages Complexity

Category
Dalal, Harale k- Means Clustering Partitioning Numerical data Large datasets are processed Sensitive to noise, O(n)
(2011) Method (crisp data set) easily, Simple to implement Depends on initial value of
and results are easy to k, poor locally optimal
interpret solutions
Dhillon, K- Means Parallel Partitioning crisp and Computation time is reduced Do not support
Modha Implementation Method homogeneous by a factor of p(no. of heterogeneous datasets
(2002) datasets processors)
Vidya, Privacy preserving K- Partitioning Crisp data sets Privacy and security Do not support
Clifton means clustering over Method heterogeneous datasets
(2003) vertical partitioning
data
Moore, Hall DCA Partitioning Both crisp and Random division and Complex due to the
(2004) Method fuzzy dataset distribution of data set to local collision in centroid
site mapping
Visalakshi, MDCA Partitioning Crisp and Useful in non uniform schema
Thangavel, Method heterogeneous
Parvathi dataset
(2008)
Coletta, PFCM Fuzzy Method Fuzzy and Overlapping clusters are Data is of similar type i.e. O(n)
Vendramin, homogeneous formed due to usage of homogeneous
Hruschka, dataset membership function, Number
Pedrycz of clusters are not to be
(2012) defined prior
Ghanem, IFDFC Fuzzy Method Fuzzy and Privacy and easy to use and Homogeneous datasets
Kechadi, Tari homogeneous implement
(2011) dataset
Xu, Jager, Parallel DBSCAN Density based Crisp and Excellent scalability and noise O(nlogn)
Kriegel method heterogeneous is differentiated efficiently
(1999) data sets
Januzaj, SDBDC Density based Crisp and Privacy Threshold identification is
Kriegel, method heterogeneous difficult
Pfeifle data sets
(2004)
Klusch, Lodi, KDEC Density based Crisp and Privacy Do not support
Moro method homogeneous heterogeneous datasets
(2003) data sets
Khac, DDC Density based Crisp and Used in arbitrary shaped
Aouad, method heterogeneous clusters, deals with noise and
Kechadi data sets errors
(2007)
Ghanem, OPTICS Density based Crisp and Communication overhead is
Kechadi, Tari method heterogeneous less
(2011) data sets
Yi-Hong Lu, Scalable K Means Clustering by At most one scan of the data is Compression schemes
Yan Huang Clustering point method required used cause a large
(2005) overhead
Yi-Hong Lu, Statistical Grid based Threshold Memory save, No nearest Assumption that initial
Yan Huang Clustering based neighbour identification clusters have data as a line,
(2005) problem in high dimensional does not work well when
data, capable of finding dimensionality of data is

2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom) 413
 clusters of arbitrary shape large
Yi-Hong Lu, Regression Analysis Mathematical Does not depend on initial
Yan Huang Model values or criteria to combine
(2005) clusters
Kauffman PAM Partitional Numerical Expensive due to the O(k(n-k)2)
Method dataset comparison of object with
entire dataset
Kyuseok BIRCH Hierarchical Numerical Handles noise easily Order sensitive O(n)
Shim, based dataset
Rajeev
Rastogi
(1998)
Sudipto CURE Hierarchical Numerical Handles large databases and O(n2logn),O
Gupta, based dataset improves quality (n)
Rajeev
Rastogi
(1998)
Deepika STING Grid based Spatial data set Arbitrary shaped clusters, O(k), k is
Singh handles noise efficiently number of
(2013) grid cells at
lowest level
Particle Swarm Unsupervised Categorical data Clusters are formed on the Real time and continuous
optimization clustering (students data) basis of three parameters i.e. data collection is needed
quality, efficiency and
accuracy
Tzung Hong Evolutionary Genetic Real world speeds up classification time, Large no of attributes
Composite Attribute Algorithm datasets (fuzzy reduces cost and good
Clustering data) performance
Daniel pyGCluster Hierarchical Overall data size is reduced
Jaeger, based
Johannes
Barth
(2014)
V. CONCLUSION Transactions on Evolutionary Computation, Vol. 18, No. 1,
This paper discusses the various clustering algorithms like February 2014 ,pp. 20-35
partitional clustering, hierarchical clustering, density based [4] Q. Liu, W.Jin, S.Wu, Y. Zhou, Clustering Research using
clustering; Grid based clustering and their time and space dynamic modelling based on granular computing, IEEE,2005,
pp. 539 - 543
complexities. [5] M. Halkidi, I. Koutsopoules, Online Clustering of Distributed
Partitional clustering represents the clusters using a prototype Streaming Data using Belief Propagation Techniques, IEEE
generally. Partitional clustering algorithms are very useful 2009, pp. 216 - 225
when the clusters are of convex shape having similar size and [6] Barhate Sachin R., Shelake Vijay M., A Survey and Future
the number of clusters can be identified prior. Vision, Second International Conference on Advanced
Due to the disability in predicting the number of clusters in Computing and Communication Technologies, IEEE 2012, pp.
advance Hierarchical clustering algorithms are used. They 96 – 100
divide the dataset into several levels of partitioning called [7] A. Fahad, N. Alshatri, Z. Tari, A. Alamri, A. Zomaya, I.
dendograms. These algorithms are very effective in mining but Khalil, S. Foufou, A. Bouras , A survey of Clustering
Algorithms for Big Data: Taxonomy and Empirical Analysis,
the cost of formation of dendograms is very high for large IEEE 2014,267 - 279
datasets. [8] Y. Hong Lu, Y. Huang, Mining Data Streams using Clustering,
Density based clustering techniques are very useful in mining Proceedings of the Fourth International Conference on
large datasets because they can easily identify noise and can Machiene Learning and Cybernetics, Guangzhou, 18-21
deal with clusters of arbitrary shape. August, 2005, IEEE 2005,pp. 2079 - 2083
[9] Deepika Singh, Anjana Gosain, A Comparative Analysis of
REFERENCES Distributed Clustering Algorithms: A survey, 2013
[1] M. Halkidi, Y. Batistakis, M. Vazirgiannis, Clustering International Symposium on Computational and Business
algorithms and validity measures, IEEE, 2001, pp.3-22 Intelligence, IEEE 2013, pp. 165 - 169
[2] Mukhopadhyay, S. Bandyopadhyay, Survey of Multiobjective [10] R. H. Sheikh, M. M. Raghuwanshi, A. N. Jaiswal, Genetic
Evolutionary Algorithms for Data Mining: Part I, IEEE Algorithm Based Clustering: A Survey, First International
Transactions on Evolutionary Computation, Vol. 18, No. 1, Conference on Emerging Trends in Engineering and
February 2014,pp.4-19 Technology, IEEE 2008, pp. 314 - 319
[3] Mukhopadhyay, S. Bandyopadhyay, Survey of Multiobjective
Evolutionary Algorithms for Data Mining: Part II, IEEE

414 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom)
 [11] S. Dhillon, Y. Guan, J. Kogan, Iterative Clustering of High
Dimensional Text Data Augmented by Local Search, ICDM ,
IEEE 2002,pp. 131 - 138
[12] M. A. Dalal, N D Harale, A survey on Clustering in data
mining, International Conference and Workshop on Emerging
Trends in Technology, TCET, Mumbai, India, ACM 2011, pp.
559-562
[13] Lin, E. Keogh, W. Truppel, Clustering of Streaming Time
Series is Meaningless, ACM, 2003, pp. 56-65
[14] X. Wu, V. Kumar, J. Ross Quinlan, J. Ghosh, Q. Yang, H.
Motoda, G. J. McLachlan, A. Ng, B. Liu, P. S. Yu, Z. H. Zhou,
M. Steinbach, D. J. Hand, D. Steinberg, Top 10 algorithms in
Data Mining: A survey, Springer, 4 December 2007
[15] D. Jaeger, J. Barth, A. Niehues, C. Fufezan, pyGCluster: A
novel hierarchical clustering approach, Vol. 30 no. 6 2014, pp.
896–898
[16] D. R. Cutting, D. R. Karger, J. O. Pedersen, J. W. Tukey, Scatter/
Gather: A Cluster based approach to browsing large document
collections, ACM 1992, pp. 318-329
[17] S. Guha, R. Rastogi, K. Shim, CURE: An efficient Clustering
Algorithm for Large Databases, ACM 1998,pp. 73-84
[18] M. S. Chen, J. Han, P. S, Yu, Data Mining: An Overview from
a Database Perspective, IEEE Transactions on Knowledge and
Data Engineering, Vol. 8, No. 6, December 1996, pp. 866 – 883
[19] R. T. Ng, J. Han, CLARANS: A Method for Clustering objects
for Spatial Data Mining, IEEE Transactions on Knowledge and
Data Engineering, Vol. 14, No. 5, September/October 2002, pp.
1003 – 1016
[20] M. Ester, H. Kriegel, J. Sander, X. Xu, A Density- Based
Algorithm for discovering clusters in large spatial databases
with noise, KDD-96 Proceedings, pp. 226-231
[21] Pavel, Survey of clustering Data mining Techniques, Accure
software,Inc.
[22] Y. Zhao, G. Karypis, Evaluation of Hierarchical Clustering
Algorithms for Document Datasets, ACM 2002, pp. 515-524.

2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom) 415

View publication stats