Ijaiem 2019 12 06 3

International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org

Volume 9, Issue 1, January 2020 ISSN 2319 - 4847
Deep Learning based Optimization of

Extended Topological Active Net for Multi
Object Segmentation
Pramila B1 Dr. M B Meenavathi2
1
Associate Professor, Department of Electronics and Communication Engineering,
East West Institute of Technology, Bangalore, Karnataka, India
2
Professor and Head, Department of EIE, Bangalore Institute of Technology, Bangalore,
Karnataka, India
Abstract— Segmenting objects in image is an important task in many computer vision based applications.
Activenets are a popular segmentation method which partitions the objects by creating holes in mesh to fit the object
within the mesh. Different types of Activenets have been proposed in literature based on the optimization methods
used for fitting the objects within mesh. Topological ActiveNets (TAN) and its extension Extended Topological
ActiveNets (ETAN) are two popular Activenets applying energy based optimization. The problem in use of ETAN
for the case of complex images is that it often leads to local optima. In this work, a deep learning based optimization
is proposed to improve the ETAN and prevent it from the local optima problem. By this way, segmentation accuracy
is improved.
Keywords: TAN, ETAN, Deep learning and Objectiveness.
I. Introduction
Image segmentation is an image processing operation used to partition the image into multiple segments. Segmentation
is the critical step in many computer vision and diagnosis applications. The success of these applications depends on
the accuracy of segmentation. Towards this end many segmentation methods have been proposed. ActiveNets is one
such technique for image segmentation. ActiveNets segmentation is based on the concept of Deformable models (DM)
[1]. It is method of placing the surface like mesh over image and deform the surface to fit the object segments in the
image. Forces both internal and external guide the deformation process. Internal, external forces guide the smoothness
and orientation towards the objects of interest respectively.Topological Active Net (TAN) and Extended Topological
Active Net (ETAN) are two geometric deformable models applying energy based functions to fit the 2D surface mesh
over the image. The holes in the mesh are created to fit the objects of interest. TAN works by placing the mesh over
the entire image and apply energy minimization on each node in mesh to deform it and fit to the objects. Greedy search
is done to find the optimal solution. Energy of node and its neighborhood in its current position is computed at each
step. Node moves to position of lowest energy value. The algorithm stops when there nodes cannot be moved further.
TAN mesh fails for the cases of complex shapes especially concavities. It is because of restriction in the TAN regarding
the links. Links cannot be deleted and links has to follow shape constraints in TAN. TAN problems are solved using
ETAN. ETAN proposes a change in energy term compared to TAN which achieves better optimization
G(v(m,n)))+ФDGevfc(p) (1)
Where
For certain shapes, the mesh links need to be cut. ETAN cuts the links based on energy threshold. The links energy is
calculated as
/ |A| (2)
Volume 9, Issue 1, January 2020 Page 5

A is area for which energy is computed and p is the pixel in that area. The original image is given as I and the
maximum intensity value is given as The cutting threshold is fixed as the mean energy of the links. Holes are
identified starting from misplaced internal nodes. For every internal node r(n) is calculated as
(3)
Where Eext is the external energy and Eint is the internal energy of the node n. The node with highest ratio is selected
and if its r value is higher then threshold for holes, a hole in opened starting from this node. The energy for all links
starting from this node and to the neighbors are calculated and highest one is removed. Like this way more links can be
deleted from the mesh and adjusted to the edge of the object.
ETAN optimization can be enhanced further to reduce the time complexity and increase segmentation accuracy for
complex images. In our previous work [1], fuzzy rule based learning is used for optimization of ETAN. The accuracy
obtained using fuzzy rule based learning can be further improved using Deep learning methods. Of the various machine
learning models Deep learning is the most recent with ability to learn unsupervised model in an automatic manner
without user explicitly giving the features. This ability of Deep learning has improved accuracy in classification.
Among multiple Deep Learning algorithms, Convolutional Neural Networks (CNN) is increasingly used for
classification. In this work CNN is used for optimizing the ETAN and increase the segmentation accuracy.
II. Related Work

A survey on TAN and ETAN optimizations is presented in this section.
Authors in [3] applied genetic optimization to improvise TAN. A novel mutation operation was introduced in this
work, which was able to generate mesh models with fewer or no cross links among mesh nodes. The work was able to
reduce the impact of local minima in noisy images. Similar to it genetic algorithm based optimization with local search
based mutation was proposed in [4]. Though it performed better than [3] in terms of segmentation, the computational
complexity is very high compared to [3]. An evolutionary multi objective optimization for TAN was proposed in [5].
Region-based and boundary-based segmentation features are integrated with energy optimization methods and the
optical parameters for fine tuning it is achieved using parental optimality of SEA algorithm. Scatter search based
optimization for ETAN was proposed in [6]. This work introduced a new energy terms to improve the segmentation
accuracy of ETAN. The computational complexity is higher in this approach compared to GA optimization.
Differential evolution based optimization of TAN was proposed in [7]. The search directions were fine tuned using
historical information. Due to most promising neighbor regions for each node are identified. Authors in [8] applied
artificial neural networks (ANN) for optimization of TAN. The movement of nodes in mesh is used to train a ANN.
This trained ANN is used to decide the movement of nodes in the mesh in steps till final segmentation is achieved.
ANN method still suffered from computational complexity to multitudes of steps involved. A hybrid algorithm for TAN
mesh optimization was proposed in [9]. Genetic and greedy algorithm was combined for automatic division of TAN
mesh in this work. The active net was partitioned into multiple subnets with each of subnets covering an object in the
image. The problem in this approach is that it could not adjust its mesh structure for the concavities in the image.
Genetic greedy algorithm for optimization of ETAN was proposed in [10]. Both of two algorithms dual surface
minimization (DSM) and hybrid real coded genetic algorithm (GAGR) proposed in this work are able to extract the
surface from noise volumetric image. But the methods were not able to segment multiple objects. Addressing the
problems of initialization, pose estimation and local minima, Genetic algorithm based optimization was applied to
deformable models in [11]. Search space reduction was realized using statistics based deformation. The computational
complexity was very high in approach due to parallel evolution of multiple models and fusion at last stage. Three new
models were proposed in [12] to solve the local minima issue in active net segmentation. These models used dual
formulation in solving the problem of active contour propagation. All the three models had reduced time complexity
compared to GA based methods but this approach fails for complex shapes.
Authors in [13] proposed a block evolution based active contour optimization. The solution is more robust as it
confirms to contour maintenance and deformation of adjacent curves. A segmentation algorithm using multi stage
active contour is proposed in [14]. The method is particularly suited for gray scale medical images which are
characterized by multiple regions of different mean intensity value. The approach created multiple contour using Vese
and Chan’s global convex formulation and dual minimization scheme. The approach could work only for gray scale
images. Authors in [15] proposed an active contour model based on higher order diffusion.
The method worked well for segmentation of images with homogenous values for intensities. It used gradient Laplace
information to converse edges and due to this it worked well for images with weak edges for the objects.
The existing solutions were found to have following problems
1. Not working for all kinds of images
2. Computational complexity is high
3. Local minima problem
The work proposes deep learning based optimization with a motivation to solve all the problems addressed above.
III. DEEP LEARNING OPTIMIZATION FOR ETAN

The proposed Deep learning based optimization involves two steps of Convolutional neural network (CNN) based
subneting and ETAN based correction of subnet. Due to reduction in area of application of ETAN, the computation
time is reduced and due to application of CNN the segmentation accuracy improves. The overall flow of the proposed
solution is given below:
Figure 1 Flow in Proposed Solution
CNN based Subneting

A training dataset of images (marked with active net grids) of size N N and its corresponding active subnet matrix as
the class is prepared using BSDS500dataset. The subnet is represented is a binary matrix with 1 indicating the
presence of link of the mesh and 0 indicating the absence of link. The goal is to train a CNN network that learns the
features from the input to produce the active subnet matrix close to the ground truth.
Among the different CNN network architecture, holistically nested architecture is used in this work due to its capability
to produce multiple outputs and a fusion is done at end to get the unified output.
Figure 2 Nested architecture

This architecture is chosen for the following advantages

1. Multi level features can be generated with higher efficiency
2. Presence of multiple stages allows to learn different scales in active subnets.
There are M output layer with each associated with a classifer in this network. The weights of these layers are denoted
as .The network must be designed in such a way to maximize the below objective function
Where W – collection of all network layer parameters and is the image level loss function for side outputs. Loss
function is computed over all pixels in the training image and active subnet. In this we use the lost function proposed in
[18] with minor revisions due to its cost effectiveness. The modified loss function is given as
(4)
Where
is the links in the active subnet

is the gaps in the active subnet
is computed as
sigmoid function( on the activation value at pixel at
j.
At each side output layer, active subnet matrix is given
as output. These outputs are then added using
weighted fusion layer. The weights needed for fusion
is found during training. The loss function at fusion
layer is given as
(5)
Where
Where ) is the fusion weight.

is the distance between the final prediction
and the ground truth label matrix. With all this , the
final goal of the nested architecture is minimization of
the objective function defined below
This minimization can be achieved using back

propagation based stochastic gradient descent.
VGGNet proposed in [19] is used for realization of
the classifier in each of the M output layer of nested
architecture with few modifications.
Figure 3 Modified VGGNet
The original VGGNet network has 13 conv layers and fully connected layer of size three. At convololution stage there
are five stages. At each stage of convolution a pooling layer is connected after convolution. Due to this architecture ,
the information captured becomes coarser. Following modifications are done to VGGNet in this work.

 Fully connected layers are cut and replaced with fully convolutional layer. Also the pooling at layer 5 is
removed to increase the stride by two times and compensates for edge localization.To attain hybrid features the
convolutional layer is set to have a kernal of size 1 × 1 and channel depth 21. The resulting feature from each
covolution are accumated with an additional layer.
 The feature map is up sampled at these additional layer with a convolutional layer of size 1×1.
 At upsamling layers , cross-entropy loss/sigmoid layer is attached
 The fusion loss/out is obtained by aggregating all the up-sampling layer output from each stages and passing to
cross entropy loss/sigmoid layer.
ETAN based correction

Given an image to segment, the mesh is laid over whole image and passed to CNN based subneting. The result from it
is a mesh with holes for all background and mesh nodes are only on the objects. Some mesh nodes and links do cross
the boundary, which must be detected and removed. This is achieved by ETAN energy minimization in each of the sub
mesh with following energy minimization function.
(6)
The overall process of optimization is given below
Figure 4 Overall Process
Instead of doing submesh optimization on all subnets , it can be made user interactive with user selective the subnets of
interest and the optimization can be done only on those subnets of interest.

IV. RESULTS
The proposed solution was tested against images of different categories and for comparison; the original ETAN
(without any optimization) and ETAN-EBLIS and Fuzzy optimization [1] are used. For each image to be tested a
ground truth image in terms of expected segmentation result is created and structural similarity between the segmented
result and the ground truth image is done.
The performance is measured in terms of two parameters

1. Spatial Accuracy Index (SAI)
2. Structural Similarity Index (SSIM)
3. Segmentation time
4. Hausdorff distance
Spatial Accuracy Index is calculated as

(7)
Where segmentation result is denoted as R and ground truth is given as T and is the cardinality of X which is
measured in terms of number of pixels it contains.
Structural Similarity Index is calculated as

(8)
Where
Mean of a
Variance of a
Co variance of a,b
a Segmented result(Actual result)
b Ground truth (Expected result)
Segmentation time is the time taken for segmentation.

Hausdorff distance is the maximum distance between two contours
(9)
It measures the worst case of segmentation. Lower value of indicates higher effectiveness of segmentation.
The results of segmentation for different category of images are given below
Figure 5 Result Comparison 1

The three solutions of Proposed, ETAN and ETAN-BLIS were tested against 20 images each from the following
dataset.
1. MSRA-1000 dataset [16]
2. CSSD dataset [17]
MSRA dataset is selected for its simple and image with versatile content and background. CSSD dataset has images
with high structural complexity. Segmented results obtained using the proposed solution is evaluated against the
ground-truth images c.

Figure 8 Performance Comparison
Table 1 Performance Results
Solution SAI SSIM Avg Segmentation Hausdorff

time distance
Proposed 0.92 70.42 100 ms 10
Solution
Fuzzy 0.85 96.28 133 ms 18
Optimization
ETAN 0.6 201.15 240 ms 37
ETAN-BLIS 0.72 130.17 170 ms 29
From the results, it can seen that SAI value is higher in the proposed solution when compared to Fuzzy optimization,
ETAN and ETAN-BLIS, demonstrating the closeness of segmentation result with ground truth in case of proposed.
SSIM value is lower in the proposed solution as the segmentation boundaries are close to ground truth.
The segmentation time is reduced in proposed deep learning optimization because of sub mesh provided by deep
learning and reduction in space of application of ETAN to those sub mesh only.
The Hausdorff distance is lowest in the proposed demonstrating the effectiveness of segmentation.
The results of segmentation accuracy for the MSRA-1000 dataset is given below
Figure 9 Accuracy for MSRA Dataset
The results of segmentation accuracy for the CSSD dataset is given below

Figure 10 Accuracy for CSSD Dataset
The precision is higher in the proposed deep learning based optimization compared to fuzzified optimization[1],
ETAN-BLISS and ETAN.
V. CONCLUSION
In this work, a deep learning based approach consisting of CNN based submesh creation and ETAN based submesh
creation is proposed. Due to reduction of mesh to sub mesh based on CNN based subneting, the subsequent ETAN
process is optimized in terms of running time and accuracy of segmentation. The comparison results with fuzzified
ETAN optimization, original ETAN and ETAN-EBLIS proves the effectiveness of proposed Deep learning based
optimization. The reduction in running time and the increase in accuracy with the ground truth values is demonstrated
for various categories of images. The work present sufficient scope for parallelism as each of subnet optimization using
ETAN energy minimization can be done in parallel after the CNN based subnet creation. By this way the clustering
time is reduced.
REFERENCES
[1] Pramila.B, Dr. M B Meenavathi "Fuzzy and Objectiveness Integrated Optimization of Extended Topological
Active Net for Multi Object Segmentation", 2019
[2] M. Bro-Nielsen, “Active nets and cubes,” IMM, Tech. Rep., 1994.
[3] O. Ibanez, J.Santos, and M. G. Penedo, “Genetic approaches for topological active nets optimization,” Pattern
Recognition., vol. 42, no. 5, pp. 907–917, 2009.
[4] J. Novo, J. Santos, “Topological active models optimization with differential evolution,” Expert Syst. Appl., vol.
39, no. 15, pp. 12165–12176, 2012.
[5] J. Novo, M. Penedo, “Evolutionary multiobjective optimization of topological active nets,” Pattern Recogniton.,
Lett., vol. 31, no. 13, pp. 1781–1794, 2010.
[6] N. Bova, O. Cordón, “Image segmentation using extended topological active nets optimized by scatter search,”
IEEE Comput. Intell. Mag., vol. 8, no. 1, pp. 16–32, Feb. 2013.
[7] Li, Yuan-Long ,Qing. “.Fast Micro-Differential Evolution for Topological Active Net Optimization” IEEE
transactions on cybernetics. 46. 10.1109/TCYB.2015.2437282.
[8] Cristina V. Sierra, José Santos "Emergent Segmentation of Topological Active Nets by Means of Evolutionary
Obtained Artificial Neural Networks" In Proceedings of the 5th International Conference on Agents and Artificial
Intelligence (ICAART-2013).
[9] N. Barreira,M. J. Santos "Automatic Topological Active Net Division in a Genetic-Greedy Hybrid Approach":
Pattern Recognition and Image Analysis 2007.
[10] Tohka, J.: “Global optimization of deformable surface meshes based on genetic algorithms.” In: Proceedings
ICIAP, pp. 459–464. IEEE Computer Society Press, Los Alamitos (2001).
[11] C. McIntosh, G. Hamarneh, "Medial-based deformable models in nonconvex shape-spaces for medical image
segmentation", IEEE Trans. Med. Imag, vol. 31, no. 1, pp. 33-50, 2012.
[12] X. Bresson, J.-P. Thiran, and S. Osher, “Fast global minimization of the active contour/snake model,” J. Math.

Imag. Vis., vol. 28, no. 2, pp. 151–167, Jun. 2007.

[13] Guoqi Liu , Haifeng Li, “Robust Evolution Method of Active Contour Models and Application in Segmentation of
Image Sequence,” Journal of Electrical and Computer Engineering, vol. 2018, 11 pages, 2018.
[14] Juan C. Moreno,V. B. S. Prasath "Brain MRI Segmentation with Fast and Globally Convex Multiphase Active
Contours" arXiv,28 Aug 2013.
[15] Guodong Wang , Jie Xu, "Active Contour Model Coupling with Higher Order Diffusion for Medical Image
Segmentation",Int J Biomed Imaging. 2014.
[16] Radhakrishna Achanta, Francisco Estrada, and Sabine Süsstrunk, “Frequency-tuned salient region detection,”
2009, pp. 1597–1604.
[17] Qiong Yan, Li Xu, “Hierarchical saliency detection,” in IEEE Conference on Computer Vision and Pattern
Recognition, 2013, pp. 1155–1162
[18] J.-J.HwangandT.-L.Liu. Pixel-wisedeeplearningforcontour detection. In ICLR, 2015
[19] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR,
2015
AUTHORS
Pramila B is research Scholar from Visvesvaraya Technological University, Belgaum pursing at,
Department of EIE, BIT, Bangalore-04 and working as Associate Professor, Department of Electronics
and Communication Engineering, East West Institute of Technology , Bangalore, Karnataka , India.
Dr M.B. Meenavathi, Prof. & Head, Department of EIE, BIT, Bangalore-04, has 29 years of Teaching
experience and has 14 publications. She is a member of IMSTE and IAENG. Her area of interest is
Image processing, signal processing and fuzzy logic.

Ijaiem 2019 12 06 3

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Ijaiem 2019 12 06 3

Hochgeladen von

Copyright:

Verfügbare Formate

International Journal of Application or Innovation in Engineering & Management (IJAIEM)

Web Site: www.ijaiem.org Email: editor@ijaiem.org

Deep Learning based Optimization of

Volume 9, Issue 1, January 2020 Page 5

II. Related Work

III. DEEP LEARNING OPTIMIZATION FOR ETAN

Figure 1 Flow in Proposed Solution

CNN based Subneting

Figure 2 Nested architecture

Volume 9, Issue 1, January 2020 Page 7

This architecture is chosen for the following advantages

is the links in the active subnet

Where ) is the fusion weight.

This minimization can be achieved using back

Figure 3 Modified VGGNet

Volume 9, Issue 1, January 2020 Page 8

ETAN based correction

The overall process of optimization is given below

Figure 4 Overall Process

Volume 9, Issue 1, January 2020 Page 9

The performance is measured in terms of two parameters

Spatial Accuracy Index is calculated as

Structural Similarity Index is calculated as

Segmentation time is the time taken for segmentation.

Figure 5 Result Comparison 1

Volume 9, Issue 1, January 2020 Page 10

Figure 6 Result Comparison 2

Figure 7 Result Comparison 3

Volume 9, Issue 1, January 2020 Page 11

Figure 8 Performance Comparison

Table 1 Performance Results

Solution SAI SSIM Avg Segmentation Hausdorff

Figure 9 Accuracy for MSRA Dataset

Volume 9, Issue 1, January 2020 Page 12

Figure 10 Accuracy for CSSD Dataset

Volume 9, Issue 1, January 2020 Page 13

Imag. Vis., vol. 28, no. 2, pp. 151–167, Jun. 2007.

Volume 9, Issue 1, January 2020 Page 14

Das könnte Ihnen auch gefallen