A Feature Subset Selection Method based on Ant Colony Optimization and Symmetric Uncertainty

Authors: Syed Imran Ali*, Dr. Wasem Shahzad. (NU-FAST)

* Presenter

Presentation Layout
Introduction
Motivation Background on Feature Selection

Proposed Technique
Ant Colony Optimization Symmetric Uncertainty

Experimentation Conclusion

Motivation
Why Feature Selection?
Curse of Dimensionality Three-fold benefits

Enhance the capabilities of Filter based methods

Data Reduction

Feature Selection Types

PROPOSED TECHNIQUE

Proposed Technique
Basic Ingredients of ACO
Graph Representation Heuristic Desirability Positive feedback process Constraint Satisfaction Solution Construction mechanism

Basic Ingredients of ACO

Graph Representation

Basic Ingredients of ACO

Heuristic desirability and Positive Feedback mechanism

10

Basic Ingredients of ACO

Constraint Satisfaction and Solution Construction

11

Information Theoretic Measure

Information Gain
IG (Y,X) = H(Y) H (Y|X)

Symmetric Uncertainty

12

ACO-SU

EXPERIMENTATION

14

Experimentation Framework
All the experiments are performed using 10-Fold Cross Validation and results of ten runs are averaged. Proposed method is compared with four other feature selection algorithms.

Performance Metrics:
Number of Features Selected. Predictive Classification Accuracy using 10-FCV.

15

Experimentation Framework
SNO. 1 2 3 4 5 6 7 8 9 10 11 12 13 Dataset * Iris Liver Disorder Diabetes Breast Cancer- W Vote Labor Hepatitis Colic-Horse Ionosphere Lymph Dermatology Lung Cancer Audiology Total Features 4 6 8 9 16 16 19 22 34 18 34 56 69 Instances 150 345 768 699 435 57 155 368 351 148 366 32 226 Classes 3 2 2 2 2 2 2 2 2 4 6 3 24

16

Experimentation Framework
Parameters Number of Ants Alpha Beta Evaporation Rate Max. Epochs Path Convergence Threshold Values 20 1 1 0.15 500 50

17

Experiment

18

Experiment

19

Experiment

20

Features Selected by ACO-SU

Dataset
Iris Liver Disorder Diabetes Breast Cancer- W Vote Labor Hepatitis Colic-Horse Ionosphere Lymph Audiology Dermatology Lung Cancer

Total
4 6 8 9 16 16 19 22 34 18 69 34 32

ACO-SU
2 2 2 4 6 6 7 6 9 7 20 12 25

21

No. of Features selected

22

Conclusion
We have proposed an efficient feature selection method based on SI and filter method techniques. Proposed method is extensively experimented over a number of benchmark datasets and classifiers. ACO-SU yields better results as compared to other SI based feature selection methods considered in the study. ACO-SU outperformed other methods in terms of predictive classification accuracy and number of features selected.

23

Thank You
Questions?