SOFT

COMPUTING

DILIP KUMAR PRATIHAR

Alpha Science International Ltd.

Oxford, U.K.
Contents

Preface vii
Nomenclature xv
Greek Symbols xvii
Abbreviations xix
1 Introduction 1
1.1 Hard Computing 1
1.1.1 Features of Hard Computing 1
1.2 Soft Computing 2
1.2.1 Features of Soft Computing 3
1.3 Hybrid Computing 3
1.4 Summary 5
1.5 Exercise 5
2 Optimization and Some Traditional Methods 7
2.1 Introduction to Optimization 7
2.1.1 A Practical Example 9
2.1.2 Classification of Optimization Problems 10
2.1.3 Principle of Optimization 12
2.1.4 Duality Principle 13
2.2 Traditional Methods of Optimization 14
2.2.1 Exhaustive Search Method 14
2.2.2 Random Walk Method 20
2.2.3 Steepest Descent Method 23
2.2.4 Drawbacks of Traditional Optimization Methods 26
2.3 Summary 27
2.4 Exercise 27
3 Introduction to Genetic Algorithms 29
3.1 Working Cycle of a Genetic Algorithm 29
3.2 Binary-Coded GA 31
3.2.1 Crossover or Mutation ? 39
3.2.2 A Hand Calculation 39
3.2.3 Fundamental Theorem of GA/Schema Theorem 41
3.2.4 Limitations of a Binary-Coded GA 43
3.3 GA-parameters Setting 44
 CONTENTS

3.4 Constraints Handling in GA 46

3.4.1 Penalty Function Approach 47
3.5 Advantages and Disadvantages of Genetic Algorithm 48
3.6 Summary 49
3.7 Exercise 50
I
4 S o m e Specialized G e n e t i c A l g o r i t h m s 53 s
4.1 Real-Coded GA 53
4.1.1 Crossover Operators 53
4.1.2 Mutation Operators 57
4.2 Micro-GA 59
4.3 Visualized Interactive GA 59
4.3.1 Mapping Methods 60
4.3.2 Simulation Results 63
4.3.3 Working Principle of the VIGA 65
4.4 Scheduling GA 66
4.4.1 Edge Recombination 67
4.4.2 Order Crossover # 1 69
4.4.3 Order Crossover # 2 70
4.4.4 Cycle Crossover 70
4.4.5 Position-Based Crossover 71
4.4.6 Partially Mapped Crossover (PMX) 72
4.5 Summary 74
4.6 Exercise 74

5 Introduction t o Fuzzy Sets 77

5.1 Crisp Sets 77
5.1.1 Notations Used in Set Theory 78
5.1.2 Crisp Set Operations 79
5.1.3 Properties of Crisp Sets 80
5.2 Fuzzy Sets 82
5.2.1 Representation of a Fuzzy Set 82
5.2.2 Difference Between Crisp Set and Fuzzy Set 87
5.2.3 A Few Definitions in Fuzzy Sets 88
5.2.4 Some Standard Operations in Fuzzy Sets 90
5.2.5 Properties of Fuzzy Sets 97
5.3 Summary 98
5.4 Exercise 98

6 Fuzzy R e a s o n i n g and Clustering 101

6.1 Introduction 101
6.2 Fuzzy Logic Controller 101
6.2.1 Two Major Forms of Fuzzy Logic Controller 102
6.2.2 Hierarchical Fuzzy Logic Controller 116
6.2.3 Sensitivity Analysis 117
6.2.4 Advantages and Disadvantages of Fuzzy Logic Controller 118
6.3 Fuzzy Clustering 118
6.3.1 Fuzzy C-Means Clustering • 118
6.3.2 Entropy-based Fuzzy Clustering 123
6.4 Summary 127
6.5 Exercise 128
CONTENTS xiii

7 F u n d a m e n t a l s of N e u r a l N e t w o r k s 131
7.1 Introduction 131
7.1.1 Biological Neuron 131
7.1.2 Artificial Neuron 132
7.1.3 A Layer of Neurons 135
7.1.4 Multiple Layers of Neurons 136
7.2 Static vs. Dynamic Neural Networks 137
7.3 Training of Neural Networks 137
7.3.1 Supervised Learning 138
7.3.2 Un-supervised Learning 138
7.3.3 Incremental Training 138
7.3.4 Batch Training 138
7.4 Summary 139
7.5 Exercise 139

8 S o m e E x a m p l e s of N e u r a l N e t w o r k s 141
8.1 Introduction 141
8.2 Multi-Layer Feed-Forward Neural Network (MLFFNN) 141
8.2.1 Forward Calculation 143
8.2.2 Training of Network Using Back-Propagation Algorithm 144
8.2.3 Steps to be Followed to Design a Suitable NN 148
8.2.4 Advantages and Disadvantages 149
8.2.5 A Numerical Example 149
8.3 Radial Basis Function Network (RBFN) 152
8.3.1 Forward Calculations 154
8.3.2 Tuning of RBFN Using Back-Propagation Algorithm 156
8.4 Self-Organizing Map (SOM) 159
8.4.1 Competition 160
8.4.2 Cooperation 160
8.4.3 Updating 161
8.4.4 Final Mapping 161
8.4.5 Simulation Results 162
8.5 Recurrent Neural Networks (RNNs) 162
8.5.1 Elman Network 163
8.5.2 Jordan Network 163
8.5.3 Combined Elman and Jordan Network 164
8.6 Summary ,* 165
8.7 Exercise 165

9 Combined G e n e t i c Algorithms: Fuzzy Logic 167

9.1 Introduction 167
9.2 Fuzzy-Genetic Algorithm 167
9.3 Genetic-Fuzzy System 170
9.3.1 A Brief Literature Review 170
9.3.2 Working Principle of Genetic-Fuzzy Systems 173
9.4 Summary 180
9.5 Exercise 180

10 Combined G e n e t i c A l g o r i t h m s : N e u r a l N e t w o r k s 183
10.1 Introduction 183
10.2 Working Principle of a Genetic-Neural System 185
10.2.1 Forward Calculation 186
xiv CONTENTS

10.2.2 A Hand Calculation 189

10.3 Summary 191
10.4 Exercise 191

11 C o m b i n e d N e u r a l N e t w o r k s : Fuzzy Logic 193

11.1 Introduction 193 i
11.2 Neuro-Fuzzy System Working Based on Mamdani Approach 194 ?
11.2.1 Tuning of the Neuro-Fuzzy System Using a Back-Propagation Algorithml99
11.2.2 Tuning of the Neuro-Fuzzy System Using a Genetic Algorithm . . . 200
11.2.3 A Numerical Example 201
11.3 Neuro-Fuzzy System Based on Takagi and Sugeno's Approach 206
11.3.1 Tuning of the ANFIS Using a Genetic Algorithm 209
11.3.2 A Numerical Example 210
11.4 Summary 214
11.5 Exercise 214

References 217

Index 227