Fundamentals of Computational Intelligence

Provides an in-depth and even treatment of the three pillars of computational intelligence and how they relate to one another
This book covers the three fundamental topics that form the basis of computational intelligence: neural networks, fuzzy systems, and evolutionary computation. The text focuses on inspiration, design, theory, and practical aspects of implementing procedures to solve real-world problems. While other books in the three fields that comprise computational intelligence are written by specialists in one discipline, this book is co-written by current former Editor-in-Chief of IEEE Transactions on Neural Networks and Learning Systems, a former Editor-in-Chief of IEEE Transactions on Fuzzy Systems, and the founding Editor-in-Chief of IEEE Transactions on Evolutionary Computation. The coverage across the three topics is both uniform and consistent in style and notation.

• Discusses single-layer and multilayer neural networks, radial-basis function networks, and recurrent neural networks
• Covers fuzzy set theory, fuzzy relations, fuzzy logic interference, fuzzy clustering and classification, fuzzy measures and fuzzy integrals
• Examines evolutionary optimization, evolutionary learning and problem solving, and collective intelligence
• Includes end-of-chapter practice problems that will help readers apply methods and techniques to real-world problems

Fundamentals of Computational intelligence is written for advanced undergraduates, graduate students, and practitioners in electrical and computer engineering, computer science, and other engineering disciplines.

Acknowledgments xi
1. Introduction to Computational Intelligence 1
1. 1 Welcome to Computational Intelligence 1
1. 2 What Makes This Book Special 1
1. 3 What This Book Covers 2
1. 4 How to Use This Book 2
1. 5 Final Thoughts Before You Get Started 3
PART I NEURAL NETWORKS 5
2. Introduction and Single-Layer Neural Networks 7
2. 1 Short History of Neural Networks 9
2. 2 Rosenblatt's Neuron 10
2. 3 Perceptron Training Algorithm 13
2. 4 The Perceptron Convergence Theorem 23
2. 5 Computer Experiment Using Perceptrons 25
2. 6 Activation Functions 28
Exercises 30
3. Multilayer Neural Networks and Backpropagation 35
3. 1 Universal Approximation Theory 35
3. 2 The Backpropagation Training Algorithm 37
3. 3 Batch Learning and Online Learning 45
3. 4 Cross-Validation and Generalization 47
3. 5 Computer Experiment Using Backpropagation 53
Exercises 56
4. Radial-Basis Function Networks 61
4. 1 Radial-Basis Functions 61
4. 2 The Interpolation Problem 62
4. 3 Training Algorithms For Radial-Basis Function Networks 64
4. 4 Universal Approximation 69
4. 5 Kernel Regression 70
Exercises 75
5. Recurrent Neural Networks 77
5. 1 The Hopfield Network 77
5. 2 The Grossberg Network 81
5. 3 Cellular Neural Networks 88
5. 4 Neurodynamics and Optimization 91
5. 5 Stability Analysis of Recurrent Neural Networks 93
Exercises 99
PART II FUZZY SET THEORY AND FUZZY LOGIC 101
6. Basic Fuzzy Set Theory 103
6. 1 Introduction 103
6. 2 A Brief History 107
6. 3 Fuzzy Membership Functions and Operators 108
6. 4 Alpha-Cuts, The Decomposition Theorem, and The Extension Principle 117
6. 5 Compensatory Operators 120
6. 6 Conclusions 124
Exercises 124
7. Fuzzy Relations and Fuzzy Logic Inference 127
7. 1 Introduction 127
7. 2 Fuzzy Relations and Propositions 128
7. 3 Fuzzy Logic Inference 131
7. 4 Fuzzy Logic For Real-Valued Inputs 135
7. 5 Where Do The Rules Come From? 138
7. 6 Chapter Summary 142
Exercises 143
8. Fuzzy Clustering and Classification 147
8. 1 Introduction to Fuzzy Clustering 147
8. 2 Fuzzy c-Means 155
8. 3 An Extension of The Fuzzy c-Means 167
8. 4 Possibilistic c-Means 169
8. 5 Fuzzy Classifiers: Fuzzy k-Nearest Neighbors 174
8. 6 Chapter Summary 179
Exercises 180
9. Fuzzy Measures and Fuzzy Integrals 183
9. 1 Fuzzy Measures 183
9. 2 Fuzzy Integrals 188
9. 3 Training The Fuzzy Integrals 191
9. 4 Summary and Final Thoughts 203
Exercises 203
PART III EVOLUTIONARY COMPUTATION 207
10. Evolutionary Computation 209
10. 1 Basic Ideas and Fundamentals 209
10. 2 Evolutionary Algorithms: Generate and Test 216
10. 3 Representation, Search, and Selection Operators 221
10. 4 Major Research and Application Areas 223
10. 5 Summary 225
Exercises 225
11. Evolutionary Optimization 227
11. 1 Global Numerical Optimization 229
11. 2 Combinatorial Optimization 233
11. 3 Some Mathematical Considerations 238
11. 4 Constraint Handling 255
11. 5 Self-Adaptation 258
11. 6 Summary 264
Exercises 265
12. Evolutionary Learning and Problem Solving 269
12. 1 Evolving Parameters of A Regression Equation 270
12. 2 Evolving The Structure and Parameters of Input-Output Systems 274
12. 3 Evolving Clusters 292
12. 4 Evolutionary Classification Models 298
12. 5 Evolutionary Control Systems 307
12. 6 Evolutionary Games 314
12. 7 Summary 320
Exercises 321
13. Collective Intelligence and Other Extensions of Evolutionary Computation 323
13. 1 Particle Swarm Optimization 323
13. 2 Differential Evolution 326
13. 3 Ant Colony Optimization 329
13. 4 Evolvable Hardware 331
13. 5 Interactive Evolutionary Computation 333
13. 6 Multicriteria Evolutionary Optimization 335
13. 7 Summary 340
Exercises 340
References 343
Index 361