Chapter 1

Introduction to ANS Technology 1

1.1 Elementary Neurophysiology 8

1.2 From Neurons to ANS 17

1.3 ANS Simulation 30

Bibliography 41

Chapter 2

Adaline and Madaline 45

2.1 Review of Signal Processing 45

2.2 Adaline and the Adaptive Linear Combiner 55

2.3 Applications of Adaptive Signal Processing 68

2.4 The Madaline 72

2.5 Simulating the Adaline 79

Bibliography 86

Chapter 3

Backpropagation 89

3.1 The Backpropagation Network 89

3.2 The Generalized Delta Rule 93

3.3 Practical Considerations 103

3.4 BPN Applications 106

3.5 The Backpropagation Simulator 114

Bibliography 124

Chapter 4

The BAM and the Hopfield Memory 727

4.1 Associative-Memory Definitions 128

4.2 The BAM 131

4.3 The Hopfield Memory 141

4.4 Simulating the BAM 156

Bibliography 167

Chapter 5

Simulated Annealing 769

5.1 Information Theory and Statistical Mechanics 171

5.2 The Boltzmann Machine 179

5.3 The Boltzmann Simulator 189

5.4 Using the Boltzmann Simulator 207

Bibliography 212

Chapter 6

The Counterpropagation Network 273

6.7 CPN Building Blocks 215

6.2 CPN Data Processing 235

6.3 An Image-Classification Example 244

6.4 the CPN Simulator 247

Bibliography 262

Chapter 7

Self-Organizing Maps 263

7.7 SOM Data Processing 265

7.2 Applications of Self-Organizing Maps 274

7.3 Simulating the SOM 279

Bibliography 289

Chapter 8

Adaptive Resonance Theory 297

8.1 ART Network Description 293

8.2 ART1 298

8.3 ART2 316

8.4 The ART1 Simulator 327

8.5 ART2 Simulation 336

Bibliography 338

Chapter 9

Spatiotemporal Pattern Classification 347

9.7 The Formal Avalanche 342

9.2 Architectures of Spatiotemporal Networks (STNS) 345

9.3 The Sequential Competitive Avalanche Field 355

9.4 Applications of STNS 363

9.5 STN Simulation 364

Bibliography 371

Chapter 10

The Neocognitron 373

10.1 Neocognitron Architecture 376

10.2 Neocognitron Data Processing 381

10.3 Performance of the Neocognitron 389

10.4 Addition of Lateral Inhibition and Feedback to the

Neocognitron 390

Bibliography 393