Second-Order Methods for Neural Networks
Fast and Reliable Training Methods for Multi-Layer Perceptrons
(Sprache: Englisch)
About This Book This book is about training methods - in particular, fast second-order training methods - for multi-layer perceptrons (MLPs). MLPs (also known as feed-forward neural networks) are the most widely-used class of neural network. Over the past...
Leider schon ausverkauft
versandkostenfrei
Buch
93.99 €
46 DeutschlandCard Punkte
sammeln
- Lastschrift, Kreditkarte, Paypal, Rechnung
- Kostenlose Rücksendung
- Ratenzahlung möglich
Produktdetails
Produktinformationen zu „Second-Order Methods for Neural Networks “
Klappentext zu „Second-Order Methods for Neural Networks “
About This Book This book is about training methods - in particular, fast second-order training methods - for multi-layer perceptrons (MLPs). MLPs (also known as feed-forward neural networks) are the most widely-used class of neural network. Over the past decade MLPs have achieved increasing popularity among scientists, engineers and other professionals as tools for tackling a wide variety of information processing tasks. In common with all neural networks, MLPsare trained (rather than programmed) to carryout the chosen information processing function. Unfortunately, the (traditional' method for trainingMLPs- the well-knownbackpropagation method - is notoriously slow and unreliable when applied to many prac tical tasks. The development of fast and reliable training algorithms for MLPsis one of the most important areas ofresearch within the entire field of neural computing. The main purpose of this book is to bring to a wider audience a range of alternative methods for training MLPs, methods which have proved orders of magnitude faster than backpropagation when applied to many training tasks. The book also addresses the well-known (local minima' problem, and explains ways in which fast training methods can be com bined with strategies for avoiding (or escaping from) local minima. All the methods described in this book have a strong theoretical foundation, drawing on such diverse mathematical fields as classical optimisation theory, homotopic theory and stochastic approximation theory.
This volume aims to develop the reader's understanding of the theoretical and practical issues involved in the development of efficient MLP training strategies, and to describe and evaluate the performance of a wide range of specific training algorithm. Particular emphasis is given to the development of methods which a strong theoretical foundation, rather than heuristic, "rule of thumb" training strategies. Second-Order Methods for Neural Networks will be of interest to academic researchers and postgraduate students working with neural networks (especially supervised learning with multi-layer perceptrons), industrial researchers and programmers developing neural network software, and professionals using neural networks as optimisation tools.
Inhaltsverzeichnis zu „Second-Order Methods for Neural Networks “
1 Multi-Layer Perceptron Training.- 1.1 Introduction to MLPs.- 1.1.1 The MLP Architecture.- 1.1.2 MLP Training.- 1.2 Error Surfaces and Local Minima.- 1.2.1 Error Surface Fundamentals.- 1.2.2 MLP Error Surfaces.- 1.3 Backpropagation.- 1.3.1 An Introduction to Backpropagation.- 1.3.2 The Bold Driver Method.- 1.3.3 Backpropagation with Momentum.- 1.3.4 On-Line Backpropagation.- 1.3.5 The Delta-Bar-Delta Method.- 2 Classical Optimisation.- 2.1 Introduction to Classical Methods.- 2.1.1 The Linear Model and Steepest Descent.- 2.1.2 The Quadratic Model and Newton's Method.- 2.1.3 Line-Search Methods vs. Model-Trust Region Methods.- 2.2 General Numerical Considerations.- 2.2.1 Finite-Precision Arithmetic and Computational Errors.- 2.2.2 Positive Definiteness and the Model Hessian.- 2.2.3 Scaling and Preconditioning.- 3 Second-Order Optimisation Methods.- 3.1 Line-Search Strategies.- 3.1.1 Line Minimisation Fundamentals.- 3.1.2 Inaccurate Line Searches.- 3.1.3 Backtracking Line Searches.- 3.2 Model-Trust Region Strategies.- 3.2.1 A Simple Model-Trust Region Algorithm.- 3.2.2 Fletcher's Method.- 3.2.3 Modern Model-Trust Region Algorithms.- 3.2.4 Møller's `Scaled' Model-Trust Region Strategy.- 3.3 Multivariate Methods for General Nonlinear Optimisation.- 3.3.1 Finite-Difference Newton's Method.- 3.3.2 Quasi-Newton Methods.- 3.3.3 The 'Memoryless' Quasi-Newton Method.- 3.3.4 Conjugate Gradient Methods.- 3.4 Special Methods for Nonlinear Least Squares.- 3.4.1 The Gauss-Newton Method.- 3.4.2 The Levenberg-Marquardt Method.- 3.5 Comparison of Methods.- 4 Second-Order Training Methods for MLPs.- 4.1 The Calculation of Second Derivatives.- 4.1.1 Exact Evaluation of the Hessian Matrix.- 4.1.2 Exact Evaluation of the Hessian Times a Vector.- 4.1.3 Exact Evaluation of the Jacobian Matrix.- 4.2 Reducing Storage and Computational Costs.- 4.2.1 Diagonal Approximations of the Hessian Matrix.- 4.2.2 Reduced Function and Gradient Evaluations.- 4.3 Second-Order On-Line Training.- 4.3.1 An
... mehr
Introduction to Second-Order On-Line Training Strategies.- 4.3.2 'Noise-Free' On-Line Training Schemes.- 4.4 Conclusion.- 5 An Experimental Comparison of MLP Training Methods.- 5.1 Benchmark Training Tasks.- 5.1.1 N-Parity.- 5.1.2 The sin(x) Problem.- 5.1.3 The sin(x)cos(2x) Problem.- 5.1.4 The tan(x) Problem.- 5.2 Initial Training Conditions.- 5.2.1 MLP Architectures.- 5.2.2 Training Algorithms.- 5.2.3 Termination Conditions.- 5.3 Experimental Results.- 5.3.1 'Global Reliability'.- 5.3.2 Training Speed Metrics.- 5.3.3 Training Speed Results.- 5.3.4 Conclusions.- 6 Global Optimisation.- 6.1 Introduction to Global Methods.- 6.1.1 Stochastic Methods.- 6.1.2 Deterministic Methods.- 6.2 Expanded Range Approximation (ERA).- 6.2.1 An Introduction to ERA.- 6.2.2 The ERA Method in Practice.- 6.3 The TRUST Method.- 6.3.1 The Tunnelling Function.- 6.3.2 The Tunnelling Phase.- 6.3.3 An Evaluation of the TRUST Method.
... weniger
Bibliographische Angaben
- Autor: Adrian J. Shepherd
- 1997, 1997., 145 Seiten, Maße: 23,5 cm, Kartoniert (TB), Englisch
- Verlag: Springer, London
- ISBN-10: 3540761004
- ISBN-13: 9783540761006
Sprache:
Englisch
Kommentar zu "Second-Order Methods for Neural Networks"
0 Gebrauchte Artikel zu „Second-Order Methods for Neural Networks“
| Zustand | Preis | Porto | Zahlung | Verkäufer | Rating |
|---|
Schreiben Sie einen Kommentar zu "Second-Order Methods for Neural Networks".
Kommentar verfassen