Numerical Analysis

Lagrange interpolation 3.1.2 Newton's divided differences 3.1.3 How many degree d polynomials pass through n points? 3.1.4 Code for interpolation 3.1.5 Representing functions by approximating polynomials 3.2 Interpolation error 3.2.1 Interpolation error formula 3.2.2 Proof of Newton form and error formula 3.2.3 Runge phenomenon 3.3 Chebyshev interpolation 3.3.1 Chebyshev's Theorem 3.3.2 Chebyshev polynomials 3.3.3 Change of interval 3.4 Cubic splines 3.4.1 Properties of splines 3.4.2 Endpoint conditions 3.5 Bezier curves REALITY CHECK 3: Constructing fonts from Bezier splines 3.6 Software and Further Reading 4. Least Squares 4.1 Least squares and the normal equations 4.1.1 Inconsistent systems of equations 4.1.2 Fitting models to data 4.2 Linear and nonlinear models 4.1.3 Conditioning of least squares 4.2 A survey of models 4.2.1 Periodic data 4.2.2 Data linearization 4.3 QR factorization 4.3.1 Gram-Schmidt orthogonalization and least squares 4.3.2 Modified Gram-Schmidt orthogonalization 4.3.3 Householder reflectors 4.4 Generalized Minimum Residual (GMRES) Method 4.4.1 Krylov methods 4.4.2 Preconditioned GMRES 4.5 Nonlinear least squares 4.5.1 Gauss-Newton method 4.5.2 Models with nonlinear parameters 4.5.3 Levenberg-Marquardt method REALITY CHECK 4: GPS, conditioning and nonlinear least squares 4.6 Software and Further Reading 5. Numerical Differentiation and Integration 5.1 Numerical differentiation 5.1.1 Finite difference formulas 5.1.2 Rounding error 5.1.3 Extrapolation 5.1.4 Symbolic differentiation and integration 5.2 Newton-Cotes formulas for numerical integration 5.2.1 Trapezoid rule 5.2.2 Simpson's Rule 5.2.3 Composite Newton-Cotes Formulas 5.2.4 Open Newton-Cotes methods 5.3 Romberg integration 5.4 Adaptive quadrature 5.5 Gaussian quadrature REALITY CHECK 5: Motion control in computer-aided modelling 5.6 Software and Further Reading 6. Ordinary Differential Equations 6.1 Initial value problems 6.1.1 Euler's method 6.1.2 Existence, uniqueness, and continuity for solutions 6.1.3 First-order linear equations 6.2 Analysis of IVP solvers 6.2.1 Local and global truncation error 6.2.2 The explicit trapezoid method 6.2.3 Taylor methods 6.3 Systems of ordinary differential equations 6.3.1 Higher order equations 6.3.2 Computer simulation: The pendulum 6.3.3 Computer simulation: Orbital mechanics 6.4 Runge-Kutta methods and applications 6.4.1 The Runge-Kutta family 6.4.2 Computer simulation: The Hodgkin-Huxley neuron 6.4.3 Computer simulation: The Lorenz equations REALITY CHECK 6: The Tacoma Narrows Bridge 6.5 Variable step-size methods 6.5.1 Embedded Runge-Kutta pairs 6.5.2 Order 4/5 methods 6.6 Implicit methods and stiff equations 6.7 Multistep methods 6.7.1 Generating multistep methods 6.7.2 Explicit multistep methods 6.7.3 Implicit multistep methods 6.8 Software and Further Reading 7. Boundary Value Problems 7.1 Shooting method 7.1.1 Solutions of boundary value problems 7.1.2 Shooting method implementation REALITY CHECK 7: Buckling of a circular ring 7.2 Finite difference methods 7.2.1 Linear boundary value problems 7.2.2 Nonlinear boundary value problems 7.3 Collocation and the Finite Element Method 7.3.1 Collocation 7.3.2 Finite elements and the Galerkin method 7.4 Software and Further Reading 8. Partial Differential Equations 8.1 Parabolic equations 8.1.1 Forward difference method 8.1.2 Stability analysis of forward difference method 8.1.3 Backward difference method 8.1.4 Crank-Nicolson method 8.2 Hyperbolic equations 8.2.1 The wave equation 8.2.2 The CFL condition 8.3 Elliptic equations 8.3.1 Finite difference method for elliptic equations REALITY CHECK 8: Heat distribution on a cooling fin 8.3.2 Finite element method for elliptic equations 8.4 Nonlinear partial differential equations 8.4.1 Implicit Newton solver 8.4.2 Nonlinear equations in two space dimensions 8.5 Software and Further Reading 9. Random Numbers and Applications 9.1 Random numbers 9.1.1 Pseudo-random numbers 9.1.2 Exponential and normal random numbers 9.2 Monte-Carlo simulation 9.2.1 Power laws for Monte Carlo estimation 9.2.2 Quasi-random numbers 9.3 Discrete and continuous Brownian motion 9.3.1 Random walks 9.3.2 Continuous Brownian motion 9.4 Stochastic differential equations 9.4.1 Adding noise to differential equations 9.4.2 Numerical methods for SDEs REALITY CHECK 9: The Black-Scholes formula 9.5 Software and Further Reading 10. Trigonometric Interpolation and the FFT 10.1 The Fourier Transform 10.1.1 Complex arithmetic 10.1.2 Discrete Fourier Transform 10.1.3 The Fast Fourier Transform 10.2 Trigonometric interpolation 10.2.1 The DFT Interpolation Theorem 10.2.2 Efficient evaluation of trigonometric functions 10.3 The FFT and signal processing 10.3.1 Orthogonality and interpolation 10.3.2 Least squares fitting with trigonometric functions 10.3.3 Sound, noise, and filtering REALITY CHECK 10: The Wiener filter 10.4 Software and Further Reading 11. Compression 11.1 The Discrete Cosine Transform 11.1.1 One-dimensional DCT 11.1.2 The DCT and least squares approximation 11.2 Two-dimensional DCT and image compression 11.2.1 Two-dimensional DCT 11.2.2 Image compression 11.2.3 Quantization 11.3 Huffman coding 11.3.1 Information theory and coding 11.3.2 Huffman coding for the JPEG format 11.4 Modified DCT and audio compression 11.4.1 Modified Discrete Cosine Transform 11.4.2 Bit quantization REALITY CHECK 11: A simple audio codec using the MDCT 11.5 Software and Further Reading 12. Eigenvalues and Singular Values 12.1 Power iteration methods 12.1.1 Power iteration 12.1.2 Convergence of power iteration 12.1.3 Inverse power iteration 12.1.4 Rayleigh quotient iteration 12.2 QR algorithm 12.2.1 Simultaneous iteration 12.2.2 Real Schur form and QR 12.2.3 Upper Hessenberg form REALITY CHECK 12: How search engines rate page quality 12.3 Singular value decomposition 12.3.1 Finding the SVD in general 12.3.2 Special case: symmetric matrices 12.4 Applications of the SVD 12.4.1 Properties of the SVD 12.4.2 Dimension reduction 12.4.3 Compression 12.4.4 Calculating the SVD 12.5 Software and Further Reading 13. Optimization 13.1 Unconstrained optimization without derivatives 13.1.1 Golden section search 13.1.2 Successive parabolic interpolation 13.1.3 Nelder-Mead search 13.2 Unconstrained optimization with derivatives 13.2.1 Newton's method 13.2.2 Steepest descent 13.2.3 Conjugate gradient search 13.2.4 Nonlinear least squares REALITY CHECK 13: Molecular conformation and numerical optimization 13.3 Software and Further Reading APPENDIX Appendix A: Matrix Algebra A.1 Matrix fundamentals A.2 Block multiplication A.3 Eigenvalues and eigenvectors A.4 Symmetric matrices A.5 Vector calculus Appendix B: Introduction to MATLAB B.1 Starting MATLAB B.2 MATLAB graphics B.3 Programming in MATLAB B.4 Flow control B.5 Functions B.6 Matrix operations B.7 Animation and movies References