Introduction to Numerical Analysis for Electrical and Computer Engineers

Christopher?J. Zarowski, PhD,?is an associate professor?in the? Department of Electrical and Computer Engineering at the ?University of Alberta, Canada. His research areas include digital communications applications (wireless, wireline, optical fiber), biomedical applications (e.g., circadian rhythm parameter estimation), structured matrix algebra, wavelet methods, signal detection and parameter estimation, computationally efficient and numerically reliable algorithms, and parallel processing algorithms. He has authored over 100 journal articles and conference papers, and?is a senior member of the IEEE.

"Zarkowski (Univ. of Alberta) offers this book as a general, advanced undergraduate work in numerical analysis, containing all of the usual topics." (CHOICE, October 2004)

• Preface xiii1 Functional Analysis Ideas 11.1 Introduction 11.2 Some Sets 21.3 Some Special Mappings: Metrics, Norms, and Inner Products 41.3.1 Metrics and Metric Spaces 61.3.2 Norms and Normed Spaces 81.3.3 Inner Products and Inner Product Spaces 141.4 The Discrete Fourier Series (DFS) 25Appendix 1.A Complex Arithmetic 28Appendix 1.B Elementary Logic 31References 32Problems 332 Number Representations 382.1 Introduction 382.2 Fixed-Point Representations 382.3 Floating-Point Representations 422.4 Rounding Effects in Dot Product Computation 482.5 Machine Epsilon 53Appendix 2.A Review of Binary Number Codes 54References 59Problems 593 Sequences and Series 633.1 Introduction 633.2 Cauchy Sequences and Complete Spaces 633.3 Pointwise Convergence and Uniform Convergence 703.4 Fourier Series 733.5 Taylor Series 783.6 Asymptotic Series 973.7 More on the Dirichlet Kernel 1033.8 Final Remarks 107Appendix 3.A COordinate Rotation DIgital Computing (CORDIC) 1073.A.1 Introduction 1073.A.2 The Concept of a Discrete Basis 1083.A.3 Rotating Vectors in the Plane 1123.A.4 Computing Arctangents 1143.A.5 Final Remarks 115Appendix 3.B Mathematical Induction 116Appendix 3.C Catastrophic Cancellation 117References 119Problems 1204 Linear Systems of Equations 1274.1 Introduction 1274.2 Least-Squares Approximation and Linear Systems 1274.3 Least-Squares Approximation and Ill-Conditioned Linear Systems 1324.4 Condition Numbers 1354.5 LU Decomposition 1484.6 Least-Squares Problems and QR Decomposition 1614.7 Iterative Methods for Linear Systems 1764.8 Final Remarks 186Appendix 4.A Hilbert Matrix Inverses 186Appendix 4.B SVD and Least Squares 191References 193Problems 1945 Orthogonal Polynomials 2075.1 Introduction 2075.2 General Properties of Orthogonal Polynomials 2075.3 Chebyshev Polynomials 2185.4 Hermite Polynomials 2255.5 Legendre Polynomials 2295.6 An Example of Orthogonal Polynomial Least-Squares Approximation 2355.7 Uniform Approximation 238References 241Problems 2416 Interpolation 2516.1 Introduction 2516.2 Lagrange Interpolation 2526.3 Newton Interpolation 2576.4 Hermite Interpolation 2666.5 Spline Interpolation 269References 284Problems 2857 Nonlinear Systems of Equations 2907.1 Introduction 2907.2 Bisection Method 2927.3 Fixed-Point Method 2967.4 Newton–Raphson Method 3057.4.1 The Method 3057.4.2 Rate of Convergence Analysis 3097.4.3 Breakdown Phenomena 3117.5 Systems of Nonlinear Equations 3127.5.1 Fixed-Point Method 3127.5.2 Newton–Raphson Method 3187.6 Chaotic Phenomena and a Cryptography Application 323References 332Problems 3338 Unconstrained Optimization 3418.1 Introduction 3418.2 Problem Statement and Preliminaries 3418.3 Line Searches 3458.4 Newton’s Method 3538.5 Equality Constraints and Lagrange Multipliers 357Appendix 8.A MATLAB Code for Golden Section Search 362References 364Problems 3649 Numerical Integration and Differentiation 3699.1 Introduction 3699.2 Trapezoidal Rule 3719.3 Simpson’s Rule 3789.4 Gaussian Quadrature 3859.5 Romberg Integration 3939.6 Numerical Differentiation 401References 406Problems 40610 Numerical Solution of Ordinary Differential Equations 41510.1 Introduction 41510.2 First-Order ODEs 42110.3 Systems of First-Order ODEs 44210.4 Multistep Methods for ODEs 45510.4.1 Adams–Bashforth Methods 45910.4.2 Adams–Moulton Methods 46110.4.3 Comments on the Adams Families 46210.5 Variable-Step-Size (Adaptive) Methods for ODEs 46410.6 Stiff Systems 46710.7 Final Remarks 469Appendix 10.A MATLAB Code for Example 10.8 469Appendix 10.B MATLAB Code for Example 10.13 470References 472Problems 47311 Numerical Methods for Eigenproblems 48011.1 Introduction 48011.2 Review of Eigenvalues and Eigenvectors 48011.3 The Matrix Exponential 48811.4 The Power Methods 49811.5 QR Iterations 508References 518Problems 51912 Numerical Solution of Partial Differential Equations 52512.1 Introduction 52512.2 A Brief Overview of Partial Differential Equations 52512.3 Applications of Hyperbolic PDEs 52812.3.1 The Vibrating String 52812.3.2 Plane Electromagnetic Waves 53412.4 The Finite-Difference (FD) Method 54512.5 The Finite-Difference Time-Domain (FDTD) Method 550Appendix 12.A MATLAB Code for Example 12.5 557References 560Problems 56113 An Introduction to MATLAB 56513.1 Introduction 56513.2 Startup 56513.3 Some Basic Operators, Operations, and Functions 56613.4 Working with Polynomials 57113.5 Loops 57213.6 Plotting and M-Files 573References 577Index 579