m a t h   p a g e: : t h e o r y   &   p r a c t i c e
main menu equations and simultaneous equations with solution
  »  EQUATIONS
  »  SIMULTANEOUS EQUATIONS
  »  SOLUTIONS
  »  Differntial Equations


ALGEBRARIC SIMULTANEOUS EQUATIONS -> Newton's Method for Nonlinear Systems




  • Nonlinear systems of equations

    Newton's Method for Nonlinear Systems

    Solving nonlinear systems iteratively is an active area of research, and there are a number of good sources to consult. Some of the most common methods are:
    1. Steepest Descent
    2. Conjugate Gradient
    3. Broyden and variable metric methods
    4. Fixed Point Methods
    5. Method of Weighted Residuals
    This list is not comprehensive, of course, but perhaps the most common method is ``Newton Raphson's Method'' which we will do next. It is also a good method to understand since many of its pros and cons are the basis for the development of other methods. Let ${\bf f}({\bf x}) \in C^2$ be an $n$-dimensional vector of functions that depend on ${\bf x} \in {\mathbb{R}}^n$. The problem we want to solve is the root-finding problem

    \begin{displaymath} {\bf f}({\bf x}) = {\bf0}. \end{displaymath}

    That is,

    \begin{eqnarray*} &&f_i(x_1,\ldots ,x_n)=0\quad 1\le i\le n\\ &&{\bf x}=(x_1, x_2\ldots, x_n)^T\\ &&{\bf f}=(f_1, f_2,\ldots, f_n)^T. \end{eqnarray*}



    Just as we did in the scalar case, we expand ${\bf f}$ about ${\bf x}$, locally and retain to first order in $\vert\vert{\bf H}\vert\vert$:

    \begin{displaymath} {\bf0}={\bf f}({\bf x}+{\bf H})\approx {\bf f}({\bf x} )+J({\bf x}){\bf H} + \mathcal{O}( \vert\vert {\bf H}\vert\vert^2). \end{displaymath}

    Here, the Jacobian Matrix is

    \begin{eqnarray*} J({\bf x})\equiv {\bf f}'({\bf x})=\left[\begin{array}{cccc} ... ...\cdots & \cdots & \partial f_n/\partial x_n \end{array}\right]. \end{eqnarray*}



    Hence, we can solve formally for the perturbation:

    \begin{displaymath} {\bf H}=-\left[J({\bf x})\right]^{-1}{\bf f} ({\bf x}), \end{displaymath}

    it is now straightforward to see that the situation is not different from the scalar case. We form an iterative problem: iteratively solve for ${\bf x}^{(k+1)}$, for $k=0, 1, 2, \ldots$, starting with some initial guess ${\bf x}^{0}$. This can be done by solving for the update ${\bf H}^k$, using

    -->\begin{displaymath} J({\bf x}^k){\bf H}^k=-{\bf f}({\bf x}^k), \end{displaymath}

    which is a linear-algebraic problem, with an update

    \begin{displaymath} {\bf x}^{(k+1)}={\bf x}^k+{\bf H}^{(k)}. \end{displaymath}

    Convergence is now measured in terms of vector norms and the conditions for success or failure are more complex, but more geometrical (which is neat!). The new issue is: How to solve the linear algebraic problem. We know how to solve these types of problems with direct methods . But as we already know, direct methods are not always the most efficient technique, and in some instances, they are computationally prohibitive. So we study next iterative techniques for the solution of linear systems.

  • Newton Raphson Method (N-R Method)




    • viagra cheap canada