Theorem (Broyden's Method for 2-dimensional Systems).  To solve the non-linear system  ,  given one initial approximation  ,  and generating a sequence    which converges to the solution  ,  i.e.  .  

Step 0.     Compute the initial Jacobian matrix  

        .  

Use it to compute the first approximation using Newton's method  

        .  

For  .  Suppose that    has been obtained, use the following steps to obtain  .  

Step 1.     Evaluate the function   .

Step 2.  Update the approximate Jacobian using     and   

          

Step 3.  Solve the linear system      for   .  

Step 4.  Compute the next approximation   .  

Proof .

Start Broyden's method with the initial approximation

          

Step 0.     Use Newton's method to construct the next approximation

        ,  

        .  

For  .  Suppose that    has been obtained, use the following steps to obtain  .  

Step 1.     Evaluate the function   .

Step 2.     This is the first Broyden iteration.  It uses the basic property of the Jacobian    to write the equation

        .  
        
But, instead of using the Jacobian matrix  ,  a matrix    is used in the quasi-Newton condition or secant equation  

        .  

    Unfortunately the secant equation does not define    uniquely, it only shows that when a displacement in the direction of the vector    is entered on the left of the equation, it is translated in space into a displacement direction that approximates . We have no other information to explain how displacements in other directions orthogonal to   are effected.  

    To simplify the situation, Broyden adds a requirement for the update of    from    for vectors    in the orthogonal complement.  The requirement is

        

for all vectors    that are orthogonal to   (i.e. all vectors satisfying   ).  This will have the effect of updating    to   only in the direction  .  The resulting formula is  
        
        .   

Where    is the Euclidean norm, and we could have used the calculation  .

Justification

    The least change modification of    to construct   is usually written as follows:

        ,

        ,  
    
        .   

The conclusion of the second step is to compute   

        .

The general step is:  

        

          

          

        

    In order to avoid finding an inverse matrix , one can rewrites the last equation  as follows  

          

Use the substitution    and get

          

Rewrite as a linear system to solve for    

Step 3.  Solve the linear system   

            for    .  

Step 4.  Compute the next iteration using the formula   

        .  

(c) John H. Mathews 2005