Theorem (Least-Squares Polynomial Curve Fitting). Given the    data points  ,  the least squares polynomial of degree  m  of the form  

        

that fits the n data points is obtained by solving the following linear system


          

for the m+1 coefficients .  These equations are referred to as the "normal equations".

Proof  Least Squares Polynomials  Least Squares Polynomials  

    One thing is certain, to find the least squares polynomial the above linear system must be solved. There are various linear system solvers that could be used for this task.  However, since this is such an important computation, most mathematical software programs have a built-in subroutine for this purpose.  In Mathematica it is called the " Fit" procedure.  Fit[data, funs, vars] finds a least­squares fit to a list of data as a linear combination of the functions funs of variables vars.

We will check the "closeness of fit" with the Root Mean Square or RMS measure for the "error in the fit."

Computer Programs  Least Squares Polynomials  Least Squares Polynomials  

Mathematica Subroutine (Least Squares Parabola).

Example 1.  Find the standard "least squares parabola"    for the data points .  
Use the subroutine LSParabola to find the line.  Compare with the line obtained with Mathematica's Fit procedure.
Solution 1.

Example 2.  Find the polynomial curve fit of degree = 2  for the points .
Use Mathematica to find the "Least Square Quadratic", and find the RMS error.
Solution 2.

Example 3.  Find the polynomial curve fit of degree = 3  for the points .
Use Mathematica to find the "Least Square Cubic", and find the RMS error.
Solution 3.

Example 4.  Find the polynomial curve fit of degree = 4  for the points .
Use Mathematica to find the "Least Square Quartic", and find the RMS error.
Solution 4.

Example 5.  Find the polynomial curve fit of degree = 5  for the points .
Use Mathematica to find the "Least Square Quintic", and find the RMS error.
Solution 5.

Example 6.  Why is the RMS error for    essentially zero ?
Solution 6.

Caution for polynomial curve fitting.   

    Something goes radically wrong if the data is radically "NOT polynomial."  This phenomenon is called "polynomial wiggle."  The next example illustrates this concept.  

Example 7.  Find the least squares polynomial fits of degree n = 2, 3, 4, 5 for the points .
Solution 7.

Linear Least Squares

    The linear least-squares problem is stated as follows.  Suppose that data points    and a set of   linearly independent functions   are given.  We want to fine   coefficients    so that the function    given by the linear combination  
    
          

will minimize the sum of the squares of the errors

        .

Theorem (Linear Least Squares).  The solution to the linear least squares problem is found by creating the matrix    whose elements are  

        
        
The coefficients are found by solving the linear system

        

where   and  .

Proof  Least Squares Polynomials  Least Squares Polynomials  

Example 8.  Use the linear least squares method to find the polynomial curve fit of degree = 3  for the points .
Solution 8.

Old Lab Project (Least Squares Polynomials  Least Squares Polynomials).  Internet hyperlinks to an old lab project.  

Research Experience for Undergraduates

Least Squares Polynomials  Least Squares Polynomials  Internet hyperlinks to web sites and a bibliography of articles.  

Download this Mathematica Notebook Least Squares Poynomials

Return to Numerical Methods - Numerical Analysis

(c) John H. Mathews 2004