Background

    We have seen how Newton polynomials and Lagrange polynomials are used to approximate on an interval .  The constructions were based on a discrete set of interpolation points in the interval.  We will now consider "least squares" approximations where the polynomial is "close" to the function throughout the interval .  Our final construction will use Legendre polynomials that were first studied by the French mathematician Adrien-Marie Legendre (1752-1833).
    
    Given a set of interpolation points the Newton polynomial and Lagrange polynomial are algebraically equivalent, and they are equivalent to the polynomial constructed with Mathematica's built in subroutine "InterpolatingPolynomial."  The subroutine "Fit" can be used to construct the discrete least squares fit polynomial.

Definition (Discrete Least Squares Approximation)  Given a function   on  and equally spaced nodes   and interpolation points .  The degree polynomial    is the discrete least squares interpolation fit provided that the coefficients    of    minimize the sum

            

Theorem (Discrete Least Squares Approximation)  The polynomial    satisfies the equations  
    
                for   .  

These equations can be simplified to obtain the normal equations for finding the coefficients   

                 for   .   

Remark. This is the degenerate case of a least squares fit (i.e. if there were   data points we would have used    instead of  ).
Information on polynomial curve fitting can be found in the module Least Squares Polynomials.

    The following example shows that if n+1 points are used to find the discrete least squares approximation polynomial of degree n , then it is the same as the Newton (and Lagrange) interpolation polynomial that passes through the n+1 points.  

Example 1.  Compare the "discrete interpolation polynomial" and "discrete least squares approximation" using equally spaced nodes on the interval  .  
1 (a).  Use the function  .  
1 (b).  Use the function  .  
Solution 1 (a).
Solution 1 (b).

Example 2.  Compare the "discrete interpolation polynomial" and "discrete least squares approximation" using equally spaced nodes.  
2 (a).  Use the function    on the interval  .  
2 (b).  Use the function    on the interval  .  
Solution 2 (a).
Solution 2 (b).

Continuous Least Squares Approximation

    Another method for approximating    on an interval    is to find a polynomial   with a small average error over the entire interval.  This can be accomplished by integrating the square of the difference    over  .  The following derivation is done on an arbitrary interval  , but we will soon see that it is advantageous to use the interval  .  

Definition (ContinuousLeast Squares Approximation)  Given a function   on  .  The nth degree polynomial    is the continuous least squares fit for provided that the coefficients minimize the integral  

            .

Theorem (Continuous Least Squares Approximation)  The polynomial    satisfies the equations  
    
                for   .  

These equations can be simplified to obtain the normal equations for finding the coefficients   

                 for   .  

Proof  Legendre Polynomials  

Computer Programs  Legendre Polynomials  

Example 3.  Compare the "discrete least squares approximation" and "continuous least squares approximation," on the interval  .  
3 (a).  Use the function  .  
3 (b).  Use the function  .  
Solution 3 (a).
Solution 3 (b).

Example 4.  Compare the "discrete least squares approximation" and "continuous least squares approximation."  
4 (a).  Use the function  ,  on the interval  .  
4 (b).  Use the function  ,  on the interval  .  
Solution 4 (a).
Solution 4 (b).

Orthogonal Polynomials

    To start we need some background regarding an the inner product.  

Definition (Inner Product).  Consider the vector space of functions whose domain is the interval .  We define the inner product of two functions as follows  

        .  

Mathematica Function (Inner Product). To compute the inner product of two real functions over .  

Remark.  The inner product is a continuous analog to the ordinary dot product that is studied in linear algebra.  If the integral is zero then are said to be orthogonal to each other on .  All the functions we use are assumed to be square-integrable, i. e. .  

Example 5 (a).  Find the inner product of     and    over .  
5 (b).  Find the inner product of     and    over .   
5 (c).  Find the inner product of     and    over .   
Solution 5 (a).
Solution 5 (b).
Solution 5 (c).

Example 6.  Show that the Legendre polynomials  ,  ,  ,  ,    and    are orthogonal on .  
Solution 6.

Basis Functions

    A basis for a vector space V of functions is a set of linear independent functions   which has the property that any can be written uniquely as a linear combination  
    
        .  

Fact.  The set    is a basis for the set of all polynomials and power series.

Definition (Orthogonal Basis)  The set     is said to be an orthogonal basis on provided that  

           when  ,
    and  
          when  .

In the special case when   for we say that is an orthonormal basis.    

Theorem (Gram-Schmidt Orthogonalization)  Given    we can construct a set of orthogonal polynomials    over the interval    as follows:

        Use the inner product  ,  and define  

                

Remark.  A set of orthonormal polynomials over the interval    is  .   

Remark.  When these polynomials are constructed over the interval and normalized so that they are called the Legendre polynomials, and form a basis for the set of polynomials and power series over the interval .  

Corollary 1.  The set of orthogonal polynomials is a basis for the set V of all polynomials and power series over the interval .  

Corollary 2.  The set of Legendre polynomials is a basis for the set V of all polynomials and power series over the interval .  

Proof  Legendre Polynomials  

Example 7.  Start with    over the interval .  Use the Gram-Schmidt orthogonalization to construct a few of the orthogonal polynomials   over the interval .  Construct the corresponding Legendre polynomials  .
Solution 7.

An Alternate Recursive Formula

    Another way to recursively define the Legendre polynomials is
    
        

Exploration

Efficient Computations

    We now present the efficient way to compute the continuous least squares approximation.  It has an additional feature that each successive term increases the degree of approximation.  Hence, an increasing sequence of of approximations can obtained recursively:  
    
          
        
Theorem (Legendre Series Approximation)  The Legendre series approximation of order for a function over    is given by
    
        

where    is the Legendre polynomial and

        

Proof  Legendre Polynomials  

Computer Programs  Legendre Polynomials  

Example 8.  Compare the "discrete interpolation polynomial" and "Legendre series approximation," on the interval  .  
8 (a).  Use the function  .   
8 (b).  Use the function  .  
Solution 8 (a).
Solution 8 (b).

The Shifted Legendre Polynomials

    The "shifted Legendre polynomials
    
          are orthogonal on ,  
    
    where are the Legendre polynomials on .   

Exploration.

Theorem (Shifted Legendre Series Interpolation)  The shifted Legendre series approximation of order for a function over    is given by
    
        

where    is the shifted Legendre polynomial and

        

Proof  Legendre Polynomials  

Computer Programs  Legendre Polynomials  

Example 9.  Compare the "discrete interpolation polynomial" and "shifted Legendre series approximation"
9 (a).  Use the function  ,  on the interval  .  
9 (b).  Use the function  ,  on the interval  .  
Solution 9 (a).
Solution 9 (b).

Research Experience for Undergraduates

Legendre Polynomials  Internet hyperlinks to web sites and a bibliography of articles.  

Download this Mathematica Notebook Legendre Polynomials

(c) John H. Mathews 2005