Taylor series

Experimental html version of downloadable textbook, see https://theartofhpc.com
\[ % mathjax inclusion. \newcommand\inv{^{-1}}\newcommand\invt{^{-t}} \newcommand\bbP{\mathbb{P}} \newcommand\bbR{\mathbb{R}} \newcommand\defined{ \mathrel{\lower 5pt \hbox{${\equiv\atop\mathrm{\scriptstyle D}}$}}} \newcommand\macro[1]{$\langle$#1$\rangle$} \newcommand\dtdxx{\frac{\alpha\Delta t}{\Delta x^2}} \newcommand\toprule{\hline} \newcommand\midrule{\hline} \newcommand\bottomrule{\hline} \newcommand\nobreak{} \newcommand\multicolumn[3]{#3} \] Back to Table of Contents

16 Taylor series

Taylor series expansion is a powerful mathematical tool. In this course it is used several times in proving properties of numerical methods.

The Taylor expansion theorem, in a sense, asks how well functions can be approximated by polynomials, that is, for a given function $f$, can we find coefficients $c_i$ with $i=1,\ldots,n$ so that

\begin{equation} f(x)\approx c_0+c_1x+c_2x^2+\cdots+c_nx^n. \end{equation}

This question obviously needs to be refined. What do we mean by `approximately equal'? This approximation formula can not hold for all functions $f$ and all $x$: the function $\sin x$ is bounded for all $x$, but any polynomial is unbounded for $x\rightarrow \pm\infty$, so any polynomial approximation to the $\sin x$ function is unbounded. Clearly we can only approximate on an interval.

We will show that a function $f$ with sufficiently many derivatives can be approximated as follows: if the $n$-th derivative $f^{(n)}$ is continuous on an interval $I$, then there are coefficients $c_0,\ldots,c_{n-1}$ such that

\begin{equation} \forall_{x\in I}\colon |f(x)-\sum_{i<n}c_ix^i|\leq cM_n \qquad\hbox{where $M_n=\max_{x\in I}|f^{(n)}(x)|$} \end{equation}

It is easy to get inspiration for what these coefficients should be. Suppose

\begin{equation} f(x) = c_0+c_1x+c_2x^2+\cdots \end{equation}

(where we will not worry about matters of convergence and how long the dots go on) then filling in

\begin{equation} \hbox{$x=0$ gives $c_0=f(0)$.} \end{equation}

Next, taking the first derivative

\begin{equation} f'(x) = c_1+2c_2x+3c_3x^2+\cdots \end{equation}

and filling in

\begin{equation} \hbox{$x=0$ gives $c_1=f'(0)$.} \end{equation}

From the second derivative

\begin{equation} f''(x) = 2c_2+6c_3x+\cdots \end{equation}

so filling in $x=0$ gives

\begin{equation} c_2=f''(0)/2. \end{equation}

Similarly, in the third derivative

\begin{equation} \hbox{filling in $x=0$ gives $c_3=\frac1{3!}f^{(3)}(0)$.} \end{equation}

Now we need to be a bit more precise. Cauchy's form of Taylor's theorem says that

\begin{equation} f(x) = f(a)+\frac1{1!}f'(a)(x-a)+\cdots+\frac1{n!}f^{(n)}(a)(x-a)^n +R_n(x) \end{equation}

where the `rest term' $R_n$ is

\begin{equation} R_n(x) = \frac1{(n+1)!}f^{(n+1)}(\xi)(x-a)^{n+1} \quad\hbox{where $\xi\in(a,x)$ or $\xi\in(x,a)$ depending.} \end{equation}

If $f^{(n+1)}$ is bounded, and $x=a+h$, then the form in which we often use Taylor's theorem is

\begin{equation} f(x) = \sum_{k=0}^n \frac1{k!}f^{(k)}(a)h^k+O(h^{n+1}). \end{equation}

We have now approximated the function $f$ on a certain interval by a polynomial, with an error that decreases geometrically with the inverse of the degree of the polynomial.

For a proof of Taylor's theorem we use integration by parts. First we write

\begin{equation} \int_a^xf'(t)dt=f(x)-f(a) \end{equation}


\begin{equation} f(x) = f(a)+\int_a^xf'(t)dt \end{equation}

Integration by parts then gives

\begin{equation} \begin{array}{r@{=}l} f(x) &f(a)+[xf'(x)-af'(a)]-\int_a^xtf''(t)dt\\ &f(a)+[xf'(x)-xf'(a)+xf'(a)-af'(a)]-\int_a^xtf''(t)dt\\ &f(a)+x\int_a^xf''(t)dt+(x-a)f'(a)-\int_a^xtf''(t)dt\\ &f(a)+(x-a)f'(a)+\int_a^x(x-t)f''(t)dt\\ \end{array} \end{equation}

Another application of integration by parts gives

\begin{equation} f(x)=f(a)+(x-a)f'(a)+\frac12(x-a)^2f''(a) +\frac12 \int_a^x(x-t)^2f'''(t)dt \end{equation}

Inductively, this gives us Taylor's theorem with

\begin{equation} R_{n+1}(x) = \frac1{n!}\int_a^x(x-t)^nf^{(n+1)}(t)dt \end{equation}

By the mean value theorem this is

\begin{equation} \begin{array}{r@{=}l} R_{n+1}(x) &\frac1{(n+1)!}f^{(n+1)}(\xi)\int_a^x(x-t)^nf^{(n+1)}(t)dt\\ &\frac1{(n+1)!}f^{(n+1)}(\xi)(x-a)^{n+1} \end{array} \end{equation}

Back to Table of Contents