Alexander Fufaev
My name is Alexander FufaeV and here I write about:

Fourier Series: Learn How to Approximate a Function

Table of contents
  1. Why Fourier series?
  2. The concept of Fourier series Here you will learn what the Fourier series of (periodic) functions looks like and how it is obtained.
  3. Fourier coefficients Here you will learn how Fourier coefficients are calculated and how its formula is obtained.
  4. Fourier basis Here we look at complex exponentials as a typical basis for Fourier expansion.
  5. Example: Fourier series for the sawtooth function Here we calculate the Fourier coefficients for a concrete function and build its Fourier series.

Why Fourier series?

You surely know the Taylor expansion, with which we can approximate a function \(f(x)\) at a point \(x = x_0\) by a simpler Taylor series. The more terms we take in the Taylor series, the better the approximation \( f_{\text{taylor}} \) becomes in the neighborhood of the chosen point \(x_0\). Taylor approximation and Fourier approximation of a function

As you can see in Illustration 1, the Taylor series represented by \( f_{\text{taylor}} \) is a good approximation of the function \(f\) in the immediate neighborhood of \(x_0\). However, if we move further away from the point, we see that the Taylor series is not a good approximation there. Taylor expansion is thus a method with which we can approximate a function only locally.

But if we care about approximating a function on an entire interval, then we need a Fourier series of the function. As we will see, the Fourier series is a linear combination of simple periodic basis functions such as cosine and sine or complex exponentials, which in sum can approximate the function \(f\) within an interval. In the following we assume an interval of length \(L\).

The concept of Fourier series

We can represent a vector \( \boldsymbol{v} \) living in a \(n\)-dimensional vector space as a linear combination of basis vectors \( \boldsymbol{e}_k \) spanning the vector space:

You should know this from linear algebra! By the way, the symbol \(\boxed{+}\) is used on this website as a modern, much better alternative for the sum symbol \(\Sigma\).

Using a basis \({ \boldsymbol{e}_k }\) we can represent every possible vector \( \boldsymbol{v} \) in this vector space. Here \(v_k\) are the components of the vector in the chosen basis. The components are not unique. By choosing another basis the vector has other components \(v_k\).

We can also apply this concept of linear combination to infinite-dimensional vectors. A function \(f(x)\) can be interpreted as an infinite-dimensional vector, which we can represent as a linear combination. The components \(v_k\) of a finite vector become Fourier coefficients \(\widetilde{f}_k\) if we represent a function rather than a finite vector as a linear combination:

If we represent a function \(f\) as a linear combination as Fourier series of \(f\).

Given a linear combination for a function, the basis vectors \(\boldsymbol{e}_k\) are called basis functions. In optics, the basis functions are also often called Fourier modes.

We think of the function values \(f(x_0)\), \(f(x_1)\), \(f(x_2)\) and so on, up to \(f(x_0 + L)\) as components of \(f\). Here we cheated a bit because the argument \(x\) is a real number and there are theoretically infinitely many values between, for example, \(x_0\) and \(x_1\). But this way you can at least imagine a function as a vector with infinitely many components:

Fourier coefficients

You can determine the Fourier coefficients in the same way as you determine the vector components in linear algebra. How does that work again in linear algebra? To get the \(k\)th component of a finite-dimensional vector \(\boldsymbol{v}\), we need to form the scalar product between the \(k\)th basis vector \(\boldsymbol{e}_k\) and the vector \(\boldsymbol{v}\):

We have written the scalar product a little more compactly in the last step with a sum sign. Here we sum over the index \(j\).

If we do not work with finite-dimensional vectors but with functions, then we have to form the scalar product between the \(k\)-th basis function and the function \(f\) to get the \(k\)-th Fourier coefficient of \(f\):

To indicate that we are working here with possibly an infinite dimensional vector space, we call the operation 4 not scalar product but inner product and write it in Bra-Ket notation (as physicists).

Let us first write out the inner product analogously to the finite-dimensional case 4. With the only difference that we interpret the components 3 of the function as function values: \(f(x_0)\), \(f(x_1)\), \(f(x_2)\) and so on. We also consider the components of the basis functions to be function values:

Here we sum up to the point \(x=x_0+L\) because, as said before, with a Fourier series we can only work with functions in a chosen interval. But don't take this summation too serous, because for now we just want to motivate the formula for the inner product for functions.

We can compactly write down the summation over the function argument with a sum sign:

And here we have a problem: how should the summation in 7 work for a function at all? The summation index \(x\) is a continuous variable in the case of functions! That means: Even between the points \(x_0\) and \(x_1\) there are theoretically infinitely many other function values, which we have simply omitted here. We can easily solve the problem. Because we are dealing with a continuous summation here, we simply have to replace the sum sign with an integral:

We also need to complex-conjugate the first argument (here: basis function) in the inner product in the intergral. This is necessary if we also allow complex-valued functions \(f: \mathbb{R} \rightarrow \mathbb{C} \). If we did not do this, the integral would not be an inner product in the case of complex-valued functions because it would not satisfy the mathematical properties of an inner product.

So, hopefully now you understand how to determine the Fourier coefficients and how the formula 8 is obtained in the first place. You simply have to form the inner product of the function \(f\) with the basis functions. That is: You have to calculate the integral 8.

Fourier basis

Let's move on to the basis functions. What basis functions can we use for the Fourier series 2? All those that satisfy the properties of a basis! For us to call a set of vectors, or as in our case, a set of functions a basis, they must satisfy two conditions:

  1. If we take two basis functions, then they must be orthonormal to each other, that is, orthogonal and normalized.
  2. The set of basis functions must be complete. In other words, they must span the space in which the functions \(f\) live. Only then we are able to represent each function \(f\) as a linear combination of these basis functions.

A typical basis used in physics is a set of complex exponential functions:

The factor \(\frac{1}{\sqrt{L}}\) ensures that the basis function is normalized.

For each different wavenumber \(k\) you get a different basis vector. Maybe you saw a different basis for the Fourier series, like cosine and sine. As I said we are free to choose a basis. Here we choose complex exponential functions as a basis, because they can be written compactly, especially for explaining the Fourier series.

The Fourier series 2 in this exponential basis would then look like this:

Property #1: Fourier basis is orthonormal

Let's look at the first property of basis functions: Orthonormality. We can check orthonormality by taking two different basis functions, \(\boldsymbol{e}_k\) and \(\boldsymbol{e}_{k'}\), and forming the inner product, as in 8.

  • For the two functions to be orthonormal, their inner product must yield \(\langle \boldsymbol{e}_k | \boldsymbol{e}_{k'} \rangle = 1\) if \(k = k'\) (that is, if we take the inner product of a function with itself).

  • And the inner product \(\langle \boldsymbol{e}_k | \boldsymbol{e}_{k'} \rangle = 0\) must be zero if \(k\neq k'\) is (that is, if we do NOT take the inner product of a function with itself).

We can combine the two cases into one equation if we use a Kronecker delta:

We can easily show that the exponential functions 9 are orthonormal:

Here we combined the two normalization factors to \(\frac{1}{L}\) and complex-conjugated the first exponential function (hence the minus sign in the exponent).

The two exponential functions can be combined:

Let us first consider the case \(k = k'\). Then the exponential function is equal to 1 and inserting the integration limits gives:

Here you also see why the factor \( \frac{1}{\sqrt{L}} \) is necessary for the basis functions: So that we get a 1 for the inner product 15, as it should be for orthonormal vectors or functions.

For the case \(k \neq k'\), that is, for two different basis functions, the integral 14 must yield zero. Integrating the exponential function returns the exponential function, and a factor in front of it:

In order for the two exponential functions in the parenthesis to be equal, we must assume periodic boundary conditions. That means the exponential function at the point \(x_0\) must be equal to the exponential function at the point \(x_0 + L\). Under this condition, the two terms in the parenthesis cancel out and the integral is zero.

As you can see, the chosen basis functions 9, together with periodic boundary conditions are orthonormal.

Discrete \(k\) values with periodic boundary conditions

For the exponential functions to be orthonormal, we have assumed that they must be \(L\)-periodic. That is, \( \text{e}^{\mathrm{i}\,x_0 \, (k'-k)} \) is equal to the exponential function \( \text{e}^{\mathrm{i}\,(x_0+L) \, (k'-k)} \), because the second exponential function is only shifted by the period length \(L\). From the periodicity \( \text{e}^{\mathrm{i}\,k\, L} \stackrel{!}{=} 1 \) we can find a condition for the values of \(k\):

Here \(n = ... -2, -1, 0, 1, 2... \) is an integer.

Property #2: Fourier basis is complete

The second property that the set of functions must satisfy to be a basis is the completeness relation. With this relation we ensure that we can represent any function \(f(x)\), using the chosen basis \( { e_k(x) } \).

Insert the formula for the Fourier coefficients 8 into the Fourier series 2:

The sum over the two basis functions together with \(f(x')\) in the integral, picks the value of the function \(f(x)\). This behavior is exhibited by the delta function \(\delta(x-x')\).

Example: Fourier series for the sawtooth function

Now you should have a solid, intuitive understanding of a Fourier series and how to theoretically calculate it for a function. Let's do a concrete example of how we can specify a Fourier series for a function. Schauen wir uns eine Sägezahnfunktion zwischen \(x=0\) und \(x=1\) an. Sie ist folgendermaßen definiert: $$ f(x) ~=~ \begin{cases} -x, &\mbox{} \left(0,~ 0.5\right) \\ 1-x, &\mbox{} \left(0.5,~ 1\right) \end{cases} $$

The total interval length is \(L=1\). Thus the normalization factor for the exponential basis functions is also 1: $$e_k(x) ~=~ \text{e}^{\mathrm{i}\, k\, x}$$

When determining Fourier series, we must always do two things:

  1. Choose a basis (we already did) and insert it into the Fourier series 2: $$f(x) ~=~ \underset{k}{\boxed{+}} ~ \widetilde{f}_k \, \text{e}^{\mathrm{i}\, k\, x}$$
  2. Calculate the Fourier coefficients \(\widetilde{f}_k\) with 8 and insert them into the Fourier series 22.

You determine the \(k\)th Fourier coefficient using the inner product between the \(k\)th basis function and the sawtooth function, as shown in Eq. 8: $$\begin{align}\widetilde{f}_k ~&=~ \langle \boldsymbol{e}_k | f \rangle \\\\ ~&=~ \int_{0}^{1} \text{d}x \, \text{e}^{-\mathrm{i}\, k\, x} \, f(x)\end{align}$$

Since the sawtooth function is defined piecewise, we split the integral: $$\widetilde{f}_k ~=~ \int_{0}^{1/2} \text{d}x \, \text{e}^{-\mathrm{i}\, k\, x} \, (-x) ~+~ \int_{1/2}^{1} \text{d}x \, \text{e}^{-\mathrm{i}\, k\, x} \, (1-x)$$

With integration by parts or with an integration calculator you can determine the two integrals. As a result you get: $$\widetilde{f}_k ~=~ \frac{1}{ \mathrm{i}\,k } \, \text{e}^{-\mathrm{i}\,k/2}$$

Since we have not inserted a concrete value for \(k\), we have determined ALL Fourier coefficients with it. For a different \(k\) value we get a different Fourier coefficient. Let's insert the Fourier coefficients into the Fourier series and combine the two exponential functions: \begin{align}f(x) ~&=~ \underset{k}{\boxed{+}} ~ \frac{1}{ \mathrm{i}\,k } \, \text{e}^{-\mathrm{i}\,k/2} \, \text{e}^{\mathrm{i}\, k\, x} \\\\ ~&=~ \underset{k}{\boxed{+}} ~ \frac{1}{ \mathrm{i}\,k } \, \text{e}^{\mathrm{i}\,k (x-1/2)} \\\\ ~&=~ \underset{n}{\boxed{+}} ~ \frac{1}{ \mathrm{i}\,2\pi\,m } \, \text{e}^{\mathrm{i}\,2\pi\,m (x-1/2)}\end{align}

In the last step we used \( k = \frac{2\pi}{L} \, m \), with \( L = 1 \). Note that we sum over negative and positive \( m \) here! Sawtooth function is approximated by Fourier series (two different approximations). With this Fourier series for the sawtooth function, we basically gained two things.

  1. We can sum the series only up to a certain maximum value \( m = m_{\text{max}} \) and thus obtain an arbitrarily good approximation for the sawtooth function.
  2. Since we have determined the Fourier coefficients, we know which \(m\) values are contained in the sawtooth function (\(m=0\) is not contained, for example). So we know which building blocks (basis functions) the sawtooth function is composed of. This "breaking down of the function into individual components" is called Fourier analysis.

In the next lessons, we'll look at how we can use Fourier series to make Fourier analysis of complicated functions. Another thing we will learn is how we can use a so-called Fourier transform to approximate a function \(f\) not only in an interval but in the whole space (\(L \rightarrow \infty \)), globally.