When you talk about randomness, stochastics, and noise, everything always seems to be going back to “Brownian motion” and “white noise”. But what exactly are these things?

The problem is that if you ask someone who researches the subject, you most likely get many different definitions at the same time. The reason is because, although there are many equivalent definitions (and terms), every single one only seems to be a small part of the intuition of what this thing actually is. So I am going to state without proof equivalent formulations of $$B(t)$$, which is the same as $B_{t}$$, which is the same as $W_{t}$$…

Einstein’s Physical Definition

Maybe I should address that first, $B$$ or $W$$? The $B$$ honors Robert Brown, a botonist who observed the random motion of pollen grains in the water in 1827. It wasn’t understood how this could happen until 1905 when Albert Einstein explained it using the atomic theory of matter. See, before the rise of the atomic theory, the main theories dealt with either fields and continuums, think of a fluid that if you keep zooming in is still a fluid. Such an object, with its implicit assumption “zoom symmetry” can’t really explain why small items in the fluid act very differently than large items. What Albert Einstein instead describe the fluid as having a whole bunch little things which we now call atoms/molecules. The model is that this pollen particle is bumping against many different water molecules which perturb its motion. To understand the properties this would have, you start big: the random walk. Say you are standing at $x=0$$ at time $t=0$$. You flip a coin every second and choose to step either once to the left or once to the right. This is Brownian motion. Einstein’s idea is that every time the pollen particle hits a water molecule, it takes one step, and then bashes into another. However, since there are so many particles, we need to look at what happens in the limit where we are taking really small random steps every few nanoseconds. This limit of the random walk is Brownian Motion. Since the standard deviation of the random walk at time $t$$ is proportional to $\sqrt{t}$$, Einstein used this in the limit to get an equation for the diffusion (we will address variance vs diffusion later) in terms of how many particles are in the water. This gave an accurate estimate of Avagadro’s number and Einstein his first, and only, Nobel Prize (he never did get one for relativity).

In actuality, Einstein’s Brownian motion wasn’t the limit of a random walk with really small timesteps and really small since mathematically this idea is troublesome (once you go to the limit, your position, the sum of the steps, becomes an integral, and you’re integrating… randomness?) so instead he worked on the level with lots of pollen particles. If you let $\rho(x,t)$$ be the density of pollen particles at $(x,t)$$, then you can start asking how, on average, the big amounts of particles tend to move when you have these collisions. The idea is that if each individual particle tends to move on average an amount $\sqrt{t}$$, then this cloud of particles expands by diffusing at a rate of $D\sim \sqrt{t}$$. Thus Einstein derived the equation

which we now know as the diffusion (or Fokker-Plank) equation with $D$$ being the diffusion constant. So Brownian motion is both the limit of random walks, and the aggregate randomness which is diffusion.

The Mathematicians Strike Back

Mathematicians were intrigued by this and wanted to formalize it in Kolmogorov’s foundations of probability. Along came Norbert Wiener (for whom $W$$ is attributed to). The idea is that if you build a probability space where you are taking random functions such that the following four properties hold:

1. $W_{t}=0$$
2. $W_{t}$$ is continuous
3. $W_{t}$$ has independent increments (how you move from time $t$$ to $s$$ is independent of the motion from time $s$$ to $r$$)
4. $W_{t}-W_{s}\sim \mathcal{N}(0,t-s)$$, that is the increments are Gaussian distributed with mean zero movement and standard deviation of $\sqrt{t-s}$$

Notice that we simply defined it to be something that is continuous and has the properties that we would expect the random walk to have if we made very small movements but kept “the same idea”. To see that this is mathematically what Albert Einstein was talking about, it was shown that if you ask for the probability that $W$$ is a value $x$$ at time $t$$, it indeed is given by the diffusion equation. That is, if you take infinitely many Wiener processes and ask what percentage of them are at a certain point at a certain time, how this changes is governed by diffusion (but now since a mathematician is doing things, you attribute it to Kolmogorov instead of the physicists, so it’s called in these circles the Forward Kolmogorov Equation).

Donsker’s Theorem then showed that, like how the Central Limit makes “everything approximately Gaussian if you wait long enough”, stochastic processes tend to give you this $W_{t}$$ when you go to the small timestep limit. This then brings everything full circle: Donsker’s Theorem can then be used to say that just about any (at least some large class) of randomness in nature involves a whole lot of steps, and so all randomness is Gaussian, and so we get Brownian Motion.

Further Down the Rabbit Hole

Those first four definitions are the main ways of intuiting Brownian motion:

1. It is the limit of random walks as the steps get small
2. It is the process that on average is diffusion
3. It is the process with “stationary and independent Gaussian increments”
4. It’s a form of a Central Limit for discrete stochastic processes

But why stop there? All of these examples show the importance of Brownian motion, and so mathematicians wanted more tools to work with it. One direction is generalizing the kinds of processes: a Levy process is a process which can be described by a probability generator. An easy way to understand it is that Brownian motion is the process whose probability distribution is generated by the diffusion equation, and so we can a whole class of processes whose probability generator is some partial differential equation. Thus Brownian motion is the Levy process whose generator is $-\frac{1}{2}\nabla$$, which is the same as saying its probability distribution changes by the diffusion equation. For these types of properties one can build many tools, and it’s easy to change the definition to any place where you can define calculus (like on manifolds, i.e. what is diffusion on the surface of a sphere?), and so this becomes one of the mathematician’s main definitions.

Another place to look is at asymptotic expansions. We understand the functions $e^{x}$$, $\sin(x)$$, etc via infinite summations: what about $W(t)$$? The Karhunen-Loeve Theorem shows that you can indeed write Brownian motion as an infinite summation. However, the mathematics get difficult because $W(t)$$ is random and almost everywhere non-differentiable, so Taylor’s series does not apply. However, it turns out you can write it as an expansion where the coefficients are random variables (and you guessed it, Gaussian random variables!) multiplied by polynomials known as the Hermite polynomials. Some books such as Liggett’s Continuous Time Markov Processes defines Brownian motion as the function associated to this expansion because this is an easy basis to start calculating properties of Brownian motion. In fact, this definition of Brownian motion is central to numerical stochastics since in the deterministic case most algorithms were derived using expansions (Taylor series, interpolation, etc.), and so this lets one apply similar tools as in the deterministic case.

The last of what I would call “the standard” ways of defining Brownian motion is via stochastic differential equations. We understand the differential equation

as defining $x$$ as a function of time via how it changes on infinitesimal timescales. Thus we take this idea to Brownian motion where we know how it changes on infinitesimal timescales (i.e. like the random walk) and write equations

where $dW_{t}$$ is in some sense “the derivative of Brownian motion”. White noise is mathematically defined as $dW_{t}$$. Brownian motion is thus what happens when you integrate the equation where $f\equiv 0$$ and $g\equiv 1$$. This takes a lot to define mathematically rigorously, but then gives you the tool to expand all differential equation models to stochastic differential equation models by adding the noise term $dW_{t}$$

Going off the rails: “Radically Elementary” Brownian Motion

Why stop there? A small (but growing) contingent of mathematicians define Brownian motion using what’s known as nonstandard analysis. Modern mathematics has abandoned the idea of infinitesimals in favor of the idea of limits, $\epsilon ‘s$$, and $\delta ‘s$$, but Abraham Robinson showed that one could make rigorous the use of infinitesimals. From this field of thought we get Edward Nelson’s Radically Elementary Probability Theory which defines Brownian motion as being the process is the random walk with infinitesimal time steps $dt$$, or by defining white noise as $dW_{t}~N(0,dt)$$ which is a Gaussian with infinitesimal variance. Edward Nelson showed that this is equivalent to the other definitions. Given what we stated before, this is really intuitive. In fact, it gives an easy way to think about simulating stochastic differential equations. Since we know the form for $dW_{t}$$, we can simulate the equations by using small but not infiniteismal timesteps $\Delta t$$ and thus we can write the stochastic differential equation as approximately stepping like:

where $\eta \sim N(0,\Delta t)$$.

Brownian Motion in Biological Processes?

The last thing to address is then, why do we model biological processes with Brownian motion as the noise component? Biological processes are modeled as having randomness that is known as “Poisson Processes”. A Poisson process is a stochastic process which models how many events occur when the underlying process is “memory-less”, that is the process doesn’t remember how long it has been. It’s like how if you flipped a coin and got heads 5 times in a row, your chance of getting tails is still 50%. It doesn’t matter how long you’ve been waiting for a radioactive particle to decay, the probability that it decays within the next 5 minutes is always the same. That is the memory-less property. Then, if you’re looking at many radioactive particles, the number of particles which do decay is a Poisson process. A key fact in this model is that the exponential distribution is the only continuous probability distribution with the memory-less property, and so you can say in some sense that things with the memory-less property at “waiting scale” are Poisson processes at the “counting scale”. So since the probability that any molecule binds to DNA is really only dependent on how long you’re looking at it (and it doesn’t what it did before), then the amount which bind to DNA in a certain amount of time is Poisson distributed.

That’s where Poisson processes come up in biology: you can think of every reaction as being a Poisson process. Obviously, if we have more particles, we would expect the rate at which reactions occur goes up, and so each Poisson process has a rate $\lambda$$ which is proportional to the number of particles and the binding affinity. However, lets say you have an on state and and off state, with a Poisson process of $\lambda$$ to go from on to off and a Poisson process of $\lambda$$ to go from off to on. If you take the amount of particles to infinity (i.e. the rate $\lambda\rightarrow \infty$$), and ask about the probability that a given percentage of the particles are at the on state, this is Brownian motion. Sounds crazy, but when you think about it the process of going from on to off can be thought of as stepping to the right, and going from off to on can be thought of as stepping to the left, and so in the end we are once again asking what happens to a random walk when we are stepping left and right infinitely often. So when you start with a bunch of reaction equations modeled with Poisson processes, the amounts of the reactants is deterministic plus a noise term which is given by Brownian motion. Since Brownian motion is not discrete (i.e. it takes non-integer values), we instead think about this as modeling the concentration of the reactants, but it’s the same idea.

Summary

So what is Brownian motion?

1. It is the limit of random walks as the steps get small
2. It is the process that on average is diffusion
3. It is the process with “stationary and independent Gaussian increments”
4. It’s a form of a Central Limit for discrete stochastic processes
5. It’s the Levy process with the diffusion equation as its generator
6. It’s the function given by a Hermite expansion with standard normal (Gaussian) coefficients
7. It’s the “Central Limit of Stochastic Processes”, i.e. lots of things become Brownian-like
8. It’s the integral of white noise, or the random function whose derivative is white noise
9. It’s a function you get from a random walk with infinitesimal steps
10. It’s the limit of two competing Poisson processes
11. It’s the model for the noise in biological/physical(/financial/climate) processes

That’s not even all of them! That’s why everyone gives you a different definition. I hope this clears up why everyone talks about it differently, uses different terminology, and also different symbols!