In physics, statistics, econometrics and signal processing, a stochastic process is said to be in an ergodic regime if an observable's ensemble average equals the time average.^[1] In this regime, any collection of random samples from a process must represent the average statistical properties of the entire regime. Conversely, a process that is not in ergodic regime is said to be in non-ergodic regime.^[2]

Specific definitions

One can discuss the ergodicity of various statistics of a stochastic process. For example, a wide-sense stationary process ${\displaystyle X(t)}$ has constant mean

${\displaystyle \mu _{X}=E[X(t)],}$

and autocovariance

${\displaystyle r_{X}(\tau )=E[(X(t)-\mu _{X})(X(t+\tau )-\mu _{X})],}$

that depends only on the lag ${\displaystyle \tau }$ and not on time ${\displaystyle t}$. The properties ${\displaystyle \mu _{X}}$ and ${\displaystyle r_{X}(\tau )}$ are ensemble averages (calculated over all possible sample functions ${\displaystyle X}$), not time averages.

The process ${\displaystyle X(t)}$ is said to be mean-ergodic^[3] or mean-square ergodic in the first moment^[4] if the time average estimate

${\displaystyle {\hat {\mu }}_{X}={\frac {1}{T}}\int _{0}^{T}X(t)\,dt}$

converges in squared mean to the ensemble average ${\displaystyle \mu _{X}}$ as ${\displaystyle T\rightarrow \infty }$.

Likewise, the process is said to be autocovariance-ergodic or d moment^[4] if the time average estimate

${\displaystyle {\hat {r}}_{X}(\tau )={\frac {1}{T}}\int _{0}^{T}[X(t+\tau )-\mu _{X}][X(t)-\mu _{X}]\,dt}$

converges in squared mean to the ensemble average ${\displaystyle r_{X}(\tau )}$, as ${\displaystyle T\rightarrow \infty }$. A process which is ergodic in the mean and autocovariance is sometimes called ergodic in the wide sense.

Discrete-time random processes

The notion of ergodicity also applies to discrete-time random processes ${\displaystyle X[n]}$ for integer ${\displaystyle n}$.

A discrete-time random process ${\displaystyle X[n]}$ is ergodic in mean if

${\displaystyle {\hat {\mu }}_{X}={\frac {1}{N}}\sum _{n=1}^{N}X[n]}$

converges in squared mean to the ensemble average ${\displaystyle E[X]}$, as ${\displaystyle N\rightarrow \infty }$.

Examples

Ergodicity means the ensemble average equals the time average. Following are examples to illustrate this principle.

Examples of non-ergodic random processes

• An unbiased random walk is non-ergodic. Its expectation value is zero at all times, whereas its time average is a random variable with divergent variance.

• Suppose that we have two coins: one coin is fair and the other has two heads. We choose (at random) one of the coins first, and then perform a sequence of independent tosses of our selected coin. Let X[n] denote the outcome of the nth toss, with 1 for heads and 0 for tails. Then the ensemble average is 1⁄2  (1⁄2 +  1) = 3⁄4; yet the long-term average is 1⁄2 for the fair coin and 1 for the two-headed coin. So the long term time-average is either 1/2 or 1. Hence, this random process is not ergodic in mean.

See also

• Ergodic hypothesis
• Ergodicity
• Ergodic theory, a branch of mathematics concerned with a more general formulation of ergodicity
• Loschmidt's paradox
• Poincaré recurrence theorem

Notes

References

• Porat, B. (1994). Digital Processing of Random Signals: Theory & Methods. Prentice Hall. p. 14. ISBN 0-13-063751-3.
• Papoulis, Athanasios (1991). Probability, random variables, and stochastic processes. New York: McGraw-Hill. pp. 427–442. ISBN 0-07-048477-5.