Continuity

The bijection stated above between probability distributions and characteristic functions is sequentially continuous. That is, whenever a sequence of distribution functions F_j(x) converges (weakly) to some distribution F(x), the corresponding sequence of characteristic functions φ_j(t) will also converge, and the limit φ(t) will correspond to the characteristic function of law F. More formally, this is stated as

Lévy’s continuity theorem: A sequence X_j of n-variate random variables converges in distribution to random variable X if and only if the sequence φ_Xj converges pointwise to a function φ which is continuous at the origin. Where φ is the characteristic function of X.^[13]

This theorem can be used to prove the law of large numbers and the central limit theorem.

Inversion formula

There is a one-to-one correspondence between cumulative distribution functions and characteristic functions, so it is possible to find one of these functions if we know the other. The formula in the definition of characteristic function allows us to compute φ when we know the distribution function F (or density f). If, on the other hand, we know the characteristic function φ and want to find the corresponding distribution function, then one of the following inversion theorems can be used.

Theorem. If the characteristic function φ_X of a random variable X is integrable, then F_X is absolutely continuous, and therefore X has a probability density function. In the univariate case (i.e. when X is scalar-valued) the density function is given by

$${\displaystyle f_{X}(x)=F_{X}'(x)={\frac {1}{2\pi }}\int _{\mathbf {R} }e^{-itx}\varphi _{X}(t)\,dt.}$$

In the multivariate case it is

$${\displaystyle f_{X}(x)={\frac {1}{(2\pi )^{n}}}\int _{\mathbf {R} ^{n}}e^{-i(t\cdot x)}\varphi _{X}(t)\lambda (dt)}$$

where ${\textstyle t\cdot x}$ is the dot product.

The density function is the Radon–Nikodym derivative of the distribution μ_X with respect to the Lebesgue measure λ:

$${\displaystyle f_{X}(x)={\frac {d\mu _{X}}{d\lambda }}(x).}$$

Theorem (Lévy).^{[note 1]} If φ_X is characteristic function of distribution function F_X, two points a < b are such that {x | a < x < b} is a continuity set of μ_X (in the univariate case this condition is equivalent to continuity of F_X at points a and b), then

• If X is scalar:
  $${\displaystyle F_{X}(b)-F_{X}(a)={\frac {1}{2\pi }}\lim _{T\to \infty }\int _{-T}^{+T}{\frac {e^{-ita}-e^{-itb}}{it}}\,\varphi _{X}(t)\,dt.}$$

  
  This formula can be re-stated in a form more convenient for numerical computation as^[14]
  $${\displaystyle {\frac {F(x+h)-F(x-h)}{2h}}={\frac {1}{2\pi }}\int _{-\infty }^{\infty }{\frac {\sin ht}{ht}}e^{-itx}\varphi _{X}(t)\,dt.}$$

  
  For a random variable bounded from below one can obtain ${\displaystyle F(b)}$ by taking ${\displaystyle a}$ such that ${\displaystyle F(a)=0.}$ Otherwise, if a random variable is not bounded from below, the limit for ${\displaystyle a\to -\infty }$ gives ${\displaystyle F(b)}$, but is numerically impractical.^[14]
• If X is a vector random variable:
  $${\displaystyle \mu _{X}{\big (}\{a<x<b\}{\big )}={\frac {1}{(2\pi )^{n}}}\lim _{T_{1}\to \infty }\cdots \lim _{T_{n}\to \infty }\int \limits _{-T_{1}\leq t_{1}\leq T_{1}}\cdots \int \limits _{-T_{n}\leq t_{n}\leq T_{n}}\prod _{k=1}^{n}\left({\frac {e^{-it_{k}a_{k}}-e^{-it_{k}b_{k}}}{it_{k}}}\right)\varphi _{X}(t)\lambda (dt_{1}\times \cdots \times dt_{n})}$$

  

Theorem. If a is (possibly) an atom of X (in the univariate case this means a point of discontinuity of F_X ) then

• If X is scalar:
  $${\displaystyle F_{X}(a)-F_{X}(a-0)=\lim _{T\to \infty }{\frac {1}{2T}}\int _{-T}^{+T}e^{-ita}\varphi _{X}(t)\,dt}$$

  
• If X is a vector random variable:^[15]
  $${\displaystyle \mu _{X}(\{a\})=\lim _{T_{1}\to \infty }\cdots \lim _{T_{n}\to \infty }\left(\prod _{k=1}^{n}{\frac {1}{2T_{k}}}\right)\int \limits _{[-T_{1},T_{1}]\times \dots \times [-T_{n},T_{n}]}e^{-i(t\cdot a)}\varphi _{X}(t)\lambda (dt)}$$

  

Theorem (Gil-Pelaez).^[16] For a univariate random variable X, if x is a continuity point of F_X then

${\displaystyle F_{X}(x)={\frac {1}{2}}-{\frac {1}{\pi }}\int _{0}^{\infty }{\frac {\operatorname {Im} [e^{-itx}\varphi _{X}(t)]}{t}}\,dt}$

where the imaginary part of a complex number ${\displaystyle z}$ is given by ${\displaystyle \mathrm {Im} (z)=(z-z^{*})/2i}$.

And its density function is:

${\displaystyle f_{X}(x)={\frac {1}{\pi }}\int _{0}^{\infty }\operatorname {Re} [e^{-itx}\varphi _{X}(t)]\,dt}$

The integral may be not Lebesgue-integrable; for example, when X is the discrete random variable that is always 0, it becomes the Dirichlet integral.

Inversion formulas for multivariate distributions are available.^[14][17]

Criteria for characteristic functions

The set of all characteristic functions is closed under certain operations:

• A convex linear combination ${\textstyle \sum _{n}a_{n}\varphi _{n}(t)}$ (with ${\textstyle a_{n}\geq 0,\ \sum _{n}a_{n}=1}$) of a finite or a countable number of characteristic functions is also a characteristic function.
• The product of a finite number of characteristic functions is also a characteristic function. The same holds for an infinite product provided that it converges to a function continuous at the origin.
• If φ is a characteristic function and α is a real number, then ${\displaystyle {\bar {\varphi }}}$, Re(φ), |φ|², and φ(αt) are also characteristic functions.

It is well known that any non-decreasing càdlàg function F with limits F(−∞) = 0, F(+∞) = 1 corresponds to a cumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given function φ could be the characteristic function of some random variable. The central result here is Bochner’s theorem, although its usefulness is limited because the main condition of the theorem, non-negative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult. Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólya-type.^[18]

Bochner’s theorem. An arbitrary function φ : Rⁿ → C is the characteristic function of some random variable if and only if φ is positive definite, continuous at the origin, and if φ(0) = 1.

Khinchine’s criterion. A complex-valued, absolutely continuous function φ, with φ(0) = 1, is a characteristic function if and only if it admits the representation

${\displaystyle \varphi (t)=\int _{\mathbf {R} }g(t+\theta ){\overline {g(\theta )}}\,d\theta .}$

Mathias’ theorem. A real-valued, even, continuous, absolutely integrable function φ, with φ(0) = 1, is a characteristic function if and only if

${\displaystyle (-1)^{n}\left(\int _{\mathbf {R} }\varphi (pt)e^{-t^{2}/2}H_{2n}(t)\,dt\right)\geq 0}$

for n = 0,1,2,..., and all p > 0. Here H_2n denotes the Hermite polynomial of degree 2n.

Pólya’s theorem can be used to construct an example of two random variables whose characteristic functions coincide over a finite interval but are different elsewhere.

Pólya’s theorem. If ${\displaystyle \varphi }$ is a real-valued, even, continuous function which satisfies the conditions

• ${\displaystyle \varphi (0)=1}$,
• ${\displaystyle \varphi }$ is convex for ${\displaystyle t>0}$,
• ${\displaystyle \varphi (\infty )=0}$,

then φ(t) is the characteristic function of an absolutely continuous distribution symmetric about 0.

Basic manipulations of distributions

Characteristic functions are particularly useful for dealing with linear functions of independent random variables. For example, if X₁, X₂, ..., X_n is a sequence of independent (and not necessarily identically distributed) random variables, and

${\displaystyle S_{n}=\sum _{i=1}^{n}a_{i}X_{i},\,\!}$

where the a_i are constants, then the characteristic function for S_n is given by

${\displaystyle \varphi _{S_{n}}(t)=\varphi _{X_{1}}(a_{1}t)\varphi _{X_{2}}(a_{2}t)\cdots \varphi _{X_{n}}(a_{n}t)\,\!}$

In particular, φ_X+Y(t) = φ_X(t)φ_Y(t). To see this, write out the definition of characteristic function:

${\displaystyle \varphi _{X+Y}(t)=\operatorname {E} \left[e^{it(X+Y)}\right]=\operatorname {E} \left[e^{itX}e^{itY}\right]=\operatorname {E} \left[e^{itX}\right]\operatorname {E} \left[e^{itY}\right]=\varphi _{X}(t)\varphi _{Y}(t)}$

The independence of X and Y is required to establish the equality of the third and fourth expressions.

Another special case of interest for identically distributed random variables is when a_i = 1/n and then S_n is the sample mean. In this case, writing X for the mean,

${\displaystyle \varphi _{\overline {X}}(t)=\varphi _{X}\!\left({\tfrac {t}{n}}\right)^{n}}$

Moments

Characteristic functions can also be used to find moments of a random variable. Provided that the n^th moment exists, the characteristic function can be differentiated n times:

$${\displaystyle \operatorname {E} \left[X^{n}\right]=i^{-n}\left[{\frac {d^{n}}{dt^{n}}}\varphi _{X}(t)\right]_{t=0}=i^{-n}\varphi _{X}^{(n)}(0),\!}$$

This can be formally written using the derivatives of the Dirac delta function:

$${\displaystyle f_{X}(x)=\sum _{n=0}^{\infty }{\frac {(-1)^{n}}{n!}}\delta ^{(n)}(x)\operatorname {E} [X^{n}]}$$

which allows a formal solution to the moment problem.

For example, suppose X has a standard Cauchy distribution. Then φ_X(t) = e^−|t|. This is not differentiable at t = 0, showing that the Cauchy distribution has no expectation. Also, the characteristic function of the sample mean X of n independent observations has characteristic function φ_X(t) = (e^−|t|/n)ⁿ = e^−|t|, using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself.

As a further example, suppose X follows a Gaussian distribution i.e. ${\displaystyle X\sim {\mathcal {N}}(\mu ,\sigma ^{2})}$. Then ${\displaystyle \varphi _{X}(t)=e^{\mu it-{\frac {1}{2}}\sigma ^{2}t^{2}}}$ and

${\displaystyle \operatorname {E} \left[X\right]=i^{-1}\left[{\frac {d}{dt}}\varphi _{X}(t)\right]_{t=0}=i^{-1}\left[(i\mu -\sigma ^{2}t)\varphi _{X}(t)\right]_{t=0}=\mu }$

A similar calculation shows ${\displaystyle \operatorname {E} \left[X^{2}\right]=\mu ^{2}+\sigma ^{2}}$ and is easier to carry out than applying the definition of expectation and using integration by parts to evaluate ${\displaystyle \operatorname {E} \left[X^{2}\right]}$.

The logarithm of a characteristic function is a cumulant generating function, which is useful for finding cumulants; some instead define the cumulant generating function as the logarithm of the moment-generating function, and call the logarithm of the characteristic function the second cumulant generating function.

Data analysis

Characteristic functions can be used as part of procedures for fitting probability distributions to samples of data. Cases where this provides a practicable option compared to other possibilities include fitting the stable distribution since closed form expressions for the density are not available which makes implementation of maximum likelihood estimation difficult. Estimation procedures are available which match the theoretical characteristic function to the empirical characteristic function, calculated from the data. Paulson et al. (1975)^[19] and Heathcote (1977)^[20] provide some theoretical background for such an estimation procedure. In addition, Yu (2004)^[21] describes applications of empirical characteristic functions to fit time series models where likelihood procedures are impractical. Empirical characteristic functions have also been used by Ansari et al. (2020)^[22] and Li et al. (2020)^[23] for training generative adversarial networks.

Example

The gamma distribution with scale parameter θ and a shape parameter k has the characteristic function

${\displaystyle (1-\theta it)^{-k}.}$

Now suppose that we have

${\displaystyle X~\sim \Gamma (k_{1},\theta ){\mbox{ and }}Y\sim \Gamma (k_{2},\theta )}$

with X and Y independent from each other, and we wish to know what the distribution of X + Y is. The characteristic functions are

${\displaystyle \varphi _{X}(t)=(1-\theta it)^{-k_{1}},\,\qquad \varphi _{Y}(t)=(1-\theta it)^{-k_{2}}}$

which by independence and the basic properties of characteristic function leads to

${\displaystyle \varphi _{X+Y}(t)=\varphi _{X}(t)\varphi _{Y}(t)=(1-\theta it)^{-k_{1}}(1-\theta it)^{-k_{2}}=\left(1-\theta it\right)^{-(k_{1}+k_{2})}.}$

This is the characteristic function of the gamma distribution scale parameter θ and shape parameter k₁ + k₂, and we therefore conclude

${\displaystyle X+Y\sim \Gamma (k_{1}+k_{2},\theta )}$

The result can be expanded to n independent gamma distributed random variables with the same scale parameter and we get

${\displaystyle \forall i\in \{1,\ldots ,n\}:X_{i}\sim \Gamma (k_{i},\theta )\qquad \Rightarrow \qquad \sum _{i=1}^{n}X_{i}\sim \Gamma \left(\sum _{i=1}^{n}k_{i},\theta \right).}$

Citations

Sources