In signal processing, time–frequency analysis comprises those techniques that study a signal in both the time and frequency domains simultaneously, using various time–frequency representations. Rather than viewing a 1-dimensional signal (a function, real or complex-valued, whose domain is the real line) and some transform (another function whose domain is the real line, obtained from the original via some transform), time–frequency analysis studies a two-dimensional signal – a function whose domain is the two-dimensional real plane, obtained from the signal via a time–frequency transform.^[1][2]

The mathematical motivation for this study is that functions and their transform representation are tightly connected, and they can be understood better by studying them jointly, as a two-dimensional object, rather than separately. A simple example is that the 4-fold periodicity of the Fourier transform – and the fact that two-fold Fourier transform reverses direction – can be interpreted by considering the Fourier transform as a 90° rotation in the associated time–frequency plane: 4 such rotations yield the identity, and 2 such rotations simply reverse direction (reflection through the origin).

The practical motivation for time–frequency analysis is that classical Fourier analysis assumes that signals are infinite in time or periodic, while many signals in practice are of short duration, and change substantially over their duration. For example, traditional musical instruments do not produce infinite duration sinusoids, but instead begin with an attack, then gradually decay. This is poorly represented by traditional methods, which motivates time–frequency analysis.

One of the most basic forms of time–frequency analysis is the short-time Fourier transform (STFT), but more sophisticated techniques have been developed, notably wavelets and least-squares spectral analysis methods for unevenly spaced data.

Motivation

In signal processing, time–frequency analysis^[3] is a body of techniques and methods used for characterizing and manipulating signals whose statistics vary in time, such as transient signals.

It is a generalization and refinement of Fourier analysis, for the case when the signal frequency characteristics are varying with time. Since many signals of interest – such as speech, music, images, and medical signals – have changing frequency characteristics, time–frequency analysis has broad scope of applications.

Whereas the technique of the Fourier transform can be extended to obtain the frequency spectrum of any slowly growing locally integrable signal, this approach requires a complete description of the signal's behavior over all time. Indeed, one can think of points in the (spectral) frequency domain as smearing together information from across the entire time domain. While mathematically elegant, such a technique is not appropriate for analyzing a signal with indeterminate future behavior. For instance, one must presuppose some degree of indeterminate future behavior in any telecommunications systems to achieve non-zero entropy (if one already knows what the other person will say one cannot learn anything).

To harness the power of a frequency representation without the need of a complete characterization in the time domain, one first obtains a time–frequency distribution of the signal, which represents the signal in both the time and frequency domains simultaneously. In such a representation the frequency domain will only reflect the behavior of a temporally localized version of the signal. This enables one to talk sensibly about signals whose component frequencies vary in time.

For instance rather than using tempered distributions to globally transform the following function into the frequency domain one could instead use these methods to describe it as a signal with a time varying frequency.

${\displaystyle x(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(3\pi t);&10\leq t<20\\\cos(2\pi t);&t>20\end{cases}}}$

Once such a representation has been generated other techniques in time–frequency analysis may then be applied to the signal in order to extract information from the signal, to separate the signal from noise or interfering signals, etc.

Time–frequency distribution functions

Ideal TF distribution function

A time–frequency distribution function ideally has the following properties:

1. High resolution in both time and frequency, to make it easier to be analyzed and interpreted.
2. No cross-term to avoid confusing real components from artifacts or noise.
3. A list of desirable mathematical properties to ensure such methods benefit real-life application.
4. Lower computational complexity to ensure the time needed to represent and process a signal on a time–frequency plane allows real-time implementations.

Below is a brief comparison of some selected time–frequency distribution functions.^[4]

	Clarity	Cross-term	Good mathematical properties	Computational complexity
Gabor transform	Worst	No	Worst	Low
Wigner distribution function	Best	Yes	Best	High
Gabor–Wigner distribution function	Good	Almost eliminated	Good	High
Cone-shape distribution function	Good	No (eliminated, in time)	Good	Medium (if recursively defined)

To analyze the signals well, choosing an appropriate time–frequency distribution function is important. Which time–frequency distribution function should be used depends on the application being considered, as shown by reviewing a list of applications.^[5] The high clarity of the Wigner distribution function (WDF) obtained for some signals is due to the auto-correlation function inherent in its formulation; however, the latter also causes the cross-term problem. Therefore, if we want to analyze a single-term signal, using the WDF may be the best approach; if the signal is composed of multiple components, some other methods like the Gabor transform, Gabor-Wigner distribution or Modified B-Distribution functions may be better choices.

As an illustration, magnitudes from non-localized Fourier analysis cannot distinguish the signals:

${\displaystyle x_{1}(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(3\pi t);&10\leq t<20\\\cos(2\pi t);&t>20\end{cases}}}$

${\displaystyle x_{2}(t)={\begin{cases}\cos(\pi t);&t<10\\\cos(2\pi t);&10\leq t<20\\\cos(3\pi t);&t>20\end{cases}}}$

But time–frequency analysis can.

TF analysis and Random Process^[6]

For a random process x(t), we cannot find the explicit value of x(t).

The value of x(t) is expressed as a probability function.

• Auto-covariance function ${\displaystyle R_{x}(t,\tau )}$

${\displaystyle R_{x}(t,\tau )=E[x(t+\tau /2)x^{*}(t-\tau /2)]}$ In usual, we suppose that ${\displaystyle E[x(t)]=0}$ for any t,

${\displaystyle E[x(t+\tau /2)x^{*}(t-\tau /2)]}$ ${\displaystyle =\iint x(t+\tau /2,\xi _{1})x^{*}(t-\tau /2,\xi _{2})P(\xi _{1},\xi _{2})d\xi _{1}d\xi _{2}}$(alternative definition of the auto-covariance function) ${\displaystyle {\overset {\land }{R_{x}}}(t,\tau )=E[x(t)x(t+\tau )]}$

• Power spectral density (PSD) ${\displaystyle S_{x}(t,f)}$

${\displaystyle S_{x}(t,f)=\int _{-\infty }^{\infty }R_{x}(t,\tau )e^{-j2\pi f\tau }d\tau }$

• Relation between the WDF (Wigner Distribution Function) and the random process

${\displaystyle E[W_{x}(t,f)]=\int _{-\infty }^{\infty }E[x(t+\tau /2)x^{*}(t-\tau /2)]\cdot e^{-j2\pi f\tau }\cdot d\tau }$ ${\displaystyle =\int _{-\infty }^{\infty }R_{x}(t,\tau )\cdot e^{-j2\pi f\tau }\cdot d\tau }$${\displaystyle =S_{x}(t,f)}$

• Relation between the ambiguity function and the random process

${\displaystyle E[A_{X}(\eta ,\tau )]=\int _{-\infty }^{\infty }E[x(t+\tau /2)x^{*}(t-\tau /2)]e^{-j2\pi t\eta }dt}$ ${\displaystyle =\int _{-\infty }^{\infty }R_{x}(t,\tau )e^{-j2\pi t\eta }dt}$

• Stationary random process: the statistical properties do not change with t. Its auto-covariance function:

${\displaystyle R_{x}(t_{1},\tau )=R_{x}(t_{2},\tau )=R_{x}(\tau )}$ for any ${\displaystyle t}$, Therefore, ${\displaystyle R_{x}(\tau )=E[x(\tau /2)x^{*}(-\tau /2)]}$ ${\displaystyle =\iint x(\tau /2,\xi _{1})x^{*}(-\tau /2,\xi _{2})P(\xi _{1},\xi _{2})d\xi _{1}d\xi _{2}}$PSD, ${\displaystyle S_{x}(f)=\int _{-\infty }^{\infty }R_{x}(\tau )e^{-j2\pi f\tau }d\tau }$ White noise:

${\displaystyle S_{x}(f)=\sigma }$ , where ${\displaystyle \sigma }$ is some constant.

• When x(t) is stationary,

${\displaystyle E[W_{x}(t,f)]=S_{x}(f)}$ , (invariant with ${\displaystyle t}$)

${\displaystyle E[A_{x}(\eta ,\tau )]=\int _{-\infty }^{\infty }R_{x}(\tau )\cdot e^{-j2\pi t\eta }\cdot dt}$ ${\displaystyle =R_{x}(\tau )\int _{-\infty }^{\infty }e^{-j2\pi t\eta }\cdot dt}$${\displaystyle =R_{x}(\tau )\delta (\eta )}$ , (nonzero only when ${\displaystyle \eta =0}$)

• For white noise,

${\displaystyle E[W_{g}(t,f)]=\sigma }$ ${\displaystyle E[A_{x}(\eta ,\tau )]=\sigma \delta (\tau )\delta (\eta )}$

Filter Design for White noise

${\displaystyle E_{x}}$: energy of the signal

${\displaystyle A}$  : area of the time frequency distribution of the signal

The PSD of the white noise is ${\displaystyle S_{n}(f)=\sigma }$

${\displaystyle SNR\approx 10\log _{10}{\frac {E_{x}}{\iint \limits _{(t,f)\in {\text{signal part}}}S_{x}(t,f)dtdf}}}$

${\displaystyle SNR\approx 10\log _{10}{\frac {E_{x}}{\sigma \mathrm {A} }}}$

• If ${\displaystyle E[W_{x}(t,f)]}$ varies with ${\displaystyle t}$ and ${\displaystyle E[A_{x}(\eta ,\tau )]}$ is nonzero when ${\displaystyle \eta =0}$, then ${\displaystyle x(t)}$ is a non-stationary random process.
• If
  1. ${\displaystyle h(t)=x_{1}(t)+x_{2}(t)+x_{3}(t)+......+x_{k}(t)}$
  2. ${\displaystyle x_{n}(t)}$'s have zero mean for all ${\displaystyle t}$'s
  3. ${\displaystyle x_{n}(t)}$'s are mutually independent for all ${\displaystyle t}$'s and ${\displaystyle \tau }$'s

${\displaystyle E[x_{m}(t+\tau /2)x_{n}^{*}(t-\tau /2)]=E[x_{m}(t+\tau /2)]E[x_{n}^{*}(t-\tau /2)]=0}$ if ${\displaystyle m\neq n}$, then

${\displaystyle E[W_{h}(t,f)]=\sum _{n=1}^{k}E[W_{x_{n}}(t,f)]}$ ${\displaystyle E[A_{h}(\eta ,\tau )]=\sum _{n=1}^{k}E[A_{x_{n}}(\eta ,\tau )]}$

1. Random process for STFT (Short Time Fourier Transform)

${\displaystyle E[x(t)]\neq 0}$ should be satisfied. Otherwise, ${\displaystyle E[X(t,f)]=E[\int _{t-B}^{t+B}x(\tau )w(t-\tau )e^{-j2\pi f\tau }d\tau ]}$ ${\displaystyle =\int _{t-B}^{t+B}E[x(\tau )]w(t-\tau )e^{-j2\pi f\tau }d\tau }$for zero-mean random process, ${\displaystyle E[X(t,f)]=0}$

1. Decompose by the AF and the FRFT Any non-stationary random process can be expressed as a summation of the fractional Fourier transform (or chirp multiplication) of stationary random process.

Applications

The following applications need not only the time–frequency distribution functions but also some operations to the signal. The Linear canonical transform (LCT) is really helpful. By LCTs, the shape and location on the time–frequency plane of a signal can be in the arbitrary form that we want it to be. For example, the LCTs can shift the time–frequency distribution to any location, dilate it in the horizontal and vertical direction without changing its area on the plane, shear (or twist) it, and rotate it (Fractional Fourier transform). This powerful operation, LCT, make it more flexible to analyze and apply the time–frequency distributions.

History

Early work in time–frequency analysis can be seen in the Haar wavelets (1909) of Alfréd Haar, though these were not significantly applied to signal processing. More substantial work was undertaken by Dennis Gabor, such as Gabor atoms (1947), an early form of wavelets, and the Gabor transform, a modified short-time Fourier transform. The Wigner–Ville distribution (Ville 1948, in a signal processing context) was another foundational step.

Particularly in the 1930s and 1940s, early time–frequency analysis developed in concert with quantum mechanics (Wigner developed the Wigner–Ville distribution in 1932 in quantum mechanics, and Gabor was influenced by quantum mechanics – see Gabor atom); this is reflected in the shared mathematics of the position-momentum plane and the time–frequency plane – as in the Heisenberg uncertainty principle (quantum mechanics) and the Gabor limit (time–frequency analysis), ultimately both reflecting a symplectic structure.

An early practical motivation for time–frequency analysis was the development of radar – see ambiguity function.

See also

• Motions in the time-frequency distribution
• Multiresolution analysis
• Spectral density estimation
• Time–frequency analysis for music signals
• Wavelet analysis

References