Classical Estimation Theory

classical estimate

It is a technique to estimate the power spectrum of a single dimensional or a multidimensional signal as it cannot be calculated accurately. Given are samples of a wide sense stationary random process and its second order statistics (measurements).The estimates are obtained by applying a multidimensional Fourier transform of the autocorrelation function of the random signal. The estimation begins by calculating a periodogram which is obtained by squaring the magnitude of the multidimensional Fourier transform of the measurements ri(n). The spectral estimates obtained from the periodogram have a large variance in amplitude for consecutive periodogram samples or in wavenumber. This problem is resolved using techniques that constitute the classical estimation theory. They are as follows: 1.Bartlett suggested a method that averages the spectral estimates to calculate the power spectrum. The measurements are divided into equally spaced segments in time and an average is taken. This gives a better estimate.^[3] 2.Based on the wavenumber and index of the receiver/output we can partition the segments. This increases the spectral estimates and decreases the variances between consecutive segments. 3.Welch suggested that we should divide the measurements using data window functions, calculate a periodogram, average them to get a spectral estimate and calculate the power spectrum using Fast Fourier Transform (FFT). This increases the computational speed.^[4] 4.Smoothing window will help us smoothen the estimate by multiplying the periodogram with a smoothening spectrum. Wider the main lobe of the smoothening spectrum, smoother it becomes at the cost of frequency resolution.^[2]

${\displaystyle P\left(K_{x},w\right)=\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\varphi _{ss}\left(x,t\right)\ e^{-j\left(wt-k'x\right)}\,dx\,dt}$

${\displaystyle \varphi _{ss}\left(x,t\right)=s\left[\left(\xi ,\tau \right)s^{*}\left(\xi -x,\tau -t\right)\right]}$ ^[2]

${\displaystyle P_{B}\left(w\right)={\frac {1}{detN}}\sum _{l}|\sum _{n}\ x\left(n+MI\right)\ e^{-j\left(w'n\right)}|^{2}}$ Bartlett's case ^[2]

${\displaystyle P_{M}\left(w\right)={\frac {1}{detN}}|\sum _{n}\ g\left(n\right)\ x\left(n\right)\ e^{-j\left(w'n\right)}|^{2}}$ Modified periodogram ^[2]

${\displaystyle P_{W}\left(w\right)={\frac {1}{detN}}\sum _{l}|\sum _{n}\ g\left(n\right)\ x\left(n+MI\right)\ e^{-j\left(w'n\right)}|^{2}}$ Welch's case ^[2]

Advantages

Straightforward method involving Fourier transforms.

Limitations

1. Since some of the above methods sample the sequence in time, the frequency resolution is reduced (aliasing).
2. Number of instances of a wide sense stationary random process are less which makes it difficult to calculate the estimates accurately.

High Resolution Spectral Estimations

This method gives a better estimate whose frequency resolution is higher than the classical estimation theory. In the high resolution estimation method we use a variable wavenumber window which allows only certain wavenumbers and suppresses the others. Capon’s ^[5] work helped us establish an estimation method by using wavenumber-frequency components. This results in an estimate with a higher frequency resolution. It is similar to maximum likelihood method as the optimization tool used is similar.

Assumption

The output obtained from the sensors is a wide sense stationary random process with zero mean.^[6]

${\displaystyle P_{C}\left(K_{o}x,w_{o}\right)=E\left[|y\left(i,n\right)|^{2}\right]={\frac {1}{\sum _{\alpha =0}^{N-1}\sum _{\beta =0}^{M-1}\sum _{l=0}^{N-1}\sum _{m=0}^{M-1}\psi _{e}\left(l,\alpha ;m,\beta \right)}}}$ ^[2]

Advantages

1. Higher frequency resolution compare to other existing methods.
2. Better frequency estimate since we are using a variable wavenumber window as compared to classical method which uses a fixed wavenumber window.
3. Faster Computational speed as it uses FFT.

Separable Spectral Estimator

In this type of estimation, we select the multidimensional signal to be a separable function.^[1] Because of this property we will be able to view the Fourier analysis taking place in multiple dimensions successively. A time delay in the magnitude squaring operation will help us process the Fourier transformation in each dimension. A Discrete time Multidimensional Fourier transform is applied along each dimension and in the end a maximum entropy estimator is applied and the magnitude is squared.

Advantages

1. The Fourier analysis is flexible as the signal is separable.
2. It preserves the phase components of every dimension unlike other spectral estimators.

All-pole Spectral Modelling

This method is an extension of a 1-D technique called Autoregressive spectral estimation. In autoregressive models, the output variables depend linearly on its own previous values. In this model, the estimation of power spectrum is reduced to estimating the coefficients from the autocorrelation coefficients of the random process which are assumed to be known for a specific region. The power spectrum ${\displaystyle P_{A}(k_{x},w)}$ of a random process ${\displaystyle r(i,n)}$ is given by:^[2]

${\displaystyle P_{A}\left(k_{x},w\right)=P_{e}\left(k_{x},w\right)|{\frac {1}{1-A\left(k_{x},w\right)}}|^{2}}$

Above, ${\displaystyle P_{e}\left(k_{x},w\right)}$ is the power spectrum of a random process ${\displaystyle e(i,n)}$ , which is given as the input to a system with a transfer function ${\displaystyle |{\frac {1}{1-A\left(k_{x},w\right)}}|}$ to obtain ${\displaystyle r(i,n)}$^[2] and ${\displaystyle A\left(k_{x},w\right)}$ is:

${\displaystyle A\left(k_{x},w\right)=\sum _{p=o}^{N-1}\sum _{q=0}^{M-1}a(p,q)exp(jk_{x}p-jwq)}$

Therefore, the power estimation reduces to estimation of coefficients of ${\displaystyle a\left(p,q\right)}$ from the auto correlation function ${\displaystyle \varphi \left(l,m\right)}$of the random process. The coefficients can also be estimated using the linear prediction formulation which deals with minimization of mean square error between the actual random signal and predicted values of the random signal.

Limitations

1. In 1-D we have the same number of linear equations with the same number of unknowns because of the autocorrelation matching property. But it may not be possible in multi-D ^[2] since the set of parameters does not contain enough degrees of freedom to match autocorrelation coefficients.
2. We assume that the array of coefficients is limited to a certain area.
3. In 1-D formulation of linear prediction, the inverse filter has minimum phase property thus proving that the filter is stable. It is not always necessarily true in multi-D case.
4. In 1-D formulation, the autocorrelation matrix is positive definite but positive definite extension may not exist in the case of multi-D.

Maximum Entropy Spectral Estimation

Maximum entropy spectral estimation.

In this method of spectral estimation, we try to find the spectral estimate whose inverse Fourier transform matches the known auto correlation coefficients. We maximize the entropy of the spectral estimate such that it matches the autocorrelation coefficients.^[2] The entropy equation is given as:^[1][2]

${\displaystyle H={\frac {1}{4\pi ^{2}}}\int _{-\pi }^{\pi }\int _{-\pi }^{\pi }logP\left(k_{x},w\right)dk_{x}dw}$

The power spectrum ${\displaystyle P\left(k,w\right)}$ can be expressed as a sum of known autocorrelation coefficients and unknown autocorrelation coefficients. By adjusting the values of unconstrained coefficients, the entropy can be maximized.

The max entropy is of the form:^[2]

${\displaystyle P_{ME}={\frac {1}{\sum _{l}\sum _{m}\lambda \left(l,m\right)exp\left(jk_{x}l-jwm\right)}}}$^[1]

λ(l,m) must be chosen such that known autocorrelation coefficients are matched.

Limitations

1. It has constrained optimization. It can be overcome by using the method of Lagrange multipliers.^[2]
2. All pole spectral estimate is not the solution to maximum entropy in multidimensional case as it is in the case of 1-D. This is because the all pole spectral model does not contain enough degree of freedom to match the know autocorrelation coefficients.

Advantages

Errors in measuring or estimating the known autocorrelation coefficients can be taken into account since exact match is not required.

Disadvantage

Too many computations are required.

Improved Maximum Likelihood Method(IMLM)

This is a relatively new approach. Improved maximum likelihood method (IMLM) is a combination of two MLM(maximum likelihood) estimators.^[1][7] The improved maximum likelihood of two 2-dimensional arrays A and B at a wave number k( gives information about the orientation of the array in space) is given by the relation:^[8]

${\displaystyle IMLM\left(k:A,B\right)={\frac {1}{{\frac {1}{MLM\left(k:A\right)}}-{\frac {1}{MLM\left(k:B\right)}}}}}$^[7]

Array B is a subset of A. Therefore, assuming that A>B, if there is a difference between the MLM of A and MLM of B then significant part of the estimated spectral energy at the frequency may be due to power leakage from other frequencies. The de-emphasis of MLM of A may improve spectral estimate. This is accomplished by multiplying by a weighted function which is smaller when there is a greater difference between MLA of B and MLA of A.

${\displaystyle IMLM\left(k:A,B\right)={\frac {MLM\left(k:A\right)MLM\left(k:B\right)}{MLM\left(k:B\right)-MLM\left(k:A\right)}}}$

${\displaystyle IMLM\left(k:A,B\right)=MLM\left(k:A\right)W_{AB}\left(k\right)}$

where ${\displaystyle W_{AB}\left(k\right)}$ is the weighting function and is given by the expression:^[7]

${\displaystyle W_{AB}\left(k\right)={\frac {MLM\left(k:B\right)}{MLM\left(k:B\right)-MLM\left(k:A\right)}}}$

Advantages

1. Used as an alternative to MLM or MEM(Maximum Entropy Method/principle of maximum entropy)
2. IMLM has better resolution than MLM and it requires lesser number of computations when compared to MEM ^[7][8]