Definitions for real matrices

An ${\displaystyle n\times n}$ symmetric real matrix ${\displaystyle M}$ is said to be positive-definite if ${\displaystyle \mathbf {x} ^{\operatorname {T} }M\mathbf {x} >0}$ for all non-zero ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {R} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ symmetric real matrix ${\displaystyle M}$ is said to be positive-semidefinite or non-negative-definite if ${\displaystyle \mathbf {x} ^{\operatorname {T} }M\mathbf {x} \geq 0}$ for all ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {R} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ symmetric real matrix ${\displaystyle M}$ is said to be negative-definite if ${\displaystyle \mathbf {x} ^{\operatorname {T} }M\mathbf {x} <0}$ for all non-zero ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {R} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ symmetric real matrix ${\displaystyle M}$ is said to be negative-semidefinite or non-positive-definite if ${\displaystyle x^{\operatorname {T} }Mx\leq 0}$ for all ${\displaystyle x}$ in ${\displaystyle \mathbb {R} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Definitions for complex matrices

The following definitions all involve the term ${\displaystyle \mathbf {x} ^{*}M\mathbf {x} }$. Notice that this is always a real number for any Hermitian square matrix ${\displaystyle M}$.

An ${\displaystyle n\times n}$ Hermitian complex matrix ${\displaystyle M}$ is said to be positive-definite if ${\displaystyle \mathbf {x} ^{*}M\mathbf {x} >0}$ for all non-zero ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {C} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ Hermitian complex matrix ${\displaystyle M}$ is said to be positive semi-definite or non-negative-definite if ${\displaystyle x^{*}Mx\geq 0}$ for all ${\displaystyle x}$ in ${\displaystyle \mathbb {C} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ Hermitian complex matrix ${\displaystyle M}$ is said to be negative-definite if ${\displaystyle \mathbf {x} ^{*}M\mathbf {x} <0}$ for all non-zero ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {C} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ Hermitian complex matrix ${\displaystyle M}$ is said to be negative semi-definite or non-positive-definite if ${\displaystyle \mathbf {x} ^{*}M\mathbf {x} \leq 0}$ for all ${\displaystyle \mathbf {x} }$ in ${\displaystyle \mathbb {C} ^{n}}$. Formally,

An ${\displaystyle n\times n}$ Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.

Consistency between real and complex definitions

Since every real matrix is also a complex matrix, the definitions of "definiteness" for the two classes must agree.

For complex matrices, the most common definition says that ${\displaystyle M}$ is positive-definite if and only if ${\displaystyle \mathbf {z} ^{*}M\mathbf {z} }$ is real and positive for every non-zero complex column vectors ${\displaystyle \mathbf {z} }$. This condition implies that ${\displaystyle M}$ is Hermitian (i.e. its transpose is equal to its conjugate), since ${\displaystyle \mathbf {z} ^{*}M\mathbf {z} }$ being real, it equals its conjugate transpose ${\displaystyle \mathbf {z} ^{*}M^{*}\mathbf {z} }$ for every ${\displaystyle z,}$ which implies ${\displaystyle M=M^{*}}$.

By this definition, a positive-definite real matrix ${\displaystyle M}$ is Hermitian, hence symmetric; and ${\displaystyle \mathbf {z} ^{\operatorname {T} }M\mathbf {z} }$ is positive for all non-zero real column vectors ${\displaystyle \mathbf {z} }$. However the last condition alone is not sufficient for ${\displaystyle M}$ to be positive-definite. For example, if

$${\displaystyle M={\begin{bmatrix}1&1\\-1&1\end{bmatrix}},}$$

then for any real vector ${\displaystyle \mathbf {z} }$ with entries ${\displaystyle a}$ and ${\displaystyle b}$ we have ${\displaystyle \mathbf {z} ^{\operatorname {T} }M\mathbf {z} =\left(a+b\right)a+\left(-a+b\right)b=a^{2}+b^{2}}$, which is always positive if ${\displaystyle \mathbf {z} }$ is not zero. However, if ${\displaystyle \mathbf {z} }$ is the complex vector with entries ${\displaystyle 1}$ and ${\displaystyle i}$, one gets

$${\displaystyle \mathbf {z} ^{*}M\mathbf {z} ={\begin{bmatrix}1&-i\end{bmatrix}}M{\begin{bmatrix}1\\i\end{bmatrix}}={\begin{bmatrix}1+i&1-i\end{bmatrix}}{\begin{bmatrix}1\\i\end{bmatrix}}=2+2i}$$

which is not real. Therefore, ${\displaystyle M}$ is not positive-definite.

On the other hand, for a symmetric real matrix ${\displaystyle M}$, the condition "${\displaystyle \mathbf {z} ^{\operatorname {T} }M\mathbf {z} >0}$ for all nonzero real vectors ${\displaystyle \mathbf {z} }$" does imply that ${\displaystyle M}$ is positive-definite in the complex sense.

Notation

If a Hermitian matrix ${\displaystyle M}$ is positive semi-definite, one sometimes writes ${\displaystyle M\succeq 0}$ and if ${\displaystyle M}$ is positive-definite one writes ${\displaystyle M\succ 0}$. To denote that ${\displaystyle M}$ is negative semi-definite one writes ${\displaystyle M\preceq 0}$ and to denote that ${\displaystyle M}$ is negative-definite one writes ${\displaystyle M\prec 0}$.

The notion comes from functional analysis where positive semidefinite matrices define positive operators. If two matrices ${\displaystyle A}$ and ${\displaystyle B}$ satisfy ${\displaystyle B-A\succeq 0}$, we can define a non-strict partial order ${\displaystyle B\succeq A}$ that is reflexive, antisymmetric, and transitive; It is not a total order, however, as ${\displaystyle B-A}$ in general may be indefinite.

A common alternative notation is ${\displaystyle M\geq 0}$, ${\displaystyle M>0}$, ${\displaystyle M\leq 0}$ and ${\displaystyle M<0}$ for positive semi-definite and positive-definite, negative semi-definite and negative-definite matrices, respectively. This may be confusing, as sometimes nonnegative matrices (respectively, nonpositive matrices) are also denoted in this way.

Uniqueness up to unitary transformations

The decomposition is not unique: if ${\displaystyle M=B^{*}B}$ for some ${\displaystyle k\times n}$ matrix ${\displaystyle B}$ and if ${\displaystyle Q}$ is any unitary ${\displaystyle k\times k}$ matrix (meaning ${\displaystyle Q^{*}Q=QQ^{*}=I}$), then ${\displaystyle M=B^{*}B=B^{*}Q^{*}QB=A^{*}A}$ for ${\displaystyle A=QB}$.

However, this is the only way in which two decompositions can differ: the decomposition is unique up to unitary transformations. More formally, if ${\displaystyle A}$ is a ${\displaystyle k\times n}$ matrix and ${\displaystyle B}$ is a ${\displaystyle \ell \times n}$ matrix such that ${\displaystyle A^{*}A=B^{*}B}$, then there is a ${\displaystyle \ell \times k}$ matrix ${\displaystyle Q}$ with orthonormal columns (meaning ${\displaystyle Q^{*}Q=I_{k\times k}}$) such that ${\displaystyle B=QA}$.^[5] When ${\displaystyle \ell =k}$ this means ${\displaystyle Q}$ is unitary.

This statement has an intuitive geometric interpretation in the real case: let the columns of ${\displaystyle A}$ and ${\displaystyle B}$ be the vectors ${\displaystyle a_{1},\dots ,a_{n}}$ and ${\displaystyle b_{1},\dots ,b_{n}}$ in ${\displaystyle \mathbb {R} ^{k}}$. A real unitary matrix is an orthogonal matrix, which describes a rigid transformation (an isometry of Euclidean space ${\displaystyle \mathbb {R} ^{k}}$) preserving the 0 point (i.e. rotations and reflections, without translations). Therefore, the dot products ${\displaystyle a_{i}\cdot a_{j}}$ and ${\displaystyle b_{i}\cdot b_{j}}$ are equal if and only if some rigid transformation of ${\displaystyle \mathbb {R} ^{k}}$ transforms the vectors ${\displaystyle a_{1},\dots ,a_{n}}$ to ${\displaystyle b_{1},\dots ,b_{n}}$ (and 0 to 0).

Square root

A Hermitian matrix ${\displaystyle M}$ is positive semidefinite if and only if there is a positive semidefinite matrix ${\displaystyle B}$ (in particular ${\displaystyle B}$ is Hermitian, so ${\displaystyle B^{*}=B}$) satisfying ${\displaystyle M=BB}$. This matrix ${\displaystyle B}$ is unique,^[6] is called the non-negative square root of ${\displaystyle M}$, and is denoted with ${\displaystyle B=M^{\frac {1}{2}}}$. When ${\displaystyle M}$ is positive definite, so is ${\displaystyle M^{\frac {1}{2}}}$, hence it is also called the positive square root of ${\displaystyle M}$.

The non-negative square root should not be confused with other decompositions ${\displaystyle M=B^{*}B}$. Some authors use the name square root and ${\displaystyle M^{\frac {1}{2}}}$ for any such decomposition, or specifically for the Cholesky decomposition, or any decomposition of the form ${\displaystyle M=BB}$; others only use it for the non-negative square root.

If ${\displaystyle M>N>0}$ then ${\displaystyle M^{\frac {1}{2}}>N^{\frac {1}{2}}>0}$.

Cholesky decomposition

A Hermitian positive semidefinite matrix ${\displaystyle M}$ can be written as ${\displaystyle M=LL^{*}}$, where ${\displaystyle L}$ is lower triangular with non-negative diagonal (equivalently ${\displaystyle M=B^{*}B}$ where ${\displaystyle B=L^{*}}$ is upper triangular); this is the Cholesky decomposition. If ${\displaystyle M}$ is positive definite, then the diagonal of ${\displaystyle L}$ is positive and the Cholesky decomposition is unique. Conversely if ${\displaystyle L}$ is lower triangular with nonnegative diagonal then ${\displaystyle LL^{*}}$ is positive semidefinite. The Cholesky decomposition is especially useful for efficient numerical calculations. A closely related decomposition is the LDL decomposition, ${\displaystyle M=LDL^{*}}$, where ${\displaystyle D}$ is diagonal and ${\displaystyle L}$ is lower unitriangular.

Induced partial ordering

For arbitrary square matrices ${\displaystyle M}$, ${\displaystyle N}$ we write ${\displaystyle M\geq N}$ if ${\displaystyle M-N\geq 0}$ i.e., ${\displaystyle M-N}$ is positive semi-definite. This defines a partial ordering on the set of all square matrices. One can similarly define a strict partial ordering ${\displaystyle M>N}$. The ordering is called the Loewner order.

Inverse of positive definite matrix

Every positive definite matrix is invertible and its inverse is also positive definite.^[9] If ${\displaystyle M\geq N>0}$ then ${\displaystyle N^{-1}\geq M^{-1}>0}$.^[10] Moreover, by the min-max theorem, the kth largest eigenvalue of ${\displaystyle M}$ is greater than or equal to the kth largest eigenvalue of ${\displaystyle N}$.

Scaling

If ${\displaystyle M}$ is positive definite and ${\displaystyle r>0}$ is a real number, then ${\displaystyle rM}$ is positive definite.^[11]

Addition

• If ${\displaystyle M}$ and ${\displaystyle N}$ are positive-definite, then the sum ${\displaystyle M+N}$ is also positive-definite.^[11]
• If ${\displaystyle M}$ and ${\displaystyle N}$ are positive-semidefinite, then the sum ${\displaystyle M+N}$ is also positive-semidefinite.
• If ${\displaystyle M}$ is positive-definite and ${\displaystyle N}$ is positive-semidefinite, then the sum ${\displaystyle M+N}$ is also positive-definite.

Multiplication

• If ${\displaystyle M}$ and ${\displaystyle N}$ are positive definite, then the products ${\displaystyle MNM}$ and ${\displaystyle NMN}$ are also positive definite. If ${\displaystyle MN=NM}$, then ${\displaystyle MN}$ is also positive definite.
• If ${\displaystyle M}$ is positive semidefinite, then ${\displaystyle A^{*}MA}$ is positive semidefinite for any (possibly rectangular) matrix ${\displaystyle A}$. If ${\displaystyle M}$ is positive definite and ${\displaystyle A}$ has full column rank, then ${\displaystyle A^{*}MA}$ is positive definite.^[12]

Trace

The diagonal entries ${\displaystyle m_{ii}}$ of a positive-semidefinite matrix are real and non-negative. As a consequence the trace, ${\displaystyle \operatorname {tr} (M)\geq 0}$. Furthermore,^[13] since every principal sub-matrix (in particular, 2-by-2) is positive semidefinite,

$${\displaystyle \left|m_{ij}\right|\leq {\sqrt {m_{ii}m_{jj}}}\quad \forall i,j}$$

and thus, when ${\displaystyle n\geq 1}$,

$${\displaystyle \max _{i,j}\left|m_{ij}\right|\leq \max _{i}m_{ii}}$$

An ${\displaystyle n\times n}$ Hermitian matrix ${\displaystyle M}$ is positive definite if it satisfies the following trace inequalities:^[14]

$${\displaystyle \operatorname {tr} (M)>0\quad \mathrm {and} \quad {\frac {(\operatorname {tr} (M))^{2}}{\operatorname {tr} (M^{2})}}>n-1.}$$

Another important result is that for any ${\displaystyle M}$ and ${\displaystyle N}$ positive-semidefinite matrices, ${\displaystyle \operatorname {tr} (MN)\geq 0}$. This follows by writing ${\displaystyle \operatorname {tr} (MN)=\operatorname {tr} (M^{\frac {1}{2}}NM^{\frac {1}{2}})}$. The matrix ${\displaystyle M^{\frac {1}{2}}NM^{\frac {1}{2}}}$ is positive-semidefinite and thus has non-negative eigenvalues, whose sum, the trace, is therefore also non-negative.

Hadamard product

If ${\displaystyle M,N\geq 0}$, although ${\displaystyle MN}$ is not necessary positive semidefinite, the Hadamard product is, ${\displaystyle M\circ N\geq 0}$ (this result is often called the Schur product theorem).^[15]

Regarding the Hadamard product of two positive semidefinite matrices ${\displaystyle M=(m_{ij})\geq 0}$, ${\displaystyle N\geq 0}$, there are two notable inequalities:

• Oppenheim's inequality: ${\displaystyle \det(M\circ N)\geq \det(N)\prod \nolimits _{i}m_{ii}.}$^[16]
• ${\displaystyle \det(M\circ N)\geq \det(M)\det(N)}$.^[17]

Kronecker product

If ${\displaystyle M,N\geq 0}$, although ${\displaystyle MN}$ is not necessary positive semidefinite, the Kronecker product ${\displaystyle M\otimes N\geq 0}$.

Frobenius product

If ${\displaystyle M,N\geq 0}$, although ${\displaystyle MN}$ is not necessary positive semidefinite, the Frobenius inner product ${\displaystyle M:N\geq 0}$ (Lancaster–Tismenetsky, The Theory of Matrices, p. 218).

Convexity

The set of positive semidefinite symmetric matrices is convex. That is, if ${\displaystyle M}$ and ${\displaystyle N}$ are positive semidefinite, then for any ${\displaystyle \alpha }$ between 0 and 1, ${\displaystyle \alpha M+\left(1-\alpha \right)N}$ is also positive semidefinite. For any vector ${\displaystyle \mathbf {x} }$:

$${\displaystyle \mathbf {x} ^{\operatorname {T} }\left(\alpha M+\left(1-\alpha \right)N\right)\mathbf {x} =\alpha \mathbf {x} ^{\operatorname {T} }M\mathbf {x} +(1-\alpha )\mathbf {x} ^{\operatorname {T} }N\mathbf {x} \geq 0.}$$

This property guarantees that semidefinite programming problems converge to a globally optimal solution.

Relation with cosine

The positive-definiteness of a matrix ${\displaystyle A}$ expresses that the angle ${\displaystyle \theta }$ between any vector ${\displaystyle \mathbf {x} }$ and its image ${\displaystyle A\mathbf {x} }$ is always ${\displaystyle -\pi /2<\theta <+\pi /2}$:

$${\displaystyle \cos \theta ={\frac {\mathbf {x} ^{T}A\mathbf {x} }{\lVert \mathbf {x} \rVert \lVert A\mathbf {x} \rVert }}={\frac {\langle \mathbf {x} ,A\mathbf {x} \rangle }{\lVert \mathbf {x} \rVert \lVert A\mathbf {x} \rVert }},\theta =\theta (\mathbf {x} ,A\mathbf {x} )={\widehat {\mathbf {x} ,A\mathbf {x} }}={\text{the angle between }}\mathbf {x} {\text{ and }}A\mathbf {x} }$$

Further properties

1. If ${\displaystyle M}$ is a symmetric Toeplitz matrix, i.e. the entries ${\displaystyle m_{ij}}$ are given as a function of their absolute index differences: ${\displaystyle m_{ij}=h(|i-j|)}$, and the strict inequality ${\textstyle \sum _{j\neq 0}\left|h(j)\right|<h(0)}$ holds, then ${\displaystyle M}$ is strictly positive definite.
2. Let ${\displaystyle M>0}$ and ${\displaystyle N}$ Hermitian. If ${\displaystyle MN+NM\geq 0}$ (resp., ${\displaystyle MN+NM>0}$) then ${\displaystyle N\geq 0}$ (resp., ${\displaystyle N>0}$).^[18]
3. If ${\displaystyle M>0}$ is real, then there is a ${\displaystyle \delta >0}$ such that ${\displaystyle M>\delta I}$, where ${\displaystyle I}$ is the identity matrix.
4. If ${\displaystyle M_{k}}$ denotes the leading ${\displaystyle k\times k}$ minor, ${\displaystyle \det \left(M_{k}\right)/\det \left(M_{k-1}\right)}$ is the kth pivot during LU decomposition.
5. A matrix is negative definite if its k-th order leading principal minor is negative when ${\displaystyle k}$ is odd, and positive when ${\displaystyle k}$ is even.
6. If ${\displaystyle M}$ is a real positive definite matrix, then there exists a positive real number ${\displaystyle m}$ such that for every vector ${\displaystyle \mathbf {v} }$, ${\displaystyle \mathbf {v} ^{\operatorname {T} }M\mathbf {v} \geq m\|\mathbf {v} \|_{2}^{2}}$.
7. A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is however not enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and −1.

Block matrices and submatrices

A positive ${\displaystyle 2n\times 2n}$ matrix may also be defined by blocks:

$${\displaystyle M={\begin{bmatrix}A&B\\C&D\end{bmatrix}}}$$

where each block is ${\displaystyle n\times n}$. By applying the positivity condition, it immediately follows that ${\displaystyle A}$ and ${\displaystyle D}$ are hermitian, and ${\displaystyle C=B^{*}}$.

We have that ${\displaystyle \mathbf {z} ^{*}M\mathbf {z} \geq 0}$ for all complex ${\displaystyle \mathbf {z} }$, and in particular for ${\displaystyle \mathbf {z} =[\mathbf {v} ,0]^{\operatorname {T} }}$. Then

$${\displaystyle {\begin{bmatrix}\mathbf {v} ^{*}&0\end{bmatrix}}{\begin{bmatrix}A&B\\B^{*}&D\end{bmatrix}}{\begin{bmatrix}\mathbf {v} \\0\end{bmatrix}}=\mathbf {v} ^{*}A\mathbf {v} \geq 0.}$$

A similar argument can be applied to ${\displaystyle D}$, and thus we conclude that both ${\displaystyle A}$ and ${\displaystyle D}$ must be positive definite. The argument can be extended to show that any principal submatrix of ${\displaystyle M}$ is itself positive definite.

Converse results can be proved with stronger conditions on the blocks, for instance, using the Schur complement.

Local extrema

A general quadratic form ${\displaystyle f(\mathbf {x} )}$ on ${\displaystyle n}$ real variables ${\displaystyle x_{1},\ldots ,x_{n}}$ can always be written as ${\displaystyle \mathbf {x} ^{\operatorname {T} }M\mathbf {x} }$ where ${\displaystyle \mathbf {x} }$ is the column vector with those variables, and ${\displaystyle M}$ is a symmetric real matrix. Therefore, the matrix being positive definite means that ${\displaystyle f}$ has a unique minimum (zero) when ${\displaystyle \mathbf {x} }$ is zero, and is strictly positive for any other ${\displaystyle \mathbf {x} }$.

More generally, a twice-differentiable real function ${\displaystyle f}$ on ${\displaystyle n}$ real variables has local minimum at arguments ${\displaystyle x_{1},\ldots ,x_{n}}$ if its gradient is zero and its Hessian (the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can be made for negative definite and semi-definite matrices.

Covariance

In statistics, the covariance matrix of a multivariate probability distribution is always positive semi-definite; and it is positive definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.

Heat conductivity matrix

Fourier's law of heat conduction, giving heat flux ${\displaystyle \mathbf {q} }$ in terms of the temperature gradient ${\displaystyle \mathbf {g} =\nabla T}$ is written for anisotropic media as ${\displaystyle \mathbf {q} =-K\mathbf {g} }$, in which ${\displaystyle K}$ is the symmetric thermal conductivity matrix. The negative is inserted in Fourier's law to reflect the expectation that heat will always flow from hot to cold. In other words, since the temperature gradient ${\displaystyle \mathbf {g} }$ always points from cold to hot, the heat flux ${\displaystyle \mathbf {q} }$ is expected to have a negative inner product with ${\displaystyle \mathbf {g} }$ so that ${\displaystyle \mathbf {q} ^{\operatorname {T} }\mathbf {g} <0}$. Substituting Fourier's law then gives this expectation as ${\displaystyle \mathbf {g} ^{\operatorname {T} }K\mathbf {g} >0}$, implying that the conductivity matrix should be positive definite.