Upper Hessenberg matrix

A square ${\displaystyle n\times n}$ matrix ${\displaystyle A}$ is said to be in upper Hessenberg form or to be an upper Hessenberg matrix if ${\displaystyle a_{i,j}=0}$ for all ${\displaystyle i,j}$ with ${\displaystyle i>j+1}$.

An upper Hessenberg matrix is called unreduced if all subdiagonal entries are nonzero, i.e. if ${\displaystyle a_{i+1,i}\neq 0}$ for all ${\displaystyle i\in \{1,\ldots ,n-1\}}$.^[4]

Lower Hessenberg matrix

A square ${\displaystyle n\times n}$ matrix ${\displaystyle A}$ is said to be in lower Hessenberg form or to be a lower Hessenberg matrix if its transpose ${\displaystyle }$ is an upper Hessenberg matrix or equivalently if ${\displaystyle a_{i,j}=0}$ for all ${\displaystyle i,j}$ with ${\displaystyle j>i+1}$.

A lower Hessenberg matrix is called unreduced if all superdiagonal entries are nonzero, i.e. if ${\displaystyle a_{i,i+1}\neq 0}$ for all ${\displaystyle i\in \{1,\ldots ,n-1\}}$.

Householder transformations

Any ${\displaystyle n\times n}$ matrix can be transformed into a Hessenberg matrix by a similarity transformation using Householder transformations. The following procedure for such a transformation is adapted from A Second Course In Linear Algebra by Garcia & Roger.^[5]

Let ${\displaystyle A}$ be any real or complex ${\displaystyle n\times n}$ matrix, then let ${\displaystyle A^{\prime }}$ be the ${\displaystyle (n-1)\times n}$ submatrix of ${\displaystyle A}$ constructed by removing the first row in ${\displaystyle A}$ and let ${\displaystyle \mathbf {a} _{1}^{\prime }}$ be the first column of ${\displaystyle A'}$. Construct the ${\displaystyle (n-1)\times (n-1)}$ householder matrix ${\displaystyle V_{1}=I_{(n-1)}-2{\frac {ww^{*}}{\|w\|^{2}}}}$ where

$${\displaystyle w={\begin{cases}\|\mathbf {a} _{1}^{\prime }\|_{2}\mathbf {e} _{1}-\mathbf {a} _{1}^{\prime }\;\;\;\;\;\;\;\;,\;\;\;a_{11}^{\prime }=0\\\|\mathbf {a} _{1}^{\prime }\|_{2}\mathbf {e} _{1}+{\frac {\overline {a_{11}^{\prime }}}{|a_{11}^{\prime }|}}\mathbf {a} _{1}^{\prime }\;\;\;,\;\;\;a_{11}^{\prime }\neq 0\\\end{cases}}}$$

This householder matrix will map ${\displaystyle \mathbf {a} _{1}^{\prime }}$ to ${\displaystyle \|\mathbf {a} _{1}^{\prime }\|\mathbf {e} _{1}}$ and as such, the block matrix ${\displaystyle U_{1}={\begin{bmatrix}1&\mathbf {0} \\\mathbf {0} &V_{1}\end{bmatrix}}}$ will map the matrix ${\displaystyle A}$ to the matrix ${\displaystyle U_{1}A}$ which has only zeros below the second entry of the first column. Now construct ${\displaystyle (n-2)\times (n-2)}$ householder matrix ${\displaystyle V_{2}}$ in a similar manner as ${\displaystyle V_{1}}$ such that ${\displaystyle V_{2}}$ maps the first column of ${\displaystyle A^{\prime \prime }}$ to ${\displaystyle \|\mathbf {a} _{1}^{\prime \prime }\|\mathbf {e} _{1}}$, where ${\displaystyle A^{\prime \prime }}$ is the submatrix of ${\displaystyle A^{\prime }}$ constructed by removing the first row and the first column of ${\displaystyle A^{\prime }}$, then let ${\displaystyle U_{2}={\begin{bmatrix}1&0&0\\0&1&0\\0&0&V_{2}\end{bmatrix}}}$ which maps ${\displaystyle U_{1}A}$ to the matrix ${\displaystyle U_{2}U_{1}A}$ which has only zeros below the first and second entry of the subdiagonal. Now construct ${\displaystyle V_{3}}$ and then ${\displaystyle U_{3}}$ in a similar manner, but for the matrix ${\displaystyle A^{\prime \prime \prime }}$ constructed by removing the first row and first column of ${\displaystyle A^{\prime \prime }}$ and proceed as in the previous steps. Continue like this for a total of ${\displaystyle n-2}$ steps.

By construction of ${\displaystyle U_{k}}$, the first ${\displaystyle k}$ columns of any ${\displaystyle n\times n}$ matrix are invariant under multiplication by ${\displaystyle U_{k}^{*}}$ from the right. Hence, any matrix can be transformed to an upper Hessenberg matrix by a similarity transformation of the form ${\displaystyle U_{(n-2)}(\dots (U_{2}(U_{1}AU_{1}^{*})U_{2}^{*})\dots )U_{(n-2)}^{*}=U_{(n-2)}\dots U_{2}U_{1}A(U_{(n-2)}\dots U_{2}U_{1})^{*}=UAU^{*}}$.

Jacobi (Givens) rotations

A Jacobi rotation (also called Givens rotation) is an orthogonal matrix transformation in the form

${\displaystyle A\to A'=J(p,q,\theta )^{T}AJ(p,q,\theta )\;,}$

where ${\displaystyle J(p,q,\theta )}$, ${\displaystyle p<q}$, is the Jacobi rotation matrix with all matrix elements equal zero except for

${\displaystyle \left\{{\begin{aligned}J(p,q,\theta )_{ii}&{}=1\;\forall i\neq p,q\\J(p,q,\theta )_{pp}&{}=\cos(\theta )\\J(p,q,\theta )_{qq}&{}=\cos(\theta )\\J(p,q,\theta )_{pq}&{}=\sin(\theta )\\J(p,q,\theta )_{qp}&{}=-\sin(\theta )\;.\end{aligned}}\right.}$

One can zero the matrix element ${\displaystyle A'_{p-1,q}}$ by choosing the rotation angle ${\displaystyle \theta }$ to satisfy the equation

${\displaystyle A_{p-1,p}\sin \theta +A_{p-1,q}\cos \theta =0\;,}$

Now, the sequence of such Jacobi rotations with the following ${\displaystyle (p,q)}$

${\displaystyle (p,q)=(2,3),(2,4),\dots ,(2,n),(3,4),\dots ,(3,n),\dots ,(n-1,n)}$

reduces the matrix ${\displaystyle A}$ to the lower Hessenberg form.^[6]