Find the number of occurrences of a pattern

In order to find the number of occurrences of a given string ${\displaystyle P}$ (length ${\displaystyle m}$) in a text ${\displaystyle T}$ (length ${\displaystyle N}$),^[3]

• We use binary search against the suffix array of ${\displaystyle T}$ to find the starting and end position of all occurrences of ${\displaystyle P}$.
• Now to speed up the search, we use LCP array, specifically a special version of the LCP array (LCP-LR below).

The issue with using standard binary search (without the LCP information) is that in each of the ${\displaystyle O(\log N)}$ comparisons needed to be made, we compare P to the current entry of the suffix array, which means a full string comparison of up to m characters. So the complexity is ${\displaystyle O(m\log N)}$.

The LCP-LR array helps improve this to ${\displaystyle O(m+\log N)}$, in the following way:

At any point during the binary search algorithm, we consider, as usual, a range ${\displaystyle (L,\dots ,R)}$ of the suffix array and its central point ${\displaystyle M}$, and decide whether we continue our search in the left sub-range ${\displaystyle (L,\dots ,M)}$ or in the right sub-range ${\displaystyle (M,\dots ,R)}$. In order to make the decision, we compare ${\displaystyle P}$ to the string at ${\displaystyle M}$. If ${\displaystyle P}$ is identical to ${\displaystyle M}$, our search is complete. But if not, we have already compared the first ${\displaystyle k}$ characters of ${\displaystyle P}$ and then decided whether ${\displaystyle P}$ is lexicographically smaller or larger than ${\displaystyle M}$. Let's assume the outcome is that ${\displaystyle P}$ is larger than ${\displaystyle M}$. So, in the next step, we consider ${\displaystyle (M,\dots ,R)}$ and a new central point ${\displaystyle M'}$ in the middle:

             M ...... M' ...... R
             |
      we know:
         lcp(P,M)==k

The trick now is that LCP-LR is precomputed such that an ${\displaystyle O(1)}$-lookup tells us the longest common prefix of ${\displaystyle M}$ and ${\displaystyle M'}$, ${\displaystyle \mathrm {lcp} (M,M')}$.

We already know (from the previous step) that ${\displaystyle M}$ itself has a prefix of ${\displaystyle k}$ characters in common with ${\displaystyle P}$: ${\displaystyle \mathrm {lcp} (P,M)=k}$. Now there are three possibilities:

• Case 1: ${\displaystyle k<\mathrm {lcp} (M,M')}$, i.e. ${\displaystyle P}$ has fewer prefix characters in common with M than M has in common with M'. This means the (k+1)-th character of M' is the same as that of M, and since P is lexicographically larger than M, it must be lexicographically larger than M', too. So we continue in the right half (M',...,R).
• Case 2: ${\displaystyle k>\mathrm {lcp} (M,M')}$, i.e. ${\displaystyle P}$ has more prefix characters in common with ${\displaystyle M}$ than ${\displaystyle M}$ has in common with ${\displaystyle M'}$. Consequently, if we were to compare ${\displaystyle P}$ to ${\displaystyle M'}$, the common prefix would be smaller than ${\displaystyle k}$, and ${\displaystyle M'}$ would be lexicographically larger than ${\displaystyle P}$, so, without actually making the comparison, we continue in the left half ${\displaystyle (M,\dots ,M')}$.
• Case 3: ${\displaystyle k=\mathrm {lcp} (M,M')}$. So M and M' are both identical with ${\displaystyle P}$ in the first ${\displaystyle k}$ characters. To decide whether we continue in the left or right half, it suffices to compare ${\displaystyle P}$ to ${\displaystyle M'}$ starting from the ${\displaystyle (k+1)}$th character.
• We continue recursively.

The overall effect is that no character of ${\displaystyle P}$ is compared to any character of the text more than once. The total number of character comparisons is bounded by ${\displaystyle m}$, so the total complexity is indeed ${\displaystyle O(m+\log N)}$.

We still need to precompute LCP-LR so it is able to tell us in ${\displaystyle O(1)}$ time the lcp between any two entries of the suffix array. We know the standard LCP array gives us the lcp of consecutive entries only, i.e. ${\displaystyle \mathrm {lcp} (i-1,i)}$ for any ${\displaystyle i}$. However, ${\displaystyle M}$ and ${\displaystyle M'}$ in the description above are not necessarily consecutive entries.

The key to this is to realize that only certain ranges ${\displaystyle (L,\dots ,R)}$ will ever occur during the binary search: It always starts with ${\displaystyle (0,\dots ,N)}$ and divides that at the center, and then continues either left or right and divide that half again and so forth. Another way of looking at it is : every entry of the suffix array occurs as central point of exactly one possible range during binary search. So there are exactly N distinct ranges ${\displaystyle (L\dots M\dots R)}$ that can possibly play a role during binary search, and it suffices to precompute ${\displaystyle \mathrm {lcp} (L,M)}$ and ${\displaystyle \mathrm {lcp} (M,R)}$ for those ${\displaystyle N}$ possible ranges. So that is ${\displaystyle 2N}$ distinct precomputed values, hence LCP-LR is ${\displaystyle O(N)}$ in size.

Moreover, there is a straightforward recursive algorithm to compute the ${\displaystyle 2N}$ values of LCP-LR in ${\displaystyle O(N)}$ time from the standard LCP array.

To sum up:

• It is possible to compute LCP-LR in ${\displaystyle O(N)}$ time and ${\displaystyle O(2N)=O(N)}$ space from LCP.
• Using LCP-LR during binary search helps accelerate the search procedure from ${\displaystyle O(M\log N)}$ to ${\displaystyle O(M+\log N)}$.
• We can use two binary searches to determine the left and right end of the match range for ${\displaystyle P}$, and the length of the match range corresponds with the number of occurrences for P.

Suffix tree construction

Given the suffix array ${\displaystyle A}$ and the LCP array ${\displaystyle H}$ of a string ${\displaystyle S=s_{1},s_{2},\ldots s_{n}\$}$ of length ${\displaystyle n+1}$, its suffix tree ${\displaystyle ST}$ can be constructed in ${\displaystyle O(n)}$ time based on the following idea: Start with the partial suffix tree for the lexicographically smallest suffix and repeatedly insert the other suffixes in the order given by the suffix array.

Let ${\displaystyle ST_{i}}$ be the partial suffix tree for ${\displaystyle 0\leq i\leq n}$. Further let ${\displaystyle d(v)}$ be the length of the concatenation of all path labels from the root of ${\displaystyle ST_{i}}$ to node ${\displaystyle v}$.

Case 1 (${\displaystyle d(v)=H[i+1]}$): Suppose the suffixes ${\displaystyle a\$}$, ${\displaystyle ana\$}$, ${\displaystyle anana\$}$ and ${\displaystyle banana\$}$ of the string ${\displaystyle S=banana\$}$ are already added to the suffix tree. Then the suffix ${\displaystyle na\$}$ is added to the tree as shown in the picture. The rightmost path is highlighted in red.

Start with ${\displaystyle ST_{0}}$, the tree consisting only of the root. To insert ${\displaystyle A[i+1]}$ into ${\displaystyle ST_{i}}$, walk up the rightmost path beginning at the recently inserted leaf ${\displaystyle A[i]}$ to the root, until the deepest node ${\displaystyle v}$ with ${\displaystyle d(v)\leq H[i+1]}$ is reached.

We need to distinguish two cases:

• ${\displaystyle d(v)=H[i+1]}$: This means that the concatenation of the labels on the root-to-${\displaystyle v}$ path equals the longest common prefix of suffixes ${\displaystyle A[i]}$ and ${\displaystyle A[i+1]}$.
  In this case, insert ${\displaystyle A[i+1]}$ as a new leaf ${\displaystyle x}$ of node ${\displaystyle v}$ and label the edge ${\displaystyle (v,x)}$ with ${\displaystyle S[A[i+1]+H[i+1],n]}$. Thus the edge label consists of the remaining characters of suffix ${\displaystyle A[i+1]}$ that are not already represented by the concatenation of the labels of the root-to-${\displaystyle v}$ path.
  This creates the partial suffix tree ${\displaystyle ST_{i+1}}$.
  

  Case 2 (${\displaystyle d(v)<H[i+1]}$): In order to add suffix ${\displaystyle nana\$}$, the edge to the previously inserted suffix ${\displaystyle na\$}$ has to be split up. The new edge to the new internal node is labeled with the longest common prefix of the suffixes ${\displaystyle na\$}$ and ${\displaystyle nana\$}$. The edges connecting the two leaves are labeled with the remaining suffix characters that are not part of the prefix.
• ${\displaystyle d(v)<H[i+1]}$: This means that the concatenation of the labels on the root-to-${\displaystyle v}$ path displays less characters than the longest common prefix of suffixes ${\displaystyle A[i]}$ and ${\displaystyle A[i+1]}$ and the missing characters are contained in the edge label of ${\displaystyle v}$'s rightmost edge. Therefore, we have to split up that edge as follows:
  Let ${\displaystyle w}$ be the child of ${\displaystyle v}$ on ${\displaystyle ST_{i}}$'s rightmost path.

1. Delete the edge ${\displaystyle (v,w)}$.
2. Add a new internal node ${\displaystyle y}$ and a new edge ${\displaystyle (v,y)}$ with label ${\displaystyle S[A[i]+d(v),A[i]+H[i+1]-1]}$. The new label consists of the missing characters of the longest common prefix of ${\displaystyle A[i]}$ and ${\displaystyle A[i+1]}$. Thus, the concatenation of the labels of the root-to-${\displaystyle y}$ path now displays the longest common prefix of ${\displaystyle A[i]}$ and ${\displaystyle A[i+1]}$.
3. Connect ${\displaystyle w}$ to the newly created internal node ${\displaystyle y}$ by an edge ${\displaystyle (y,w)}$ that is labeled ${\displaystyle S[A[i]+H[i+1],A[i]+d(w)-1]}$. The new label consists of the remaining characters of the deleted edge ${\displaystyle (v,w)}$ that were not used as the label of edge ${\displaystyle (v,y)}$.
4. Add ${\displaystyle A[i+1]}$ as a new leaf ${\displaystyle x}$ and connect it to the new internal node ${\displaystyle y}$ by an edge ${\displaystyle (y,x)}$ that is labeled ${\displaystyle S[A[i+1]+H[i+1],n]}$. Thus the edge label consists of the remaining characters of suffix ${\displaystyle A[i+1]}$ that are not already represented by the concatenation of the labels of the root-to-${\displaystyle v}$ path.
5. This creates the partial suffix tree ${\displaystyle ST_{i+1}}$.

A simple amortization argument shows that the running time of this algorithm is bounded by ${\displaystyle O(n)}$:

The nodes that are traversed in step ${\displaystyle i}$ by walking up the rightmost path of ${\displaystyle ST_{i}}$ (apart from the last node ${\displaystyle v}$) are removed from the rightmost path, when ${\displaystyle A[i+1]}$ is added to the tree as a new leaf. These nodes will never be traversed again for all subsequent steps ${\displaystyle j>i}$. Therefore, at most ${\displaystyle 2n}$ nodes will be traversed in total.

LCP queries for arbitrary suffixes

The LCP array ${\displaystyle H}$ only contains the length of the longest common prefix of every pair of consecutive suffixes in the suffix array ${\displaystyle A}$. However, with the help of the inverse suffix array ${\displaystyle A^{-1}}$ (${\displaystyle A[i]=j\Leftrightarrow A^{-1}[j]=i}$, i.e. the suffix ${\displaystyle S[j,n]}$ that starts at position ${\displaystyle j}$ in ${\displaystyle S}$ is stored in position ${\displaystyle A^{-1}[j]}$ in ${\displaystyle A}$) and constant-time range minimum queries on ${\displaystyle H}$, it is possible to determine the length of the longest common prefix of arbitrary suffixes in ${\displaystyle O(1)}$ time.

Because of the lexicographic order of the suffix array, every common prefix of the suffixes ${\displaystyle S[i,n]}$ and ${\displaystyle S[j,n]}$ has to be a common prefix of all suffixes between ${\displaystyle i}$'s position in the suffix array ${\displaystyle A^{-1}[i]}$ and ${\displaystyle j}$'s position in the suffix array ${\displaystyle A^{-1}[j]}$. Therefore, the length of the longest prefix that is shared by all of these suffixes is the minimum value in the interval ${\displaystyle H[A^{-1}[i]+1,A^{-1}[j]]}$. This value can be found in constant time if ${\displaystyle H}$ is preprocessed for range minimum queries.

Thus given a string ${\displaystyle S}$ of length ${\displaystyle n}$ and two arbitrary positions ${\displaystyle i,j}$ in the string ${\displaystyle S}$ with ${\displaystyle A^{-1}[i]<A^{-1}[j]}$, the length of the longest common prefix of the suffixes ${\displaystyle S[i,n]}$ and ${\displaystyle S[j,n]}$ can be computed as follows: ${\displaystyle \operatorname {LCP} (i,j)=H[\operatorname {RMQ} _{H}(A^{-1}[i]+1,A^{-1}[j])]}$.