LSH with respect to a similarity measure

Alternatively^[8] it is possible to define an LSH family on a universe of items U endowed with a similarity function ${\displaystyle \phi \colon U\times U\to [0,1]}$. In this setting, a LSH scheme is a family of hash functions H coupled with a probability distribution D over H such that a function ${\displaystyle h\in H}$ chosen according to D satisfies ${\displaystyle Pr[h(a)=h(b)]=\phi (a,b)}$ for each ${\displaystyle a,b\in U}$.

Amplification

Given a ${\displaystyle (d_{1},d_{2},p_{1},p_{2})}$-sensitive family ${\displaystyle {\mathcal {F}}}$, we can construct new families ${\displaystyle {\mathcal {G}}}$ by either the AND-construction or OR-construction of ${\displaystyle {\mathcal {F}}}$.^[1]

To create an AND-construction, we define a new family ${\displaystyle {\mathcal {G}}}$ of hash functions g, where each function g is constructed from k random functions ${\displaystyle h_{1},\ldots ,h_{k}}$ from ${\displaystyle {\mathcal {F}}}$. We then say that for a hash function ${\displaystyle g\in {\mathcal {G}}}$, ${\displaystyle g(x)=g(y)}$ if and only if all ${\displaystyle h_{i}(x)=h_{i}(y)}$ for ${\displaystyle i=1,2,\ldots ,k}$. Since the members of ${\displaystyle {\mathcal {F}}}$ are independently chosen for any ${\displaystyle g\in {\mathcal {G}}}$, ${\displaystyle {\mathcal {G}}}$ is a ${\displaystyle (d_{1},d_{2},p_{1}^{k},p_{2}^{k})}$-sensitive family.

To create an OR-construction, we define a new family ${\displaystyle {\mathcal {G}}}$ of hash functions g, where each function g is constructed from k random functions ${\displaystyle h_{1},\ldots ,h_{k}}$ from ${\displaystyle {\mathcal {F}}}$. We then say that for a hash function ${\displaystyle g\in {\mathcal {G}}}$, ${\displaystyle g(x)=g(y)}$ if and only if ${\displaystyle h_{i}(x)=h_{i}(y)}$ for one or more values of i. Since the members of ${\displaystyle {\mathcal {F}}}$ are independently chosen for any ${\displaystyle g\in {\mathcal {G}}}$, ${\displaystyle {\mathcal {G}}}$ is a ${\displaystyle (d_{1},d_{2},1-(1-p_{1})^{k},1-(1-p_{2})^{k})}$-sensitive family.

Bit sampling for Hamming distance

One of the easiest ways to construct an LSH family is by bit sampling.^[7] This approach works for the Hamming distance over d-dimensional vectors ${\displaystyle \{0,1\}^{d}}$. Here, the family ${\displaystyle {\mathcal {F}}}$ of hash functions is simply the family of all the projections of points on one of the ${\displaystyle d}$ coordinates, i.e., ${\displaystyle {\mathcal {F}}=\{h\colon \{0,1\}^{d}\to \{0,1\}\mid h(x)=x_{i}{\text{ for some }}i\in \{1,\ldots ,d\}\}}$, where ${\displaystyle x_{i}}$ is the ${\displaystyle i}$th coordinate of ${\displaystyle x}$. A random function ${\displaystyle h}$ from ${\displaystyle {\mathcal {F}}}$ simply selects a random bit from the input point. This family has the following parameters: ${\displaystyle P_{1}=1-R/d}$, ${\displaystyle P_{2}=1-cR/d}$. That is, any two vectors ${\displaystyle x,y}$ with Hamming distance at most ${\displaystyle R}$ collide under a random ${\displaystyle h}$ with probability at least ${\displaystyle P_{1}}$. Any ${\displaystyle x,y}$ with Hamming distance at least ${\displaystyle cR}$ collide with probability at most ${\displaystyle P_{2}}$.

Min-wise independent permutations

Suppose U is composed of subsets of some ground set of enumerable items S and the similarity function of interest is the Jaccard index J. If π is a permutation on the indices of S, for ${\displaystyle A\subseteq S}$ let ${\displaystyle h(A)=\min _{a\in A}\{\pi (a)\}}$. Each possible choice of π defines a single hash function h mapping input sets to elements of S.

Define the function family H to be the set of all such functions and let D be the uniform distribution. Given two sets ${\displaystyle A,B\subseteq S}$ the event that ${\displaystyle h(A)=h(B)}$ corresponds exactly to the event that the minimizer of π over ${\displaystyle A\cup B}$ lies inside ${\displaystyle A\cap B}$. As h was chosen uniformly at random, ${\displaystyle Pr[h(A)=h(B)]=J(A,B)\,}$ and ${\displaystyle (H,D)\,}$ define an LSH scheme for the Jaccard index.

Because the symmetric group on n elements has size n!, choosing a truly random permutation from the full symmetric group is infeasible for even moderately sized n. Because of this fact, there has been significant work on finding a family of permutations that is "min-wise independent" — a permutation family for which each element of the domain has equal probability of being the minimum under a randomly chosen π. It has been established that a min-wise independent family of permutations is at least of size ${\displaystyle \operatorname {lcm} \{\,1,2,\ldots ,n\,\}\geq e^{n-o(n)}}$,^[18] and that this bound is tight.^[19]

Because min-wise independent families are too big for practical applications, two variant notions of min-wise independence are introduced: restricted min-wise independent permutations families, and approximate min-wise independent families. Restricted min-wise independence is the min-wise independence property restricted to certain sets of cardinality at most k.^[20] Approximate min-wise independence differs from the property by at most a fixed ε.^[21]

Open source methods

Random projection

${\displaystyle {\frac {\theta (u,v)}{\pi }}}$ is proportional to ${\displaystyle 1-\cos(\theta (u,v))}$ on the interval [0, ${\displaystyle \pi }$

]

The random projection method of LSH due to Moses Charikar^[8] called SimHash (also sometimes called arccos^[25]) uses an approximation of the cosine distance between vectors. The technique was used to approximate the NP-complete max-cut problem.^[8]

The basic idea of this technique is to choose a random hyperplane (defined by a normal unit vector r) at the outset and use the hyperplane to hash input vectors.

Given an input vector v and a hyperplane defined by r, we let ${\displaystyle h(v)=\operatorname {sgn}(v\cdot r)}$. That is, ${\displaystyle h(v)=\pm 1}$ depending on which side of the hyperplane v lies. This way, each possible choice of a random hyperplane r can be interpreted as a hash function ${\displaystyle h(v)}$.

For two vectors u,v with angle ${\displaystyle \theta (u,v)}$ between them, it can be shown that

${\displaystyle Pr[h(u)=h(v)]=1-{\frac {\theta (u,v)}{\pi }}}$

${\displaystyle {\frac {\theta (u,v)}{\pi }}}$ is proportional to ${\displaystyle 1-\cos(\theta (u,v))}$. In fact the ratio is always within a factor of .87856.^[8][26] Meaning the probability of the two vectors being on the same side of the random hyperplane is proportional to the cosine distance between them.

Stable distributions

The hash function ^[27] ${\displaystyle h_{\mathbf {a} ,b}({\boldsymbol {\upsilon }}):{\mathcal {R}}^{d}\to {\mathcal {N}}}$ maps a d-dimensional vector ${\displaystyle {\boldsymbol {\upsilon }}}$ onto the set of integers. Each hash function in the family is indexed by a choice of random ${\displaystyle \mathbf {a} }$ and ${\displaystyle b}$ where ${\displaystyle \mathbf {a} }$ is a d-dimensional vector with entries chosen independently from a stable distribution and ${\displaystyle b}$ is a real number chosen uniformly from the range [0,r]. For a fixed ${\displaystyle \mathbf {a} ,b}$ the hash function ${\displaystyle h_{\mathbf {a} ,b}}$ is given by ${\displaystyle h_{\mathbf {a} ,b}({\boldsymbol {\upsilon }})=\left\lfloor {\frac {\mathbf {a} \cdot {\boldsymbol {\upsilon }}+b}{r}}\right\rfloor }$.

Other construction methods for hash functions have been proposed to better fit the data. ^[28] In particular k-means hash functions are better in practice than projection-based hash functions, but without any theoretical guarantee.

Semantic hashing

Semantic hashing is a technique that attempts to map input items to addresses such that closer inputs have higher semantic similarity.^[29] The hashcodes are found via training of an artificial neural network or graphical model.

Improvements

When t is large, it is possible to reduce the hashing time from ${\displaystyle O(n^{\rho })}$. This was shown by^[30] and^[31] which gave

• query time: ${\displaystyle O(t\log ^{2}(1/P_{2})/P_{1}+n^{\rho }(d+1/P_{1}))}$;
• space: ${\displaystyle O(n^{1+\rho }/P_{1}+\log ^{2}(1/P_{2})/P_{1})}$;

It is also sometimes the case that the factor ${\displaystyle 1/P_{1}}$ can be very large. This happens for example with Jaccard similarity data, where even the most similar neighbor often has a quite low Jaccard similarity with the query. In^[32] it was shown how to reduce the query time to ${\displaystyle O(n^{\rho }/P_{1}^{1-\rho })}$ (not including hashing costs) and similarly the space usage.