Intersection and union of two sets A and B

Intersection over Union as a similarity measure for object detection on images - an important task in computer vision.

The Jaccard index, also known as the Jaccard similarity coefficient, is a statistic used for gauging the similarity and diversity of sample sets.

It was developed by Grove Karl Gilbert in 1884 as his ratio of verification (v)^[1] and now is frequently referred to as the Critical Success Index in meteorology.^[2] It was later developed independently by Paul Jaccard, originally giving the French name coefficient de communauté,^[3] and independently formulated again by T. Tanimoto.^[4] Thus, the Tanimoto index or Tanimoto coefficient are also used in some fields. However, they are identical in generally taking the ratio of Intersection over Union.

Overview

The Jaccard coefficient measures similarity between finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets:

${\displaystyle J(A,B)={{|A\cap B|} \over {|A\cup B|}}={{|A\cap B|} \over {|A|+|B|-|A\cap B|}}.}$

Note that by design, ${\displaystyle 0\leq J(A,B)\leq 1.}$ If A intersection B is empty, then J(A,B) = 0. The Jaccard coefficient is widely used in computer science, ecology, genomics, and other sciences, where binary or binarized data are used. Both the exact solution and approximation methods are available for hypothesis testing with the Jaccard coefficient.^[5]

Jaccard similarity also applies to bags, i.e., Multisets. This has a similar formula,^[6] but the symbols mean bag intersection and bag sum (not union). The maximum value is 1/2.

${\displaystyle J(A,B)={{|A\cap B|} \over {|A\uplus B|}}={{|A\cap B|} \over {|A|+|B|}}.}$

The Jaccard distance, which measures dissimilarity between sample sets, is complementary to the Jaccard coefficient and is obtained by subtracting the Jaccard coefficient from 1, or, equivalently, by dividing the difference of the sizes of the union and the intersection of two sets by the size of the union:

${\displaystyle d_{J}(A,B)=1-J(A,B)={{|A\cup B|-|A\cap B|} \over |A\cup B|}.}$

An alternative interpretation of the Jaccard distance is as the ratio of the size of the symmetric difference ${\displaystyle A\triangle B=(A\cup B)-(A\cap B)}$ to the union. Jaccard distance is commonly used to calculate an n × n matrix for clustering and multidimensional scaling of n sample sets.

This distance is a metric on the collection of all finite sets.^[7][8][9]

There is also a version of the Jaccard distance for measures, including probability measures. If ${\displaystyle \mu }$ is a measure on a measurable space ${\displaystyle X}$, then we define the Jaccard coefficient by

${\displaystyle J_{\mu }(A,B)={{\mu (A\cap B)} \over {\mu (A\cup B)}},}$

and the Jaccard distance by

${\displaystyle d_{\mu }(A,B)=1-J_{\mu }(A,B)={{\mu (A\triangle B)} \over {\mu (A\cup B)}}.}$

Care must be taken if ${\displaystyle \mu (A\cup B)=0}$ or ${\displaystyle \infty }$, since these formulas are not well defined in these cases.

The MinHash min-wise independent permutations locality sensitive hashing scheme may be used to efficiently compute an accurate estimate of the Jaccard similarity coefficient of pairs of sets, where each set is represented by a constant-sized signature derived from the minimum values of a hash function.

Similarity of asymmetric binary attributes

Given two objects, A and B, each with n binary attributes, the Jaccard coefficient is a useful measure of the overlap that A and B share with their attributes. Each attribute of A and B can either be 0 or 1. The total number of each combination of attributes for both A and B are specified as follows:

${\displaystyle M_{11}}$ represents the total number of attributes where A and B both have a value of 1.

${\displaystyle M_{01}}$ represents the total number of attributes where the attribute of A is 0 and the attribute of B is 1.

${\displaystyle M_{10}}$ represents the total number of attributes where the attribute of A is 1 and the attribute of B is 0.

${\displaystyle M_{00}}$ represents the total number of attributes where A and B both have a value of 0.

A B	0	1
0	${\displaystyle M_{00}}$	${\displaystyle M_{10}}$
1	${\displaystyle M_{01}}$	${\displaystyle M_{11}}$

Each attribute must fall into one of these four categories, meaning that

${\displaystyle M_{11}+M_{01}+M_{10}+M_{00}=n.}$

The Jaccard similarity coefficient, J, is given as

${\displaystyle J={M_{11} \over M_{01}+M_{10}+M_{11}}.}$

The Jaccard distance, d_J, is given as

${\displaystyle d_{J}={M_{01}+M_{10} \over M_{01}+M_{10}+M_{11}}=1-J.}$

Statistical inference can be made based on the Jaccard similarity coefficients, and consequently related metrics.^[5] Given two sample sets A and B with n attributes, a statistical test can be conducted to see if an overlap is statistically significant. The exact solution is available, although computation can be costly as n increases.^[5] Estimation methods are available either by approximating a multinomial distribution or by bootstrapping.^[5]

Weighted Jaccard similarity and distance

If ${\displaystyle \mathbf {x} =(x_{1},x_{2},\ldots ,x_{n})}$ and ${\displaystyle \mathbf {y} =(y_{1},y_{2},\ldots ,y_{n})}$ are two vectors with all real ${\displaystyle x_{i},y_{i}\geq 0}$, then their Jaccard similarity coefficient (also known then as Ruzicka similarity) is defined as

${\displaystyle J_{\mathcal {W}}(\mathbf {x} ,\mathbf {y} )={\frac {\sum _{i}\min(x_{i},y_{i})}{\sum _{i}\max(x_{i},y_{i})}},}$

and Jaccard distance (also known then as Soergel distance)

${\displaystyle d_{J{\mathcal {W}}}(\mathbf {x} ,\mathbf {y} )=1-J_{\mathcal {W}}(\mathbf {x} ,\mathbf {y} ).}$

With even more generality, if ${\displaystyle f}$ and ${\displaystyle g}$ are two non-negative measurable functions on a measurable space ${\displaystyle X}$ with measure ${\displaystyle \mu }$, then we can define

${\displaystyle J_{\mathcal {W}}(f,g)={\frac {\int \min(f,g)d\mu }{\int \max(f,g)d\mu }},}$

where ${\displaystyle \max }$ and ${\displaystyle \min }$ are pointwise operators. Then Jaccard distance is

${\displaystyle d_{J{\mathcal {W}}}(f,g)=1-J_{\mathcal {W}}(f,g).}$

Then, for example, for two measurable sets ${\displaystyle A,B\subseteq X}$, we have ${\displaystyle J_{\mu }(A,B)=J(\chi _{A},\chi _{B}),}$ where ${\displaystyle \chi _{A}}$ and ${\displaystyle \chi _{B}}$ are the characteristic functions of the corresponding set.

Probability Jaccard similarity and distance

The weighted Jaccard similarity described above generalizes the Jaccard Index to positive vectors, where a set corresponds to a binary vector given by the indicator function, i.e. ${\displaystyle x_{i}\in \{0,1\}}$. However, it does not generalize the Jaccard Index to probability distributions, where a set corresponds to a uniform probability distribution, i.e.

${\displaystyle x_{i}={\begin{cases}{\frac {1}{|X|}}&i\in X\\0&{\text{otherwise}}\end{cases}}}$

It is always less if the sets differ in size. If ${\displaystyle |X|>|Y|}$, and ${\displaystyle x_{i}=\mathbf {1} _{X}(i)/|X|,y_{i}=\mathbf {1} _{Y}(i)/|Y|}$ then

${\displaystyle J_{\mathcal {W}}(x,y)={\frac {|X\cap Y|}{|X\setminus Y|+|X|}}<J(X,Y).}$

The probability Jaccard index can be interpreted as intersections of simplices.

Instead, a generalization that is continuous between probability distributions and their corresponding support sets is

${\displaystyle J_{\mathcal {P}}(x,y)=\sum _{x_{i}\neq 0,y_{i}\neq 0}{\frac {1}{\sum _{j}\max \left({\frac {x_{j}}{x_{i}}},{\frac {y_{j}}{y_{i}}}\right)}}}$

which is called the "Probability" Jaccard.^[10] It has the following bounds against the Weighted Jaccard on probability vectors.

${\displaystyle J_{\mathcal {W}}(x,y)\leq J_{\mathcal {P}}(x,y)\leq {\frac {2J_{\mathcal {W}}(x,y)}{1+J_{\mathcal {W}}(x,y)}}}$

Here the upper bound is the (weighted) Sørensen–Dice coefficient. The corresponding distance, ${\displaystyle 1-J_{\mathcal {P}}(x,y)}$, is a metric over probability distributions, and a pseudo-metric over non-negative vectors.

The Probability Jaccard Index has a geometric interpretation as the area of an intersection of simplices. Every point on a unit ${\displaystyle k}$-simplex corresponds to a probability distribution on ${\displaystyle k+1}$ elements, because the unit ${\displaystyle k}$-simplex is the set of points in ${\displaystyle k+1}$ dimensions that sum to 1. To derive the Probability Jaccard Index geometrically, represent a probability distribution as the unit simplex divided into sub simplices according to the mass of each item. If you overlay two distributions represented in this way on top of each other, and intersect the simplices corresponding to each item, the area that remains is equal to the Probability Jaccard Index of the distributions.

Optimality of the Probability Jaccard Index

A visual proof of the optimality of the Probability Jaccard Index on three element distributions.

Consider the problem of constructing random variables such that they collide with each other as much as possible. That is, if ${\displaystyle X\sim x}$ and ${\displaystyle Y\sim y}$, we would like to construct ${\displaystyle X}$ and ${\displaystyle Y}$ to maximize ${\displaystyle \Pr[X=Y]}$. If we look at just two distributions ${\displaystyle x,y}$ in isolation, the highest ${\displaystyle \Pr[X=Y]}$ we can achieve is given by ${\displaystyle 1-{\text{TV}}(x,y)}$ where ${\displaystyle {\text{TV}}}$ is the Total Variation distance. However, suppose we weren't just concerned with maximizing that particular pair, suppose we would like to maximize the collision probability of any arbitrary pair. One could construct an infinite number of random variables one for each distribution ${\displaystyle x}$, and seek to maximize ${\displaystyle \Pr[X=Y]}$ for all pairs ${\displaystyle x,y}$. In a fairly strong sense described below, the Probability Jaccard Index is an optimal way to align these random variables.

For any sampling method ${\displaystyle G}$ and discrete distributions ${\displaystyle x,y}$, if ${\displaystyle \Pr[G(x)=G(y)]>J_{\mathcal {P}}(x,y)}$ then for some ${\displaystyle z}$ where ${\displaystyle J_{\mathcal {P}}(x,z)>J_{\mathcal {P}}(x,y)}$ and ${\displaystyle J_{\mathcal {P}}(y,z)>J_{\mathcal {P}}(x,y)}$, either ${\displaystyle \Pr[G(x)=G(z)]<J_{\mathcal {P}}(x,z)}$ or ${\displaystyle \Pr[G(y)=G(z)]<J_{\mathcal {P}}(y,z)}$.^[10]

That is, no sampling method can achieve more collisions than ${\displaystyle J_{\mathcal {P}}}$ on one pair without achieving fewer collisions than ${\displaystyle J_{\mathcal {P}}}$ on another pair, where the reduced pair is more similar under ${\displaystyle J_{\mathcal {P}}}$ than the increased pair. This theorem is true for the Jaccard Index of sets (if interpreted as uniform distributions) and the probability Jaccard, but not of the weighted Jaccard. (The theorem uses the word "sampling method" to describe a joint distribution over all distributions on a space, because it derives from the use of weighted minhashing algorithms that achieve this as their collision probability.)

This theorem has a visual proof on three element distributions using the simplex representation.

Tanimoto similarity and distance

Various forms of functions described as Tanimoto similarity and Tanimoto distance occur in the literature and on the Internet. Most of these are synonyms for Jaccard similarity and Jaccard distance, but some are mathematically different. Many sources^[11] cite an IBM Technical Report^[4] as the seminal reference. The report is available from several libraries.

In "A Computer Program for Classifying Plants", published in October 1960,^[12] a method of classification based on a similarity ratio, and a derived distance function, is given. It seems that this is the most authoritative source for the meaning of the terms "Tanimoto similarity" and "Tanimoto Distance". The similarity ratio is equivalent to Jaccard similarity, but the distance function is not the same as Jaccard distance.

Jaccard index in binary classification confusion matrices

In confusion matrices employed for binary classification, the Jaccard index can be framed in the following formula:

${\displaystyle {\text{Jaccard index}}={\frac {TP}{TP+FP+FN}}}$

where TP are the true positives, FP the false positives and FN the false negatives.^[13]

See also

• Overlap coefficient
• Simple matching coefficient
• Hamming distance
• Sørensen–Dice coefficient, which is equivalent: ${\displaystyle J=S/(2-S)}$ and ${\displaystyle S=2J/(1+J)}$ (${\displaystyle J}$: Jaccard index, ${\displaystyle S}$: Sørensen–Dice coefficient)
• Tversky index
• Correlation
• Mutual information, a normalized metricated variant of which is an entropic Jaccard distance.

References

Further reading

External links

• Introduction to Data Mining lecture notes from Tan, Steinbach, Kumar
• SimMetrics a sourceforge implementation of Jaccard index and many other similarity metrics
• A web-based calculator for finding the Jaccard Coefficient
• Intersection over Union (IoU) for object detection
• Kaggle Dstl Satellite Imagery Feature Detection - Evaluation
• Similarity and dissimilarity measures used in data science