Example

Suppose that one is forecasting the probability ${\displaystyle P}$ that it will rain on a given day. Then the Brier score is calculated as follows:

• If the forecast is 100% (${\displaystyle P}$ = 1) and it rains, then the Brier Score is 0, the best score achievable.
• If the forecast is 100% and it does not rain, then the Brier Score is 1, the worst score achievable.
• If the forecast is 70% (${\displaystyle P}$ = 0.70) and it rains, then the Brier Score is (0.70−1)² = 0.09.

• In contrast, if the forecast is 70% (${\displaystyle P}$ = 0.70) and it does not rain, then the Brier Score is (0.70−0)² = 0.49.
• Similarly, if the forecast is 30% (${\displaystyle P}$ = 0.30) and it rains, then the Brier Score is (0.30−1)² = 0.49.
• If the forecast is 50% (${\displaystyle P}$ = 0.50), then the Brier score is (0.50−1)² = (0.50−0)² = 0.25, regardless of whether it rains.

Original definition by Brier

Although the above formulation is the most widely used, the original definition by Brier^[1] is applicable to multi-category forecasts as well as it remains a proper scoring rule, while the binary form (as used in the examples above) is only proper for binary events. For binary forecasts, the original formulation of Brier's "probability score" has twice the value of the score currently known as the Brier score.

${\displaystyle BS={\frac {1}{N}}\sum \limits _{t=1}^{N}\sum \limits _{i=1}^{R}(f_{ti}-o_{ti})^{2}\,\!}$

In which ${\displaystyle R}$ is the number of possible classes in which the event can fall, and ${\displaystyle N}$ the overall number of instances of all classes. ${\displaystyle f_{ti}}$ is the predicted probability for class ${\displaystyle i.o_{ti}}$ is ${\displaystyle 1}$ if it is ${\displaystyle i}$-th class in instance ${\displaystyle t}$; ${\displaystyle 0}$, otherwise. For the case Rain / No rain, ${\displaystyle R=2}$, while for the forecast Cold / Normal / Warm, ${\displaystyle R=3}$.

3-component decomposition

The Brier score can be decomposed into 3 additive components: Uncertainty, Reliability, and Resolution. (Murphy 1973)^[2]

${\displaystyle BS=REL-RES+UNC}$

Each of these components can be decomposed further according to the number of possible classes in which the event can fall. Abusing the equality sign:

${\displaystyle BS={\frac {1}{N}}\sum \limits _{k=1}^{K}{n_{k}(\mathbf {f_{k}} -\mathbf {\bar {o}} _{\mathbf {k} })}^{2}-{\frac {1}{N}}\sum \limits _{k=1}^{K}{n_{k}(\mathbf {{\bar {o}}_{k}} -{\bar {\mathbf {o} }})}^{2}+\mathbf {\bar {o}} \left({1-\mathbf {\bar {o}} }\right)}$

With ${\displaystyle \textstyle N}$ being the total number of forecasts issued, ${\displaystyle \textstyle K}$ the number of unique forecasts issued, ${\displaystyle \mathbf {\bar {o}} ={\sum _{t=1}^{N}}\mathbf {o_{t}} /N}$ the observed climatological base rate for the event to occur, ${\displaystyle n_{k}}$ the number of forecasts with the same probability category and ${\displaystyle \mathbf {\overline {o}} _{\mathbf {k} }}$ the observed frequency, given forecasts of probability ${\displaystyle \mathbf {f_{k}} }$. The bold notation in the above formula indicates vectors, which is another way of denoting the original definition of the score and decomposing it according to the number of possible classes in which the event can fall. For example, a 70% chance of rain and an occurrence of no rain are denoted as ${\displaystyle \mathbf {f} =(0.3,0.7)}$ and ${\displaystyle \mathbf {o} =(1,0)}$ respectively. Operations like the square and multiplication on these vectors are understood to be component wise. The Brier Score is then the sum of the resulting vector on the right hand side.

Two-component decomposition

An alternative (and related) decomposition generates two terms instead of three.

${\displaystyle BS=CAL+REF}$

${\displaystyle BS={\frac {1}{N}}\sum \limits _{k=1}^{K}{n_{k}(\mathbf {f_{k}} -\mathbf {\bar {o}} _{\mathbf {k} })}^{2}+{\frac {1}{N}}\sum \limits _{k=1}^{K}{n_{k}(\mathbf {{\bar {o}}_{k}} (1-\mathbf {{\bar {o}}_{k}} }))}$

The first term is known as calibration (and can be used as a measure of calibration, see statistical calibration), and is equal to reliability. The second term is known as refinement, and it is an aggregation of resolution and uncertainty, and is related to the area under the ROC Curve.

The Brier Score, and the CAL + REF decomposition, can be represented graphically through the so-called Brier Curves,^[3] where the expected loss is shown for each operating condition. This makes the Brier Score a measure of aggregated performance under a uniform distribution of class asymmetries.^[4]