Skip to content

Brier score practice for probabilistic supercell forecasts

Calculate the Brier score by averaging the squared difference between the forecast probability and the binary observation. Smaller scores indicate smaller squared probability errors.

Start with How do you read a reliability diagram for storm probabilities?.

Worked values
WindowForecast pObservation oSquared error
10.8010.04
20.4000.16

Worked example

Use probability p between zero and one, and outcome o = 1 if the event occurred or 0 otherwise. For probabilities 0.8 and 0.4 paired with outcomes 1 and 0, the squared errors are 0.04 and 0.16. BS = (0.04+0.16)/2 = 0.10. Keep the event definition and eligible verification windows fixed when comparing two systems.

Hint for the exercises

Write a binary observation next to each forecast probability and square each error separately.

Practice questions

  1. Find the squared error for the first window.
  2. Compute the score for both windows.
  3. Add a forecast of 0.90 for a third window that stays storm-free.
  4. Compare the original forecast to a constant 0.5 forecast on the same two windows.
  5. A spreadsheet squares 80−1 for the first window. What conversion is missing?
  6. Why does a confident miss affect the score so strongly?

Worked answers

Answer 1

(0.8−1)² = 0.04.

Answer 2

(0.04+0.16)/2 = 0.10.

Answer 3

The new term is 0.81. BS = (0.04+0.16+0.81)/3 ≈ 0.337.

Answer 4

The constant forecast scores (0.25+0.25)/2 = 0.25. The original scores 0.10.

Answer 5

Convert 80% to 0.8 before comparing with a binary outcome. The correct squared error is 0.04.

Answer 6

A forecast near zero paired with outcome 1 produces a large absolute error. Squaring gives large errors extra weight relative to small errors.

Sources