SURVEY OF COMMON VERIFICATION METHODS IN METEOROLOGY

Henry R. Stanski, Laurence J. Wilson and William R. Burrows

Atmospheric Environment Service
Forecast Research Division

Second Edition, July 1989
Research Report No. MSRB 89-5

WMO World Weather Watch Technical Report No.8, WMO/TD No. 358
 

ABSTRACT

Commonly used verification methods for weather elements and numerical weather prediction model forecasts are described in terms of a general verification model. The advantages and disadvantages of each verification method are identified and discussed. Furthermore, numerous examples using meteorological data are provided. New ideas in verification are included in a separate chapter.

Download Part I:  (PDF format, 0.4 MB)
  Abstract
  Foreward
  Preface to the Electronic Edition
  Acknowledgements
  Table of Contents
  1. A VERIFICATION FRAMEWORK
    1.1 A verification model
    1.2 Predictand types
    1.3 Survey of verification measures
    1.4 A NWP verification model
    1.5 Attributes of a forecast

Download Part II:  (PDF format, 2.4 MB)
  2. COMMON VERIFICATION METHODS
    2.1 Scatter plots
    2.2 Bias or mean (algebraic) error - measure of overall reliability
    2.3 Mean absolute error (MAE) - measures overall accuracy
    2.4 Root mean square error (RMSE) - measures overall accuracy
    2.5 Reduction of variance (RV)
    2.6 Contingency tables and associated scores - stratification of data
    2.7 Reliability diagrams - measures reliability and sharpness
    2.8 The Brier score (PS), Brier skill score (BSS) - summary score
    2.9 Ranked probability score (RPS) and skill score (RPSS) - summary score

Download Part III:  (PDF format, 2.6 MB)
  3. VERIFICATION METHODS FOR NWP MODEL OUTPUT
    3.1 Definitions
    3.2 Objective measures of NWP model forecast skill
    3.3 Subjective measures of NWP model skill
  4. NEW IDEAS IN VERIFICATION
    4.1 Signal detection theory
    4.2 Statistically forecasting the error in NWP models
    4.3 Interrelationships between objective and subjective guidance
  5. VERIFICATION, CANADIAN EXPERIENCE
    5.1 Probability interpretation conversion
    5.2 Verification suggestions
  6. REFERENCES
  7. DATA SOURCES