Weather Forecasting Group

• E-mail : E.Ebert@bom.gov.au
• Fax : +61-3-9669-4660 
• Phone:  +61-3-9669-4688
  

Postal Address:

Bureau of Meteorology Research Centre


GPO Box 1289,

Melbourne, Victoria 3001

Australia

• Education
• Interests and Research
• Recent talks
• Publications

Education: 

• B.S. Atmospheric Sciences, University of California, 1981
• M.S. Meteorology, University of Wisconsin, Madison, 1984
• Ph.D. Meteorology, University of Wisconsin, Madison, 1987

Interests and Research:

Validation of satellite precipitation estimates:

Rainfall products from operational satellite precipitation algorithms are easily obtainable via the web or FTP, and are being used for many diverse meteorological, climate, hydrological, agricultural, and other applications. It is therefore important to have an idea of their accuracy and expected error characteristics. 

The Australian gridded rain gauge analysis is being used to intercompare and validate several operational and semi-operational satellite precipitation algorithms on daily and longer time scales. These include GPCP products, as well as several algorithms from NASA GSFC, NOAA NESDIS, NOAA CPC, Naval Research Laboratory, and the University of Arizona. More products will be added as they become available. The validation results are updated on a daily basis, and results are displayed on the SatRainVal web site. 

 Verification of quantitative precipitation forecasts:

In order to improve the accuracy of quantitative precipitation forecasts (QPFs) from the Bureau's numerical weather prediction (NWP) models, it is first necessary to understand the strengths and weaknesses of the models. The 24- and 48-hour QPFs from the Bureau's regional and global NWP models, as well as NWP models from several operational centres overseas, are verified on a daily basis against the operational rainfall analyses. Results show that the models have good skill in the southern part of Australia, but less skill in the tropical north. More details may be found in Ebert and McBride (1997), McBride and Ebert (2000), and Ebert et al. (2003). 

As part of the QPF verification process, I have developed an interactive verification tool called RAINVAL. It produces maps, time series, and statistics to show the skill of the QPFs when compared to the rainfall analysis and to each other. It is currently used to monitor the daily rainfall forecasts over Australia from a large number of NWP models. It is also being used in the Bureau to assess new versions of the regional and global models, as well as rainfall output from overseas models. 

One approach to verifying QPFs is to focus on individual weather systems (contiguous rain areas, or CRAs), as opposed to point-by-point domain rainfall verifications. The QPF error for an individual rain system has components due to incorrect location, incorrect magnitude, and incorrect pattern (shape). By using pattern translation and minimisation of the squared difference between forecast and observed rainfall, the displacement of the forecast can be determined. Overlaying the two patterns, the remaining error can partitioned into contributions from magnitude and shape errors. This verification strategy can reveal systematic errors in NWP QPFs, which in turn can be related to weaknesses in model dynamics, rainfall parameterisations, surface topography, etc. The forecast for the rain event inself can be classified as a "hit", "miss", etc., using this technique. Details may be Ebert and McBride (2000). The IDL code can be downloaded by clicking here.

"Fuzzy" verification approaches reward closeness by relaxing the requirement for exact matches between forecasts and observations. The key to the fuzzy approach is the use of a spatial window or neighborhood surrounding the forecast and/or observed points. The treatment of the points within the window may include averaging (upscaling), thresholding, or generation of a PDF, allowing a variety of continuous, categorical, and probabilistic verification metrics to be employed. The size of this neighborhood can be varied to provide verification results at multiple scales, thus allowing the user to determine at which scales the forecast has useful skill. Other windows could be included to represent closeness in time, closeness in intensity, and/or closeness in some other important aspect. I have developed a framework for fuzzy verification that incorporates several fuzzy verification methods appearing in the literature over recent years. A recently published paper describes the fuzzy verification framework, and the IDL code can be downloaded by clicking here.

The WWRP Sydney 2000 Forecast Demonstration Project was held over a 3-month period spanning the Olympic games, to demonstrate the skill and utility of advanced nowcasting systems. I was part of the verification "team" that assessed the performance of the nowcasting systems. We found that the systems indeed showed significant skill, but by and large did not perform any better than simple extrapolation (Ebert et al., 2004).  Another Forecast Demonstration Project is planned for the Beijing 2008 Olympics. I am developing a real-time nowcast verification system to be used by forecasters and scientists during this FDP, and which will be general enough to be used in the Bureau and hopefully other centers. In additional to the "usual" visual and statistical verification, some of the advanced verification techniques that have been developed in recent years will be included (see 2005 talk on Nowcast Verification).


I am a member of the WWRP/WGNE Joint Verification Working Group established in 2003. One of our activities is to maintain a Forecast Verification Web Page that describes both the standard and the newer diagnostic verification methods. It addresses various issues in verification such as grid box vs. point verification, confidence intervals on the verification results, pooling vs. stratifying results, etc., and has a section for FAQ. It also has a few downloadable verification datasets that can be used to test verification methods. 


 Poor man's ensemble:

We run a poor man's ensemble (PME) forecast for rainfall, in which QPFs from several operational NWP models are combined to give deterministic and probabilistic rainfall forecasts. This approach is cheap and efficient, and gives deterministic forecasts that are more accurate, on average, than any one of the component models. In particular, the location of the rain system is much much improved using the poor man's ensemble. It also gives useful probabilistic forecasts out to several days. In fact, the poor man's ensemble had greater probabilistic skill than the 51-member ECMWF Ensemble Prediction System out to 48 h. The details of this study are given in Ebert (2001, 2002). Forecasts from the PME can be viewed by clicking here.

Recent talks:

Fuzzy forecast verification - Giving credit to close forecasts. NCAR, 26 February 2007.

Verification of nowcasts and very short range forecasts. WWRP Symposium on Nowcasting and Very Short Range Forecasting, Toulouse, France, 5-9 September 2005.

Verification of ensembles. TIGGE 1^st Workshop, ECMWF, Reading, UK, 1-3 March 2005.

Monitoring the quality of operational and semi-operational satellite precipitation estimates – the IPWG validation/intercomparison study. 2^nd International Precipitation Working Group (IPWG) Workshop, Monterey, CA, 25-28 October 2004.

Verification of nowcasts. WWRP/WMO Latin American Nowcasting Training Workshop, Brasilia, 3-14 November 2003.

Probabilistic precipitation forecasts from deterministic forecast models. 10th Natl Australian Meteorological and Oceanographic Society Conference, Perth, 10-12 February 2003.

Verifying satellite precipitation estimates for weather and hydrological applications. 1st Intl. Precipitation Working Group (IPWG) Workshop, Madrid, Spain, 23-27 September 2002.

Publications:

• Refereed papers, 1996-1999
• Conference papers, 1996-present
• Recent refereed publications:

• Ebert, E.E., 2008: Fuzzy verification of high resolution gridded forecasts: A review and proposed framework. Meteorol. Appls., 15, 51-64.
  

• Ebert, E.E., J. Janowiak and C. Kidd, 2007: Comparison of near real time precipitation estimates from satellite observations and numerical models . Bull. Amer. Met. Soc., 88, 47-64.
  

• Ebert, E.E., S. Kusselson and M. Turk, 2005: Validation of NESDIS operational tropical rainfall potential (TRaP) forecasts for Australian tropical cyclones. Aust. Meteorol. Mag., 54, 121-135. Click here to get a PDF version.
  

• Grams, J.S., W.A. Gallus, L.S. Wharton, S. Koch, A. Loughe, and E. E. Ebert, 2005: The use of a modified Ebert-McBride technique to evaluate mesoscale model QPF as a function of convective system morphology during IHOP 2002. Wea. Forecasting, 21, 288-306.

• Ebert, E., L.J. Wilson, B.G. Brown, P. Nurmi, H.E. Brooks, J. Bally, and M. Jaeneke, 2004: Verification of nowcasts from the WWRP Sydney 2000 Forecast Demonstration Project. Wea. Forecasting, 19, 73-96.

• May, P.T., T. Keenan, R. Potts, R. Webb, A. Treloar, E. Spark, S. Lawrence, E. Ebert, J. Bally, J. Wilson, and P. Joe, 2004: The Sydney observing network during the Sydney 2000 Olympic Games Forecast Demonstration Project: Addressing forecast issues and nowcasting requirements. Wea. Forecasting, 19, 115-130.

• Pierce, C.E., E.E. Ebert, A. Seed, N. Fox, M. Sleigh, C.G. Collier, N. Donaldson, J. Wilson, R. Roberts and C. Mueller, 2004: The nowcasting of precipitation during Sydney 2000: An appraisal of the QPF algorithms. Wea. Forecasting, 19, 7-21.

• Wilson, J.W., E. Ebert, T. Saxen, C. Pierce, M. Sleigh, A. Seed, R. Roberts and C. Mueller, 2004: Sydney 2000 Forecast Demonstration Project: Convective storm nowcasting. Wea. Forecasting, 19, 131-150.

• Ebert, E.E., U. Damrath, W. Wergen and M.E. Baldwin, 2003: The WGNE assessment of short-term quantitative precipitation forecasts (QPFs) from operational numerical weather prediction models. Bull. Amer. Met. Soc., 84, 481-492.

• Keenan, T., P. Joe, J. Wilson, C. Collier, B. Golding, D. Burgess, P. May, C. Pierce, J. Bally, A. Crook, D. Sills, L. Berry, I. Bell, N. Fox, R. Pielke Jr., E. Ebert, M. Eilts, K. O'Loughlin, R. Webb, R. Carbone, K. Browning, R. Roberts, C. Mueller, 2002: The Sydney 2000 World Weather Research Programme Forecast Demonstration Project: Overview and current status. Bull. Amer. Met. Soc., 84, 1041-1054.

• Ebert, E.E., 2001: Ability of a poor man's ensemble to predict the probability and distribution of precipitation. Mon. Wea. Rev., 129, 2461-2480. See also Ebert, E.E., 2002: CORRIGENDUMMon. Wea. Rev., 130, 1661-1663.

• Ebert, E.E. and J.L. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrology, 239, 179-202.

• McBride, J.L. and E.E. Ebert, 2000: Verification of quantitative precipitation forecasts from operational numerical weather prediction models over Australia.  Wea. Forecasting, 15, 103-121.

• Ebert, E.E. and Weymouth, G.T., 1999, Incorporating satellite observations of "no rain" in an Australian daily rainfall analysis, J. Appl. Meteor., 38, 1, 44-56.

• Weymouth, G., Mills, G., Jones, D., Ebert, E. and Manton, M., 1999, A continental-scale daily rainfall analysis scheme, Aust. Met. Mag., 48, 169-179.