Home | About Us | Contacts | Help | Feedback | SEARCH Global | Australia | NSW | Vic. | Qld | WA | SA | Tas. | ACT | NT | Ant. | Weather & Warnings | Hydrology | Climate | Numerical Prediction | About Services | Learn About Meteorology | Registered User Services |

Benchmarking and evaluation of the ACOM 1 and ACOM 2 ocean models

1. Introduction

   1. Development of ACOM x

   2. Initial use of ACOM in CGCM

   3. The LWRRDC Project

2. The Ocean Model: description

   1. ACOM 1

      1. Power et al

      2. Wang et al (1999,2001)

   2. ACOM 2

      1. Differences and enhancements

      
      

      Table 1. Characteristics of the ACOM 2 model (taken from Schiller et al 2000)

      Characteristic	ACOM 1	ACOM 2
      Horizontal resolution	Longitudes: 2° uniform, Latitudes: 0.5° within 7° of the equator then increasing gradually to 5.85° near the poles	Longitudes: 2° uniform, Latitudes: 0.5° within 9° of the equator then increasing gradually to 1.5° near the poles
      Vertical resolution	25 levels, with 12 levels in the top 185 m; 5000 m	25 levels, with 12 levels in the top 185 m; 5000 m
      Diffusion, viscosity
      Wind stress	FSU winds (Legler et al. 1989, Stricherz et al. 1992) with Cd = 1.5 x 10-3 and blended with Hellerman and Rosenstein (1983) wind stresses poleward of 30°N–30°S.	FSU winds (Legler et al. 1989, Stricherz et al. 1992) with Cd = 1.5 x 10 -3 and blended with Hellerman and Rosenstein (1983) wind stresses poleward of 30°N–30°S.
      Heat flux	Parameterized, using ISCCP net solar shortwave radiation plus flux correction [see Schiller et al. (1998)]	Parameterized, using ISCCP net solar shortwave radiation plus flux correction [see Schiller et al. (1998)]
      Freshwater flux	Newtonian restoring to Levitus et al. 1994 data	Newtonian restoring to Levitus et al. 1994 data
      Special features	Hybrid mixed layer model (Chen et al. 1994b; Power et al. 1995); increased vertical diffusion and viscosity in Indonesian archipelago to simulate tidal mixing.	Hybrid mixed layer model (Chen et al. 1994b; Power et al. 1995); increased vertical diffusion and viscosity in Indonesian archipelago to simulate tidal mixing. No Mediterranean Sea.

      
      

      2. Schiller et al (1998) for ITF and Indian Ocean dynamics [Schiller, A., J. S. Godfrey, P. C. McIntosh, G. Meyers, S. E. Wijffels, 1998: Seasonal Near-Surface Dynamics and Thermodynamics of the Indian Ocean and Indonesian Throughflow in a Global Ocean General Circulation Model. Journal of Physical Oceanography, 28, 2288–2312]
         Schiller et al (2000) on how sea surface temperature anomalies in the tropical and sub-tropical Indian and Pacific Ocean are maintained and changed on interannual timescales. [Schiller, A., J. S. Godfrey, P. C. McIntosh, G. Meyers, R. Fiedler, 2000: Interannual Dynamics and Thermodynamics of the Indo–Pacific Oceans. Journal of Physical Oceanography: Vol. 30, No. 5, pp. 987–1012.]

      

      (Fig. 5 from Schiller et al 2000). Hovmoeller plots of (a) modeled and (b) observed (Reynolds and Smith 1994 ) SST anomalies along the equator for 1985 to 1990. C.I.: 1°C. The dashed lines at 66°E,
      140°W, and 110°W indicate locations of XBT/buoy data chosen for comparison with model data. 

      

      (Fig. 20. Model to data comparison along XBT line IX1. (a, b) Variation in depth of the 20°C isotherm. The long-term mean depth in each observed (modeled) bin was subtracted from observed (modeled) values. (c, d) Anomaly in depth of the 20°C isotherm. The observed (modeled) mean annual cycle from 1985 through 1990 was subtracted from the observed (modeled) values. (e, f) Anomaly of SST. The mean observed (modeled) annual cycle for the period 1985 through 1990 was subtracted from the observed (modeled) values. The C.I. is 10 m (a–d) and 0.5°C (e and f).

      
      

      3. ...

3. Benchmarking

   1. ACOM 1

      1. Power et al results

         • No interannual

      2. Climatology 1980-1999

         • Spin-up strategy (from Power et al)

         • Other differences in set-up

      3. Diagnostics from coupled implementation

       

      Figure 3a. Differences between the climatology of the BMRC implementations of ACOM 1 and ACOM 2 and Levitus

      	Meridional Section at 140W	Zonal Section along equator
      ACOM 1
      ACOM 2

       

   2. ACOM 2

      1. Schiller et al tests, papers

      2. Climatology

         • Comparison with IX1

         • Other

4. Evaluation and intercomparison

   1. Seasonal cycle away from spin-up?

   2. Sub-sampled along IX1 for control assimilation

   3. Forecast tendencies in assimilation (Analysis - Forecast)

   Results from the MOM1 assimilation run

   These are results from the BMRC MOM1 (ACOM I) data assimilation run from 1980 through to 1998 (Wang et al 2001). The analysis technique is described in Smith (1991).

   For MOM1, a "uarch" file was created at each analysis step (10 days apart) that contained full configuration details, the model state before analysis (just T), the estimated error of that first guess, the analysis, and the estimated analysis error. Also, all original observations and formed super-obs are written out with information on the fate of those observations in the analysis. For the 693 analyses performed from Jan 1980 through to end of 1998, we have calculated the mean difference between the analysis minus the forecast (Fig. 4c.1). 

   Figure 4c.1.  Analysis minus forecast mean differences for (i) along the equator, (ii) at 160E, (iii) 140W, (iv) at the surface, (v) 50m, (vi) 200m, and (vii) averaged over  the top 300m (all thumbnails).

   (i)	(ii)	(iii)
   (iv)	(v)
   (vi)	(vii)

   On the equator, the forecast has a significant cool bias below 100 m stretching into the central Pacific (the analysis is warm relative to the forecast) while, at the depth of the thermocline, there is a warm bias in the forecast from the central Pacific to the east (analysis cooler than forecast, or model thermocline consistently too deep). The cross sections at 160E and 140W also show significant biases in the vicinity of the thermocline, particularly in the west. However, some of the differences in the west seem to be non-dynamic (eg, evidence of errors just south of the equator), perhaps associated with salinity errors.

   The geographic distribution of the differences is shown in (iv) through (vii). There are cool biases off the equator in the western Pacific and a warm bias on the equator (in the 300m depth-averaged difference). The location does suggest some connection to wind and dynamics.

   These and other statistics from the MOM1 assimilation can be found here

   4. Comparative statistics from assimilation
      (analysis by Oscar)

   Every 10 days the assimilation is performed by combining the observations with the model background field. The model model background field is a 10 day forecast from the previous analyses using FSU stresses. Just before the observations are inserted we calculate the departures of the background from all observations (ones that pass QC). Tis gives a measure of how well the model background agrees with the independent observations (independent since they have not yet been assimilated).

   The plots show the rms and mean of the errors taken over different horizontal regions. rms indicates a magnitude of the background error and the mean indicates the model background bias.

   Note: the background error depends on - model error, forcing error and the previous assimilation.

   Plots show the full field over 1992-1996, as well as the mean over this period. I have also broken down into seasonal cycle and interannual anomalies (only for period 1992-1995).

   Also shown is statistics regarding the QC -ie the percentage of obs that fail.

   Figure 4d.1 shows the data rejection statistics for the NINO3 and NINO4 regions and the Equatorial Pacific  Ocean as a whole. In all cases the ACOM 1 model is rejecting more data than ACOM 2 which suggests the former model has greater systematic error in the equatorial region.

   Figure 4d.1. Percentage of data rejected as a function of depth for (left) the equatorial Pacific (5S-5N); (middle) the NINO4 region 150E-150W, 5S-5N; and (right) the NINO3 region 150W-90W, 5S-5N.

   1. NINO3 bias and rms
   
   
   • The rms differences are quite similar, with ACOM 1 having slightly smaller errors near the surface, and slightly greater errors at the thermocline.
   • The rms differences  have interannual variability, related to variability of thermocline.
   • The bias of ACOM 2 is greatest at the surface but significantly better at the depth of the thermocline. We  have identified a difference between the two models in the shortwave forcing that contributes to the ACOM 2 surface error
   • The mean seasonal differences (left column of (c)) and anomalies (right column) suggest a significant portion of the ACOM 1 errors from the seasonal cycle.
   
   

   Figure 4d(i). Plots of assimilation statistics as a function of depth for the NINO3 region for (top) ACOM 1 and (bottom) ACOM 2.  The mean of the entire period is shown at the right.

   (a)

   (b)

   (c)

   
   

   2. NINO4 bias and rms

   
   
   • The rms differences between ACOM 1 and 2 are small
   • The bias of ACOM 2 is worse at the surface but is everywhere else better than ACOM 1.
   • The errors in the seasonal cycle appear less pronounced compared with NINO3
   
   

   Figure 4d(ii). Plots of assimilation statistics as a function of depth for the NINO4 region. 

   (a)

   (b)

   (c)

   
   
   3. Indian Ocean bias and rms
   • The region is defined by 30E-120E, 10S-10N, and so includes a significant number of western Pacific observations
   • The data coverage in the Indian is sparse at 10-day intervals
   • On rms statistics, ACOM 2 does out-perform ACOM 1
   • The bias statistics are more difficult to interpret.
   • As with all results, the role of wind errors is not clear
   
   

   Figure 4d(iii). Plots of assimilation statistics as a function of depth for the Indian Ocean equatorial region. 

   (a)

   (b)

   (c)

    

   A selection of other results

   Equatorial Pacific stats

    

   Obs stats from Indian Ocean re rejection

    

   Discussion of 1992 QC problems

   In several of the statistics above the period May-Dec 1992 appeared to be anomalous, generating higher than expected error (data rejection) rates, particularly in the "Indian Ocean".

   Below are some more detailed diagnostics from that period, from the routine D20 and 400m depth-averaged temperature analyses, and at 100m from the MOM2 assimilation run.

   Figure QC1. Quality control analysis for 1992 for the BMRC D20 and T400 analyses (columns 1 and 2) and for the ACOM 1 data assimilation run at z=100m. The top panels show the monthly anomalies (D20 and T400) and mean analysis increment (T(100m)) for each month of 1992. Superimposed are the observation locations. Rejected data are marked red. The Bottom panels show statistics for quality control; blue is the total number of original observations; red the number that remained unflagged; green is the number of effective observations after redundancy is removed; and black is the number of deleted observations. Note that T400 is for a month whereas the others are 10 day data windows. (click on thumbnails to get enlargement)

   D20 anal (10 day periods)	T400 anal (1 month period)	T(100m) assim.(10 day)

   The statistics confirm that the number of rejections did rise in the routine BMRC analyses as well as in the data assimilation run. Note that there are variations in the data sets used in each case because of reception of delayed mode data and due to the way data a gathered for each analysis (the 100m analysis will see less than D20, which in turn are both greater than T400).

   The short story is that the QC problem is not so much an Indian Ocean one as a TOGA COARE problem. It seems many data are rejected from the western Pacific region. We can only surmise what the problem might be; it does not appear to be the wind since the rejection pattern in the analyses is similar. More likely there was an encoding (or decoding) problem with R/T data.

   NS

   5. Intercomparison from Wang coupled model runs

   
   

   Figure 4e.1.  Annual mean longitude-depth sections at the equator (thumbnails) from (a) the BMRC sub-surface analysis, (b) 60 coupled model 6-month lead forecasts with ACOM 1, and (c) as (b) but with ACOM 2

Fig. 4e.1 shows the mean longitude-depth sections of subsurface temperature in the  equatorial region (2N-2S mean) from the BMRC analysis and CM1 and CM2 hindcasts. The data from the hindcasts are six month lead forecasts initialized at Feb, May, Aug and Nov of years 1981 through 1995 (sixty in total) for the coupled model means; and the corresponding 60 monthly mean is used for the analysis. From the figure the overall improvement of CM2 may include slightly better thermocline gradient over the central to western Pacific and shoaling of the thermocline depth towards western Pacific boundary.



Figure 4e.2. Standard deviation of the fields shown in Fig. 4e.1 

The analysis of variability (Fig. 4e.2) shows a maximum variability around the depth of the thermocline with typical amplitude 1.5 to 2°C. Lack of variability towards east coast region in the analysis is due to little observation there so that the analysis is actually gone back to the background or climatology. The two coupled model results show increasing variability right across to the east boundary which is consistently following the depth of the thermocline where larger variability is expected due to stronger temperature gradient as seen in Fig. 1. The variability over the central and western Pacific is stronger in CM2 than that in CM1, possibly due to sharper thermocline gradient in CM2 then that in CM1 as seen in the Fig. 4e.1.

Figure 4e.3. The autocorrelation  (top) and rms difference (bottom) of the 6-month forecasts with NINO3 SST. Persistence is shown in black, the CM1 model in green, and the CM2 model in red. 

It is seen (Fig 4e.3) that both coupled models can produce skilful forecasts compared with persistence for all lead times. CM2 appears to have higher skill compared with CM1. Using a correlation of 0.6 as the threshold for useful forecasts then the maximum lead-time for useful NINO3 anomaly forecasts would be 11 and 9 month for CM2 and CM1 respectively; that is, an increase of 1-2 months in useful lead time. A feature we find that maybe corresponds to high skill is the interannual variability of the variables concerned. For example for NINO3 SST anomaly the standard deviations (see Fig. 4b, with same color code as in Fig. 4a except the black now being the observed standard deviation) from both CM1 and CM2 are able to maintain at a high level that is close to observation for lead times up to 12 months.

Some results from Aihong's Report on BAM-ACOM 2



These figures show the forecast bias at 8-month lead (left), the skill (middle) and the spatial pattern of skill (ACC) at 6-month lead.




5. Discussion and conclusions