 |
GMTB Workflow Documentation
|
|
|
GMTB Workflow Documentation
|
|
The Model Evaluation Tools (MET; DTC 2015) software package is used by the GMTB team to perform objective verification. A number of tools available in MET are employed to conduct grid-to-grid (Grid-stat) and grid-to-point (Point-stat) comparisons. ... While a subset of MET configuration file settings used specifically for GMTB purposes will be described below, please note that all MET configuration files have a variety of user-defined settings and a full description regarding each option can be found in the MET Users’ Guide (DTC 2016).
The MET point-stat tool matches gridded forecast output to point observations. For GMTB, MET point-stat is used verify a variety of surface and upper-air fields against either the NAM Data Assimilation System (NDAS) or Global Data Assimilation System (GDAS) prepbufr point observations. There are two specific tasks that handle point-based verification within the GFS_VX.xml:
The two point verification tasks will be detailed in the following sub-sections. In addition, all verification tasks require a set of common environment variables to be set in the master source file, vx_env.ent, including:
The task to conduct point verification for surface fields in GFS_VX.xml is called met_point_sfc, which is under the verification_06hr metatask. When met_point_sfc is submitted it calls task/met_point_sfc.ent, which, in addition to sourcing the vx_env.ent environment variables mentioned above, it sets:
In order for the surface point verification task to be run by the workflow, the following dependencies are set in met_point_sfc.ent:
When all dependencies above are satisfied, the met_point_sfc.ent task calls the bin/met_point_verf_sfc.ksh script.
The met_point_verf_sfc.ksh script first processes the prepbufr observation files using the pb2nc tool within MET. The pb2nc configuration file (PB2NCConfig_GMTB.vx) describes the directives on how to process the observation files. A subset defined for GMTB include:
The script then moves on to run point-stat for each verification domain using the point-stat configuration files parm/met_config/PointStatConfig_ADPSFC_NDAS_REGRID and parm/met_config/PointStatConfig_ADPSFC_MPR_NDAS_REGRID. Again, the configuration files have a number of user-defined settings; for GMTB, the following fields are set:
The MET point-stat task to conduct point verification for upper-air fields in GFS_VX.xml is called met_point_ua, which is under the verification_12hr metatask. When met_point_ua is submitted it calls task/met_point_ua.ent, which sources the vx_env.ent environment variables mentioned above and sets similar environment variables as the surface point task, including:
In order for the upper-air point verification task to be run by the workflow, the following dependencies are set in met_point_ua.ent:
When all dependencies above are satisfied, the met_point_ua.ent task calls the bin/met_point_verf_ua.ksh script.
The met_point_verf_ua.ksh script first processes the prepbufr observation files using the pb2nc tool within MET, which may or may not differ from the observation files used for the surface point verification. In the case presented here, the GMTB uses the same pb2nc configuration file (parm/met_config/PB2NCConfig_GMTB) for both surface and upper-air observation processing and the only difference is that the surface task processes NDAS prepbufr files, while the upper-air task processes GDAS prepbufr files.
The script then moves on to run point-stat for each verification domain using the point-stat configuration files parm/met_config/PointStatConfig_ADPUPA_REGRID_G218 and parm/met_config/PointStatConfig_ADPUPA_REGRID_G3. A subset of user-defined settings used within GMTB include:
The MET grid-stat tool matches gridded forecast output to gridded observations or analyses. For GMTB, MET grid-stat is used to verify a variety of gridded fields and the comparison against GFS analyses to calculate the 500 hPa anomaly correlation will be described in this section. The specific task in GFS_VX.xml is:
As a reminder, all verification tasks require a set of common environment variables to be set in the master source file, vx_env.ent, including:
The task to compute the anomaly correlation for 500 hPa heights in GFS_VX.xml is called met_grid_anl, which is under the verification_06hr metatask. When met_grid_anl is submitted it calls task/met_grid_anl.ent, where the common environment variables are sourced though the vx_env.ent and a few additional environment variables are set, including:
In order for the anomaly correlation verification task to be run by the workflow, the following dependencies are set in .ent:
When all dependencies above are satisfied, the met_grid_anl.ent task calls the bin/met_grid_anl_verf_.ksh script.
The met_grid_anl_verf.ksh script runs grid-stat for each verification domain using the grid-stat configuration file parm/met_config/GridStatConfig_AC_REGRID. A subset of the user-defined settings for GMTB include:
The MET grid-stat tool matches a gridded forecast with a gridded forecast or analyses. For GMTB, MET grid-stat is used to verify quantitative precipitation forecasts (QPF). There are three specific tasks that handle precipitation verification within the GFS_VX.xml:
All three tasks all call the same korn shell script that executes MET grid-stat; it is the environment variables and configuration files that allow for met_qpf_verf_all.ksh to run to specification for each individual task. All three QPF verification tasks require the following environment variables to be set to the vx_env.ent file, which is the file all verification-related ent files source because it contains several common environment variables:
For 6-hr precipitation verification over the CONUS, the Climatology-Calibrated Precipitation Analyses (CCPA) are used as “truth” and have been pulled from the NOAA HPSS (add location). The CCPA files used for GMTB verification are files available every six hours and are in GRIB2 format. The native resolution of CCPA data is 1/8th degree??.
The task for 6-hr precipitation verification in the GFS_VX.xml is called met_qpf_6hr, which is under the 6_acc metatask (i.e., metatask for 6-hr QPF). When met_qpf_6hr is submitted, it calls task/met_qpf_06.ent, which set the following environment variables:
In order for the 6-hr QPF task to be run by the workflow, the following dependencies are set in met_qpf_06.ent:
When all dependencies above are satisfied, the met_qpf_06.ent task calls the met_qpf_verf_all.ksh script, which uses the configuration file parm/met_config/GridStatConfig_APCP06h (need to get actual name; then link to sample config file). The configuration file has user-defined settings; for GMTB, the following fields are set:
The scripts first processes the observations and model output using the MET tool, pcp_combine which has capabilities to add and subtract fields to get necessary accumulations for verification
For 24-hr precipitation verification over the CONUS, the Climatology-Calibrated Precipitation Analyses (CCPA) are used as “truth” had have been pulled from the NOAA HPSS (add location). The CCPA files used for GMTB verification are files available every six hours and are in GRIB2 format. The native resolution of CCPA data is ??. For daily QPF verification, both over the CONUS and globe, the daily accumulations are set to be from 12 UTC – 12 UTC, which is consistent with how NCEP performs daily QPF verification. Given the files are in 6-hr intervals, a tool in MET, pcp-combine, is used to sum the individual 6-hr CCPA files to create a daily file. This step is accounted for in the met_qpf_verf_all.ksh, given all environment variables are properly set.
The met_qpf_24.ent task calls the met_qpf_verf_all.ksh script, which uses the configuration file parm/met_config/GridStatConfig_APCP24h, which set the following environment variables:
In order for the 24-hr CONUS QPF task to be run by the workflow, the following dependencies are set in met_qpf_24.ent:
When all dependencies above are satisfied, the met_qpf_24.ent task calls the met_qpf_verf_all.ksh script, which uses the configuration file parm/met_config/GridStatConfig_APCP24h (need to get actual name). The configuration file has user-defined settings; for GMTB, the following fields are set:
The scripts first processes the observations and model output using the MET tool, pcp_combine which has capabilities to add and subtract fields to get necessary accumulations for verification
For 24-hr precipitation verification over the globe, the CMORPH (define, add reference) data are used as “truth” had have been pulled from a NOAA ftp site (add location). A simple pre-processing script was used convert the hourly data from binary to NetCDF (should we provide this?). The CMORPH files used in GMTB verification are files available both hourly and daily and are in NetCDF format. The native resolution of CMORPH data is 8-km, and it covers the globe from ~60°S - ~60°N. Also, CMORPH has known biases over land; therefore, verification was done over land+sea and sea only.
The met_qpf_24_global.ent task calls the met_qpf_verf_all.ksh script, which uses the configuration file parm/met_config/GridStatConfig_APCP24h_global, which set the following environment variables:
In order for the met_qpf_global_24hr task to be run by the workflow, the following dependencies are set in met_qpf_global_24hr.ent:
When all dependencies above are satisfied, the met_qpf_global_24hr.ent task calls the met_qpf_verf_all.ksh script, which uses the configuration file parm/met_config/GridStatConfig_APCP24h_CMORPH (need to get actual name). The configuration file has user-defined settings; for GMTB, the following fields are set:
Developmental Testbed Center, 2016: MET: Version 5.2 Model Evaluation Tools Users' Guide. Available at: http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.2.pdf 328 pp.
1.8.10