HWRF HC35/HDTR Forecast Comparison
HC35/HDTR Forecast Comparison Executive Summary
• For each of the two HWRF configurations, 517 retrospective forecasts for the 2012 season of the eastern North Pacific (EP) and North Atlantic (AL) basins were conducted in order to produce a large sample from which robust conclusions could be derived.
•The experimental configuration (HDTR) slightly improved track errors in the AL, but degrade them in the EP.
• The impact of HDTR on storm intensity was mixed. In the EP, it decreased intensity, especially after 24h, resulting in a negative bias of up to 8 kt at 102 h, while the control had near zero bias. In the AL, the control had positive bias after the 12-h forecast lead time, stabilizing at 5 kt after 72 h. The HDTR configuration exacerbated this bias up to 48 h but acted to reduce the bias to near zero later in the forecast. Mean absolute intensity errors were lower for HDTR than for the control in the AL up to the 48-h lead time, but were higher in both basins at all lead times.
• The original intent of this test was to call the RRTMG radiation parameterization four times as often (every 15 minutes) than the GFDL radiation is called in the operational HWRF (every 60 minutes). An error in configuration led it to be called four times less often (every 4 hours). A follow up investigation was performed by the DTC to quantify the impact of this mistake on the results. Twenty-five cycles of Hurricane Daniel, a storm whose error characteristics were representative of the overall EP results, were run with RRTMG called every 15 minutes and compared against the control (HC35). Results indicated that the error increase in HDTR was not alleviated with this change.
• Based on the track and intensity results, EMC decided not to consider the HDTR innovations for operational implementation. Instead, additional investigation and development were recommended. The DTC followed up on that suggestion and, in 2014, developed and tested a partial cloudiness parameterization that was implemented along with RRTMG in the 2015 operational HWRF (more information here). Additionally, in 2015 the DTC is continuing to investigate and test the Thompson microphysics parameterization.
• The precipitation verification for Hurricane Sandy indicated that both configurations overpredicted precipitation when the entire parent domain or the subdomain centered around the track were considered. However, in the subdomain limited to the location of the storm at any given time, both configurations overpredicted earlier in the forecast but underpredicted towards the end. Sizeable differences in precipitation bias between the two configurations were noted only near the center of the storm, with HDTR leading to higher accumulations.
• Comprehensive results of large scale evaluation were provided, along with diagnostics of the storm-scale environment. A few hypothesis were raised, such as a possible link between negative bias in upper level winds and excessive storm intensity in the AL. Those hypotheses require further investigation.
• Model output files have been archived and are available to the community for future studies. Verification graphics, along with additional information are available in the DTC website.
Codes Employed
The software packages used in the HWRF HNR2/HR20 Forecast Comparison Test included:
•WRF - revision 4594
•WPS - revision 573
•WPP - official release v3.2
•GSI - official release v2.5
•Vortex relocation and initialization, prep_hybrid, miscellaneous
libraries and tools - hwrf-utilities revision 173
•Princeton Ocean Model (POM) and POM initialization - revision 60
•NCEP Coupler - revision 35
•GFDL Vortex Tracker - revision 49
•National Hurricane Center Verification System - revision 3
•Statistical Programming Language R for aggregation of verification
results and computation of confidence intervals
HWRF Model: HNR2 Configuration
2011 Operational HWRF Baseline configured from the community code repositories and run by DTC.
HWRF Model: HR20 Configuration
2011 Operational HWRF Baseline configured from the NCEP/EMC code repositories and run by EMC.
Differences in Configuration:| HNR2 | HR20 | |
| Institution | DTC | EMC |
| Platform | Linux | IBM |
| Source code | Community | EMC |
| Scripts | DTC | EMC |
| Automation | NOAA GSD Workflow Manager | EMC HWRF History Sequence Manager |
| I/O format | NetCDF | Binary |
| WPP | WPP v3.2 | NAM Post modified for HWRF |
| Tracker | Community repository | EMC operational |
| Sharpening in ocean init | Used in spin up Phases 3 and 4 | Used in Phase 3 only (known bug) |
| Snow Albedo | Older dataset | Newer dataset |
Domain Configuration
The HWRF domain was configured the same way as used in the NCEP/EMC operational system. The atmospheric model employed a parent and a movable nested grid. The parent grid covered a 75x75 deg area with approximately 27 km horizontal grid spacing. There were a total of 216 x 432 grid points in the parent grid. The nest covered a 5.4 x 5.4 deg area with approximately 9 km grid spacing. There were a total of 60 x 100 grid points in the nest. The location of the parent and nest, as well as the pole of the projection, varied from run to run and were dictated by the location of the storm at the time of initialization.
HWRF was run coupled to the POM ocean model for Atlantic storms and in atmosphere-only mode for East Pacific storms. The POM domain for the Atlantic storms depended on the location of the storm at the initialization time and on the 72-h NHC forecast for the storm location. Those parameters defined whether the East Atlantic or United domain of the POM was used.
The image shows the atmospheric parent and nest domains (yellow) and the United POM domain (blue).
Click for larger image.
Cases Run
• Storms: 53 complete storms from 2008, 2009, and 2010.
• 2008 Atlantic: Fay, Gustav, Hanna, Ike
• 2008 Pacific: Elida, Fausto, Genevieve, Marie, Norbert
• 2009 Atlantic: Bill, Claudette, Danny, Erika, Fred, Henri, Ida
• 2009 Pacific: Felicia, Guillermo, Hilda, Ignacio, Jimena, Linda,
Olaf, Rick
• 2010 Atlantic: Alex, Bonnie, Collin, Danielle, Earl, Fiona,
Gaston, Hermine, Igor, Julia, Karl, Lisa, Matthew, Nicole, Otto,
• 2010 Pacific: Blas, Celia, Darby, Six, Estelle, Eight, Frank, Ten,
Eleven, Georgette
• Initializations: Every 6 h, in cycled mode.
• Forecast Length: 126 hours; output files available every 6 hours
Verification
The characteristics of the forecast storm (location, intensity, structure) were compared against the Best Track using the National Hurricane Center (NHC) Verification System (NHCVx). The HNR2 ATCF files were produced by the DTC as part of this test, while the HR20 ATCF files were supplied by NOAA/NCEP/EMC. The NHCVx was run separately for each case, at 6-hourly forecast lead times, out to 120 h, in order to generate a distribution of errors. Verification was performed for any geographical location for which Best Track was available, including over land. No verification was performed when the observed storm was classified as a low or wave.
An R-statistical language script was run on an homogenous sample of the HNR2 and HR20 datasets to aggregate the errors and to create summary metrics including the mean and median of track error, intensity error, absolute intensity error, and radii of 34, 50, and 64 kt wind in all four quadrants. All metrics are accompanied of 95% confidence intervals to describe the uncertainty in the results due to sampling limitations.
For the purposes of comparing the HNR2 and HR20 forecasts, pairwise differences (HNR2-HR20) of track error, and absolute intensity error, and absolute wind radii error were computed and aggregated with a R-statistical language script. Ninety-five percent confidence intervals on the median were computed to determine if there is a statistically significant difference between the two configurations.