Funding to maintain WRF-NMM at the DTC has ended and support will no longer be provided. To download HWRF, please see: www.dtcenter.org/HurrWRF/users. To download WRF-ARW, please see: www2.mmm.ucar.edu/wrf/users . To download NEMS-NMMB, please see: www.dtcenter.org/nems_nmmb/users.To download UPP, please see: www.dtcenter.org/upp/users.

Testing for WRF

The release of the WRF source code was preceded by months of testing by developers, scientists, and software engineers. The source code is maintained in a repository, where contributions are tested weekly. Before any modifications to the repository are accepted, the developers have typically worked for months to incorporate their contributions into the WRF modeling system framework. During this pre-commit period, the developers fine tune their code with case studies and statistics. Once the code is accepted and committed, then any change to the contributed code requires an OK from that specific developer. The tests described here are primarily conducted by DTC, EMC/NOAA, GSD/NOAA, MMM/NCAR, and RAL/NCAR.

Automated Tests

The purpose of automated testing is to catch mistakes that accidently get incorporated into the repository, which makes this more of an engineering task than science related. These commit errors tend to break *other* portions of code rather than in the section where the developer was working (such as side effects in nesting, communications, or in a different dynamical core). These tests are a backstop to catch the obvious problems. There are several automatic tests that are conducted on the WRF source code.

Every week, after the approved commits to the repository, a series of 16 tests that compare bit-wise results (between 1 and 4 processors) is conducted on the NCAR IBM bluefire machine. Towards the end of the release cycle, this suite is also run on two other architecture/compiler combinations: Mac/g95 and Linux/PGI. These tests cover single domains, 7 different physics combinations for ARW real data cases, 3 physics/dynamics/boundary settings for each of two different 3-d idealized cases, nesting, IO quilting, real*8 data types, adaptive time stepping, a global domain, two sets of Chemistry tests, usage of the ESMF libraries, Grib and binary IO, and observation and analysis nudging.

A second large set of automated tests includes 11 NMM and 3 ARW members, each with different combinations of physics. The NMM tests also include restarts, and 1- and 2-way nesting. These tests are conducted weekly on the IBM throughout the year, and on other architecture/compiler combinations as the release approaches (Linux/PGI, Linux/gfortran, wjet/ifort).

The last large suite of automated tests covers the WRF DA system. A set of 7 cases (different domains, different WRF DA options) are conducted on the NCAR IBM machine.

Manual Testing

The purpose of manual testing is more scientific. These tests are conducted by scientists who evaluate the results, not just a simple pass/fail. The concern for manual testing tends towards "does this scheme produce better results than before", "how does this new parameter table impact my hurricane track", "does the model remain stable with this option". Of course, every contributor expends months (if not years) of time testing, evaluating, and tuning a scheme. Once those schemes are part of the WRF repository, the local Developer's Committee and Release Committee members conduct manual tests intermittently throughout the year, with increasing focus as the release date approaches. Where the automated testing handles thousands of tests, manual testing drills down to a few well known cases, with a specificity of validation appropriate to the case.

A single case for ARW is used to provide the vehicle for a suite of tests, such as high-resolution 12/4 km nesting, different physics combinations, moving nests, ndown, various nudging combinations, and restarts. A high-resolution 3-km CONUS domain with a well-studied convective event is also run.

A series of 24-h forecasts covering a one-month period for August 2006 is run with multiple Chemistry options. The advantage of this testing is the objective validation vs observations that is performed.

Pre- and Post-Processing Testing

As the WRF model evolves, the pre- and post-processor developers continue to chase a moving target. The WPS has a regression tests that runs through multiple domains and configurations. The diagnostic and graphics systems are tested for the new output.

Architectures with Regression Tests and Other Tests

The primary testing system at NCAR has been the two IBM machines, bluevista and blueice. During the last month prior to the release, the regression testing has included a multi-processor MAC/Intel running the g95 compiler. Just prior to the code release, the testing has included an Intel IA32 with PGI.

bluefire AIX, xlfrte 12.1.0.8

Dell 64-bit running Linux, with Intel chips, PGI pgf90 10.3

MAC 64-bit running OSX, with Intel chips, g95 gcc version 4.0.3 (g95 0.92!), PGI pgf90 10.3

Other compilers tested:

• gfortran 4.3.2
• ifort 10.1 (will not work with ifort 9), 11.1
• pgi 8.0-5
• Cray XT4/5 Peter Johnsen (Cray)