NAM Configuration Testing

Executive Summary

  • Surface
  • Upper Air

Temperature

BCRMSE

    • A number of SS pair-wise differences are noted in both the East and West, with none being PS. In general, in the West, most SS pair-wise difference favor ThompsonMP and occur most frequently at times valid around 21-00 UTC. In the East most SS pair-wise differences favor NAMOC, with the main exceptions during the summer season.
    • Over Alaska, the largest differences are found in winter for the 30-42-h forecast lead times of the 00 UTC initialization where NAMOC is PS better than ThompsonMP. During the summer, even though the differences are only SS, a number of lead times show ThompsonMP as the better performer for both initializations.

Bias

    • Nearly all pair-wise differences are PS for both the East and the West. In the West, during the summer, all PS pair-wise differences favor ThompsonMP during the day, while in the winter, all PS pair-wise differences favor NAMOC at valid times between 18-00 UTC. For fall, PS pair-wise differences favor NAMOC at times valid around 00 UTC, with all other PS differences favoring ThompsnoMP. During the Spring, most PS differences favor ThompsonMP. In the East, a majority of the PS pair-wise differences favor ThompsonMP with exceptions in the winter where NAMOC is the better performer at valid times between 15-03 UTC.
    • Over Alaska, during the summer, all PS pair-wise differences favor ThompsonMP, while no PS differences are found during the winter. For the spring aggregation, NAMOC is favored at valid times from 18-03 UTC, while ThompsonMP is closer to observations around times valid at 12 UTC. Trends in the fall are similar but have less PS pair-wise differences.

Dew Point Temperature

BCRMSE

    • For both the East and the West, SS pair-wise differences tend to favor NAMOC more often in the early forecast hours, especially for the 00 UTC initialization, while ThompsonMP is favored in the later forecast hours, especially for the spring and summer aggregations. The fewest number of differences are noted during the winter. Only one instance is PS.
    • Over Alaska, during the summer aggregation, when SS pair-wise differences occur, they favor ThompsonMP. On the other hand, differences found during the winter show NAMOC as the better performer with several of the differences being PS. The fewest pair-wise differences are found during the fall and spring, with none being PS.

Bias

    • A number of pair-wise differnces are present in both the East and the West, many favor ThompsonMP with PS during the summer in the East. For the 00 UTC initializations, NAMOC is the SS better performer in the first 12 hours of the forecast, except spring, where no SS pair-wise differences are observed. For the later hours, ThompsonMP is favored for all seasons except winter, where few differences are noted. For the 12 UTC initializations, all differences favor ThompsonMP, except one instance during the winter.
    • Over Alaska, with a few exceptions for the fall 12 UTC initializations, SS pair-wise dfferences are noted favoring NAMOC for the fall and winter aggregations; a few pair-wise differences during the winter are PS. For summer and spring, SS pair-wise differences favoring ThompsonMP are observed for all but one of the instances when differences are found.

Wind

BCRMSE

    • For both the East and West regions, when pair-wise differences are found, they are SS with a majority favoring ThompsonMP; the most frequent exception is observed during the 12 UTC winter aggregation in the East where NAMOC is more often favored. The fewest number of SS pair-wise differences occur during the winter and spring aggregations, with the most found during the summer. No PS differences are noted.
    • Over Alaska, all pair-wise differences are SS with all favoring ThompsonMP for the 00 UTC initializations. For the 12 UTC initializations, ThompsonMP is still favored during the summer and fall; however, NAMOC is preferred during the winter and spring.

Bias

    • For both the East and the West, a number of SS pair-wise difference are found, with non being PS. For the East region, nearly all SS differences favor ThompsonMP with the main exception found in the winter aggregation where NAMOC is frequently favored. On the other hand, in the West, NAMOC is favored most often with the main exception during the summer when ThompsonMP is generally favored between the 15-18 UTC valid times.
    • Over Alaska, SS pair-wise differences are found with the favored configuration dependent on season and valid time. In general, the NAMOC configuration is favored most often during the spring, while ThompsonMP is favored more in the summer. The fewest differences occur during the fall and winter.

Daily Precipitation Accumulation

GSS

    • In general, ThompsonMP has higher GSS values at the lowest threshold (> 0.01"); however, there is only one instance at this threshold where there is a SS pair-wise difference. At higher thresholds, the two configurations are very similar, with only one instance favoring NAMOC. Large CIs are observed for both configurations, which is likely the reason for the small amount of SS pair-wise differences.

Frequency Bias

    • At the lowest thresholds, both configurations are generally similar in magnitude for frequency bias. Only one instance of SS pair-wise difference is noted at the lowest threshold, favoring ThompsonMP. At higher thresholds, NAMOC is often higher in magnitude; however, the only SS pair-wise difference found at > 3" favors ThompsonMP.

Composite Reflectivity

GSS

    • In general, composite reflectivity GSS decreases with increasing threhold for both configurations in both the East and the West. While ThompsonMP is frequently higher in magnitude than NAMOC at the lowest threshold (> 10 dBZ), only one SS pair-wise differences is found, and favors NAMOC. A larger number of SS pair-wise differences are found at > 40 and >50 dBZ thresholds; however, the favored configuration depends on season and forecast lead time. A larger number of SS pair-wise differences favor NAMOC over ThompsonMP for both the East and West.

Frequency Bias

    • In general, NAMOC frequency bias values are larger compared to ThompsonMP in both the East and West. Nearly all SS pair-wise differences favor ThompsonMP, with only a few exceptions at higher thresholds. At the >10 dBZ threshold, there are SS pair-wise differences for all forecast lead times during the Winter in the East and during spring in the West. The fewest pair-wise differences occur in summer for both regions. As threshold increases, fewer SS pair-wise difference are noted.

Temperature

BCRMSE

    • A small number of pair-wise differences are observed in both the East and the West, with none being PS. In the West, SS pair-wise differences during the fall, winter and spring favor ThompsonMP, while NAMOC is favored for all but one instance during the summer. In the East, differences favoring NAMOC are more frequently observed in the summer and fall, with only a few differences, mostly favoring ThompsonMP, are noted in spring and winter.
    • Over Alaska, few pair-wise differences are found, with ThompsonMP being SS favored for all but one instance. No PS differences are observed.

Bias

    • For all seasons, any PS pair-wise differences at and above 300 hPa favor NAMOC, while those that favor ThompsonMP are located at and below 400 hPa. However, all PS pair-wise differences in fall and winter favor NAMOC. In the East, PS pair-wise differences favor ThompsonMP mainly at levels between 850-700 hPA with the mid-to-upper levels showing NAMOC as the better performer. All PS differences favor NAMOC in the Winter.
    • Over Alaska, a number of pair-wise differences are found, with all PS differences occuring within the low to mid-upper levels (850-300 hPa). For PS pair-wise differences, with exception to the summer aggregation where ThompsonMP is the better performer, NAMOC is preferred.

Dew Point Temperature

BCRMSE

    • For both the East and the West, very few pair-wise differences are observed between the two configurations. Differences depend on region, season, level and forecast hour, with no pair-wise differences found during the summer aggregation.
    • Over Alaska, in general, the pair-wise differences that occur in the summer and spring at low levels (850-700 hPA) favor ThompsonMP, while for the fall, winter and spring, the SS/PS differences in the upper levels (500-300 hPA) favor NAMOC.

Bias

    • For all seasonal aggregations, pair-wise differences are found in both regions, with most PS differences found in the upper levels from 400-300 hPa. In general, the favored configuration varies by level and lead time; however, for both the East and the West in the spring and West during the summer, when SS/PS differences are observed, NAMOC is favored.
    • Over Alaska, a number of pair-wise differences are observed with most PS differences found in the upper levels of 400-300 hPA. During the summer, all differences favor NAMOC with PS differences found at the 300 hPa level. During the fall, spring and winter aggregations, ThompsonMP is favored with PS pair-wise differences at nearly all lead times at 300 hPa. Below 300 hPa, the better performer depends on level, season and lead time.

Wind

BCRMSE

    • Very few pair-wise differences are noted between the two configurations; however, when differences are seen, they are more often found in the West compared to the East. When pair-wise differences occur, they are SS and ThompsonMP is frequently the better performer.
    • Over Alaska, very few pair-wise differences are observed, with all being SS. No differences are noted during the spring aggregation.

Bias

    • For both the East and West, several SS pair-wise differences are found, with none being PS. Across the East, NAMOC is more frequently favored, especially during the fall aggregation in the mid-to-upper levels. In the West, the favored configuration depends on level, lead time and season.
    • Over Alaska, several SS pair-wise differences are present, with none being PS. NAMOC is favored most often at levels 300 hPa and below, while ThompsonMP is favored at 200 hPa and above. The fewest pair-wise differences are noted in the winter, with the most found during the spring.