Temperature
BCRMSE
• A number of SS and PS pair-wise differences are noted, with most being PS and occurring in the mid-to-later portion of the forecast period. The fewest number of PS pair-wise differences tend to occur during the winter and fall aggregations. Nearly all of the pair-wise differences favor ACM2PX, regardless of initialization or seasonal aggregation.
Bias
• All differences are PS, with a weak diurnal signal in which configuration is favored. For times valid between 00-09 UTC, ACM2PX is generally the better performer; for valid times between 12-21 UTC, AFWAOC is favored more often during the summer and fall aggregations, while ACM2PX is favored more often during the winter and spring aggregations. There is a dependence on initialization and seasonal aggregation.
Dew Point Temperature
BCRMSE
• PS pair-wise differences are generally noted between the 18-06 UTC valid times and mostly favor the AFWAOC. The fewest PS pair-wise differences are noted for the winter aggregation and the most in the spring aggregation.
Bias
• When pair-wise differences are noted, all are PS, with the least amount of differences occurring in the winter and spring temporal aggregations, which is likely due to wider CIs in these seasons. More PS differences favor AFWAOC; however, there are PS pair-wise differences that favor ACM2PX, especially during the fall at times valid 09-15 UTC and the winter and spring for the 12 UTC initializations.
Wind
BCRMSE
• No PS pair-wise differences are seen; however, a large number of SS pair-wise differences are noted. A majority of the SS pair-wise differences favor the ACM2PX, with the only exception noted between 18-03 UTC valid times for the summer aggregation, which favor AFWAOC. The fewest number of SS pair-wise differences occur during the fall and spring aggregations, while all forecast lead times show SS differences favoring ACM2PX for the winter aggregation.
Bias
• While a number of SS pair-wise differences are observed, none are PS. AFWAOC is the better performer for all times that indicate a SS pair-wise difference for the summer and fall aggregations and for most of the spring aggregation valid times (exception: 03 UTC, where ACM2PX is favored). For the winter aggregation, the signal indicates that ACM2PX is the better performer for all times with SS pair-wise differences except at the valid time of 18 UTC, which favors AFWAOC.
3-hour Precipitation Accumulation
GSS
• In the summer aggregation, aggregate GSS values are higher for AFWAOC than ACM2PX for 00 UTC initializations at the 48-h forecast time, with SS pair-wise differences noted at 0.1" and 0.15" thresholds. There are no SS pair-wise differences between the two configurations during the winter, fall or spring aggregations. It is noted that the CIs bounding the aggregate values are wider for ACM2PX; the large CIs likely contribute to the small number of SS pair-wise differences.
Frequency Bias
• Despite some of the large differences in aggregate values there are only two instances of SS pair-wise differences, both favoring ACM2PX (Table 7); the large width of the CIs and the conservative method of calculating SS are likely contributors to the small number of SS pair-wise differences.
24-hour Precipitation Accumulation
GSS
• In all seasonal aggregations there are large CIs bounding the aggregated GSS value, with ACM2PX typically having larger CIs than AFWAOC; in addition, the CIs associated with the difference line are also wide. While there are no SS pair-wise differences, in the summer aggregation, it is noted AFWAOC has higher aggregated GSS values in the low-to-middle thresholds (0.5"-1.25").
Frequency Bias
• Generally, regardless of configuration, initialization hour, or forecast lead hour, a high bias is present at most thresholds, with occasional exception to the lowest and the higher thresholds in all seasonal aggregations except winter. Both configurations have wide CIs bounding the aggregate values, with ACM2PX having consistently larger CIs than AFWAOC. There are no SS pair-wise differences; however, in general, ACM2PX has higher aggregated frequency bias values than AFWAOC, and differences between values generally grow with increasing threshold.
Temperature
BCRMSE
• In general, the largest differences between the two configurations are seen at 850 and 700 hPa; PS pair-wise differences only occur at 700 hPa or below, with most PS pair-wise differences occurring in the winter and spring and showing ACM2PX as being more skilled. In addition, there are a number of SS pair-wise differences, with most favoring ACM2PX in the middle-to-upper levels. An exception is noted in the summer aggregation, where AFWAOC is favored in low-to-middle levels, with one difference being PS.
Bias
• The SS/PS pair-wise differences are a reflection of the temperature bias distributions varying between seasons and vertical level. A majority of the PS differences show ACM2PX as the better performing configuration, with the strongest signals occurring in the lowest levels (i.e., 850 and 700 hPa), upper levels (i.e., 300-150 hPa) and/or at longer forecast lead times. Exceptions are noted in the fall and spring aggregations with AFWAOC being closer to the observations at 850 hPa at most forecast lead times.
Dew Point Temperature
BCRMSE
• A number of PS pair-wise differences are seen, with all but one difference favoring AFWAOC. The largest number of PS pair-wise differences are noted for the summer aggregation where a majority of levels and forecast lead times indicate AFWAOC is the better performer. A smaller number of PS pair-wise differences favoring the AFWAOC are also seen for the annual and fall aggregations; however, only one lead time and level is PS during either the winter or spring aggregations.
Bias
• As highlighted above, there are a number of pair-wise differences with all but one PS, with a majority showing better performance in ACM2PX for the annual, summer, and fall aggregations. Most of the differences favoring ACM2PX are at 700 and 500 hPa, spanning multiple forecast lead times. Superior performance by AFWAOC is noted in the summer aggregation at all forecast lead times at 850 hPa.
Wind
BCRMSE
• Overall, both configurations have very similar distributions of BCRMSE; this is reflected when examining the statistical significance, where only a few scattered SS pair-wise differences are noted and none being PS. Most differences show ACM2PX as the better performer and are generally in the annual, winter, and spring aggregations with a concentration of differences at 850 hPa.
Bias
• While there are a number of SS pair-wise differences, only one is PS. In general, there is a split in the better performing configuration, with AFWAOC having lower errors in the annual, summer, and spring aggregations between 700-400 hPa and ACM2PX performing better in annual winter, fall, and spring aggregations at 850 hPa and at 300 hPa and above.
