We are using HWRF's produtil package to ingest various datasets needed for our experiment.

Scripts and Configuration Files

The main script used to pull data is hwrf_pull_inputs.py which is located in the parm directory.

The following configuration files are located in the ush directory. Data locations and naming conventions are specified in these files.

hwrf_input.conf
gfs_fcst_input.pull
gfs_refcst_input.pull
gfs_vx_input.pull

Input Tasks

Task Name	Task File	Purpose	Configuration File	Dataset
fcst_in_task	input_fcst.ent	Pull in SIGIO Initial Condition Files	gfs_fcst_input.pull	gfs_sanl, gfs_sfcanl
gfs_fcst_in_task	Input_gfsfcst.ent	Pull in GFS forecasts used for baseline	gfs_refcst_input.pull	gfs_gribA, gfs_gribB
vx_in_task	input_vx.ent	Pull in GDAS forecasts used for vx	gfs_vx_input.pull	gfs_pgrb_anl

Task Name	Production Archive Tar File	Production Filename	Parallel Filename	Output Directory
fcst_in_task	com_gfs_prod_gfs.yyyymmddhh.anl.tar	gfs.t00z.sanl, gfs.t00z.sfcanl	gfnanl.gfs.yyyymmddhh, sfnanl.gfs.yyyymmddhh	GMTB_HOME/chgres/input
gfs_fcst_in_task	com2_gfs_prod_gfs.yyyymmddhh.pgrb2_0p25.tar	gfs.t00z.pgrb2.0p25.fhhh	pgrbqfhhh.gfs.yyyymmddhh.grib2	ROTDIR/yyyymmddhh
vx_in_task	com2_gfs_prod_gdas.yyyymmddhh.tar	gdas1.t00z.pgrbanl	pgrbqanl.gdas.yyyymmddhh.grib2	GFS_ANL_OBS_DIR

Command to pull data:

 GMTB_UTIL/ush/hwrf_pull_inputs.py -v -w OUTPUT_DIR -o OUTPUT_DIR GMTB_UTIL/ush/PULL_CONF_FILE yyyymmddhh

      where GMTB_HOME        = /scratch4/BMC/gmtb/gmtb-tierIII
            ROTDIR           = GMTB_HOME/model                             
            GMTB_UTIL        = GMTB_HOME/util/gmtb_utils
            GFS_ANL_OBS_DIR  = /scratch3/BMC/dtc-hwrf/GMTB/vx_data/analyses/gfs_0p25/yyyymmddhh


             model = refcst, sasctrl, gftest