5. Autogenerated Physics Caps

The connection between the host model and the physics schemes through the CCPP-Framework is realized with caps on both sides as illustrated in Figure 1.1. The CCPP prebuild script discussed in Chapter 3 generates the caps that connect the physics schemes to the CCPP-Framework, and a large fraction of code that can be included in the host model cap. The host model cap connects the framework (Physics Driver) to the host model and must be created manually incorporating the autogenerated code. This chapter describes the physics caps, while the host model caps are described in Chapter 6. Note that the dynamic build, which can be used with the SCM and with the UFS Atmosphere, produces individual physics scheme caps, while the static build (for UFS Atmosphere only) produces group and suite caps. The physics caps autogenerated by ccpp_prebuild.py reside in the directory defined by the CAPS_DIR variable (see example in Listing 8.1).

5.1. Dynamic Build Caps

With the dynamic build, the CCPP-Framework and physics are dynamically linked to the executable to allow maximum runtime flexibility. A cap is generated using the metadata requested in each scheme argument table and the variables provided in the host model metadata tables (see left side of Figure 3.2). The CCPP prebuild step for the dynamic build performs the following tasks:

• Checks requested vs provided variables by standard_name.
• Checks units, rank, type.
• Creates Fortran code that adds pointers to the host model variables and stores them in the ccpp-data structure (ccpp_fields_*.inc). A hash table that is part of cdata is populated with key = standard_name of a variable and value = location of that variable in memory (i.e. a c-pointer).
• Creates one cap per physics scheme.
• Populates makefiles with schemes and caps.

The prebuild step will produce the following files for the UFS Atmosphere model:

• List of variables provided by host model and required by physics:

ccpp/framework/doc/DevelopersGuide/CCPP_VARIABLES_FV3.tex

• Makefile snippets that contain all caps to be compiled:

ccpp/physics/CCPP_CAPS.{cmake,mk}

• Makefile snippets that contain all schemes (and their dependencies) to be compiled:

ccpp/physics/CCPP_SCHEMES.{cmake,mk}

• List of variables provided by host model:

ccpp/physics/CCPP_VARIABLES_FV3.html

• One cap per physics scheme:

ccpp/physics/*_cap.F90

• *.inc files that contain module use and ccpp_field_add statements that populate the ccpp data type (cdata) with the necessary information on where (in memory) to find required variables:

FV3/atmos_cubed_sphere/driver/fvGFS/ccpp_modules_{fast,slow}_physics.inc
FV3/atmos_cubed_sphere/driver/fvGFS/ccpp_fields_{fast,slow}_physics.inc
FV3/ipd/ccpp_modules_{fast,slow}_physics.inc
FV3/ipd/ccpp_fields_{fast,slow}_physics.inc

The variables added to *_fast_physics.inc do not use GFS_typedefs.F90 or CCPP_data.F90.

• Autogenerated code to include in host model caps (called TARGET FILES) via CPP (C preprocessor) directives:

FV3/ipd/IPD_CCPP_driver.F90 for slow physics
FV3/atmos_cubed_sphere/driver/fvGFS/atmosphere.F90 for fast physics

For each cap, ccpp_prebuild.py generates “use” statements based on the host model template. Only the public caps (init, run and finalize) are exposed (see code example below). Each cap consists of a module containing three functions. For example, scheme_pre_cap.F90 would contain the functions scheme_pre_init_cap, scheme_pre_run_cap and scheme_pre_finalize_cap, which perform the functions below.

• Declare data types cptr, cdims and ckind.
• Create a pointer to the Fortran data type cdata.
• Call ccpp_field_get for each variable in metadata table and pulls data from the cdata structure.

The index defined in each call speeds up memory access by avoiding a binary search, since variables are no longer searched by name; the order of the data in cdata are known.

• Call the corresponding scheme entry-point at the end with an explicit argument list.

For example, the autogenerated scheme cap for rrtmg_lw_pre_cap.F90 is shown in Listing 5.1.

module rrtmg_lw_pre_cap
 use, intrinsic :: iso_c_binding, only: c_f_pointer, &
                   c_ptr, c_int32_t
 use            :: ccpp_types,  only: ccpp_t, CCPP_GENERIC_KIND
 use            :: ccpp_fields, only: ccpp_field_get
 use            :: ccpp_errors, only: ccpp_error, ccpp_debug
 use            :: rrtmg_lw_pre, only: rrtmg_lw_pre_run, &
                   rrtmg_lw_pre_init,rrtmg_lw_pre_finalize
 ! Other modules required, e.g. type definitions
 use GFS_typedefs, only: GFS_control_type,GFS_grid_type, &
                         GFS_sfcprop_type,GFS_radtend_type
 use machine, only: kind_phys
 implicit none
 private
 public :: rrtmg_lw_pre_run_cap,rrtmg_lw_pre_init_cap, &
           rrtmg_lw_pre_finalize_cap
 contains
 function rrtmg_lw_pre_init_cap(ptr) bind(c) result(ierr)
     integer(c_int32_t)         :: ierr
     type(c_ptr), intent(inout) :: ptr
     type(ccpp_t), pointer           :: cdata
     type(c_ptr)                     :: cptr
     integer, allocatable            :: cdims(:)
     integer                         :: ckind
     ierr = 0
     call c_f_pointer(ptr, cdata)
     call rrtmg_lw_pre_init()
 end function rrtmg_lw_pre_init_cap

 function rrtmg_lw_pre_run_cap(ptr) bind(c) result(ierr)
     integer(c_int32_t)         :: ierr
     type(c_ptr), intent(inout) :: ptr
     type(ccpp_t), pointer           :: cdata
     type(c_ptr)                     :: cptr
     integer, allocatable            :: cdims(:)
     integer                         :: ckind
     type(GFS_control_type), pointer     :: Model
     type(GFS_grid_type), pointer     :: Grid
     type(GFS_sfcprop_type), pointer     :: Sfcprop
     type(GFS_radtend_type), pointer     :: Radtend
     integer, pointer :: im
     real(kind_phys), pointer :: tsfg(:)
     real(kind_phys), pointer :: tsfa(:)
     ierr = 0
     call c_f_pointer(ptr, cdata)
     call ccpp_field_get(cdata,'GFS_control_type_instance',cptr,&
          ierr=ierr, kind=ckind, index=2)
     call c_f_pointer(cptr, Model)
     call ccpp_field_get(cdata,'GFS_grid_type_instance',cptr,&
          ierr=ierr, kind=ckind, index=6)
     call c_f_pointer(cptr, Grid)
     call ccpp_field_get(cdata, 'GFS_sfcprop_type_instance', &
          cptr, ierr=ierr, kind=ckind, index=10)
     call c_f_pointer(cptr, Sfcprop)
     call ccpp_field_get(cdata, 'GFS_radtend_type_instance', &
          cptr, ierr=ierr, kind=ckind, index=9)
     call c_f_pointer(cptr, Radtend)
     call ccpp_field_get(cdata, 'horizontal_loop_extent', im,&
          ierr=ierr, kind=ckind, index=390)
     call ccpp_field_get(cdata, &
          'surface_ground_temperature_for_radiation', &
           tsfg, ierr=ierr, dims=cdims, kind=ckind, index=770)
     deallocate(cdims)
     call ccpp_field_get(cdata, &
          'surface_air_temperature_for_radiation', &
           tsfa, ierr=ierr, dims=cdims, kind=ckind, index=724)
     deallocate(cdims)
     call rrtmg_lw_pre_run(Model=Model,Grid=Grid, &
          Sfcprop=Sfcprop,Radtend=Radtend,im=im, &
          tsfg=tsfg,tsfa=tsfa, &
          errmsg=cdata%errmsg,errflg=cdata%errflg)
     ierr=cdata%errflg
 end function rrtmg_lw_pre_run_cap
 function rrtmg_lw_pre_finalize_cap(ptr) bind(c) result(ierr)
     integer(c_int32_t)         :: ierr
     type(c_ptr), intent(inout) :: ptr
     type(ccpp_t), pointer           :: cdata
     type(c_ptr)                     :: cptr
     integer, allocatable            :: cdims(:)
     integer                         :: ckind
     ierr = 0
     call c_f_pointer(ptr, cdata)
     call rrtmg_lw_pre_finalize()
 end function rrtmg_lw_pre_finalize_cap
end module rrtmg_lw_pre_cap

Listing 5.1: Condensed version of the autogenerated scheme cap rrtmg_lw_pre_cap.F90 for the dynamic build. Note the calls to ccpp_field_get for each variable.

The fields accessed from cdata are determined by the metadata in the scheme argument table(s). In this example, rrtmg_lw_pre_init and rrtmg_lw_pre_finalize are empty subroutines, i.e. they have no arguments passed in or out, no metadata table, and no calls to ccpp_field_get. However, rrtmg_lw_pre_run has a metadata table, so ccpp_field_get is called for each variable in the table and the value put into the call to rrtmg_lw_pre_run.

5.2. Static Build Caps

With a static build, the CCPP-Framework and physics are statically linked to the executable. This allows the best performance and efficient memory use. Similar to the dynamic build, the static build requires metadata provided by the host model and variables requested from the physics scheme. Unlike a dynamic build where all variables are kept and pulled multiple times for various parameterizations, a static build only keeps variables for specified suites, and therefore requires one or more SDFs (see left side of Figure 3.3) as arguments to the ccpp_prebuild.py script. The CCPP prebuild step for the static build performs the tasks below.

• Check requested vs provided variables by standard_name.
• Check units, rank, type.
• Filter unused schemes and variables.
• Create Fortran code for the static Application Programming Interface (API) that replaces the dynamic API (CCPP-Framework). The hash table used by the dynamic build to store variables in memory is left empty.
• Create caps for groups and suite(s).
• Populate makefiles with schemes and caps.

The prebuild step for the static build will produce the following files for the UFS Atmosphere:

• List of variables provided by host model and required by physics:

ccpp/framework/doc/DevelopersGuide/CCPP_VARIABLES_FV3.tex

• Makefile snippets that contain all caps to be compiled:

ccpp/physics/CCPP_CAPS.{cmake,mk}

• Makefile snippets that contain all schemes to be compiled:

ccpp/physics/CCPP_SCHEMES.{cmake,mk}

• List of variables provided by host model:

ccpp/physics/CCPP_VARIABLES_FV3.html

• One cap per physics group (fast_physics, physics, radiation, time_vary, stochastic, …) for each suite:

ccpp/physics/ccpp_{suite_name}_{group_name}_cap.F90

• Cap for each suite:

ccpp/physics/ccpp_{suite_name}_cap.F90

• Autogenerated API for static build that replaces the dynamic API (aka CCPP-Framework), the interface is identical between the two APIs:

FV3/gfsphysics/CCPP_layer/ccpp_static_api.F90

• Same TARGET FILES as for the dynamic build

ccpp_static_api.F90 replaces the entire dynamic CCPP-Framework with an equivalent interface, which contains subroutines ccpp_physics_init, ccpp_physics_run and ccpp_physics_finalize. Each subroutine uses a suite_name and an optional argument, group_name, to call the groups of a specified suite (e.g. fast_physics, physics, time_vary, radiation, stochastic, etc.), or to call the entire suite. For example, ccpp_static_api.F90 would contain module ccpp_static_api with subroutines ccpp_physics_{init, run, finalize}. The subroutine ccpp_physics_init from the autogenerated code using suites FV3_GFS_v15 and FV3_CPT_v0 is shown in Listing 5.2.

subroutine ccpp_physics_init(cdata, suite_name, group_name, ierr)
  use ccpp_types, only : ccpp_t
  implicit none
  type(ccpp_t),               intent(inout) :: cdata
  character(len=*),           intent(in)    :: suite_name
  character(len=*), optional, intent(in)    :: group_name
  integer,                    intent(out)   :: ierr
  ierr = 0
  if (trim(suite_name)=="FV3_GFS_v15") then
    if (present(group_name)) then
      if (trim(group_name)=="fast_physics") then
        ierr = FV3_GFS_v15_fast_physics_init_cap(cdata=cdata, CCPP_interstitial=CCPP_interstitial)
      else if (trim(group_name)=="time_vary") then
        ierr = FV3_GFS_v15_time_vary_init_cap(GFS_Interstitial=GFS_Interstitial, &
               cdata=cdata,GFS_Data=GFS_Data, GFS_Control=GFS_Control)
      else if (trim(group_name)=="radiation") then
        ierr = FV3_GFS_v15_radiation_init_cap()
      else if (trim(group_name)=="physics") then
        ierr = FV3_GFS_v15_physics_init_cap(cdata=cdata, GFS_Control=GFS_Control)
      else if (trim(group_name)=="stochastics") then
        ierr = FV3_GFS_v15_stochastics_init_cap()
      else
        write(cdata%errmsg, '(*(a))') "Group " // trim(group_name) // " not found"
        ierr = 1
      end if
    else
      ierr = FV3_GFS_v15_init_cap(GFS_Interstitial=GFS_Interstitial, cdata=cdata,GFS_Control=GFS_Control, &
            GFS_Data=GFS_Data, CCPP_interstitial=CCPP_interstitial)
    end if
  else if (trim(suite_name)=="FV3_CPT_v0") then
    if (present(group_name)) then
      if (trim(group_name)=="time_vary") then
        ierr = FV3_CPT_v0_time_vary_init_cap(GFS_Interstitial=GFS_Interstitial, &
               cdata=cdata,GFS_Data=GFS_Data, GFS_Control=GFS_Control)
      else if (trim(group_name)=="radiation") then
        ierr = FV3_CPT_v0_radiation_init_cap()
      else if (trim(group_name)=="physics") then
        ierr = FV3_CPT_v0_physics_init_cap(con_hfus=con_hfus, &
                  GFS_Control=GFS_Control,con_hvap=con_hvap, &
                  con_rd=con_rd,con_rv=con_rv,con_g=con_g, &
                  con_ttp=con_ttp,con_cp=con_cp,cdata=cdata)
      else if (trim(group_name)=="stochastics") then
        ierr = FV3_CPT_v0_stochastics_init_cap()
      else
        write(cdata%errmsg, '(*(a))') "Group " // trim(group_name) // " not found"
        ierr = 1
      end if
    else
      ierr = FV3_CPT_v0_init_cap(con_g=con_g, GFS_Data=GFS_Data,GFS_Control=GFS_Control, &
             con_hvap=con_hvap,GFS_Interstitial=GFS_Interstitial, con_rd=con_rd,con_rv=con_rv, &
             con_hfus=con_hfus, con_ttp=con_ttp,con_cp=con_cp,cdata=cdata)
    end if
  else
    write(cdata%errmsg,'(*(a))'), 'Invalid suite ' // trim(suite_name)
    ierr = 1
  end if
  cdata%errflg = ierr
end subroutine ccpp_physics_init

Listing 5.2: Code sample of subroutine ccpp_physics_init contained in the autogenerated file ccpp_static_api.F90 for the multi-suite static build. This cap was generated using suites FV3_GFS_v15 and FV3_CPT_v0. Examples of the highlighted functions are shown below in Listing 5.3 and Listing 5.4.

Note that if group_name is set, specified groups (i.e. FV3_GFS_v15_physics_init_cap) are called for the specified suite_name. These functions are defined in ccpp_{suite_name}_{group_name}_cap.F90, in this case ccpp_FV3_GFS_v15_physics_cap.F90. For example:

function FV3_GFS_v15_physics_init_cap(cdata,GFS_Control)&
        result(ierr)
   use ccpp_types, only: ccpp_t
   use GFS_typedefs, only: GFS_control_type
   implicit none
   integer                     :: ierr
   type(ccpp_t), intent(inout) :: cdata
   type(GFS_control_type), intent(in) :: GFS_Control
   ierr = 0
   if (initialized) return
   call lsm_noah_init(me=GFS_Control%me,isot=GFS_Control%isot,&
         ivegsrc=GFS_Control%ivegsrc,nlunit=GFS_Control%nlunit, &
         errmsg=cdata%errmsg,errflg=cdata%errflg)
   if (cdata%errflg/=0) then
     write(cdata%errmsg,'(a)') "An error occured in lsm_noah_init"
     ierr=cdata%errflg
     return
   end if
   call gfdl_cloud_microphys_init(me=GFS_Control%me, &
        master=GFS_Control%master,nlunit=GFS_Control%nlunit, &
        input_nml_file=GFS_Control%input_nml_file, &
        logunit=GFS_Control%logunit,fn_nml=GFS_Control%fn_nml, &
        imp_physics=GFS_Control%imp_physics, &
        imp_physics_gfdl=GFS_Control%imp_physics_gfdl, &
        do_shoc=GFS_Control%do_shoc, &
        errmsg=cdata%errmsg,errflg=cdata%errflg)
   if (cdata%errflg/=0) then
     write(cdata%errmsg,'(a)') "An error occured in &
           gfdl_cloud_microphys_init"
     ierr=cdata%errflg
     return
   end if
   initialized = .true.
end function FV3_GFS_v15_physics_init_cap

Listing 5.3: The FV3_GFS_v15_physics_init_cap contained in the autogenerated file ccpp_FV3_GFS_v15_physics_cap.F90 showing calls to the lsm_noah_init , and gfdl_cloud_microphys_init subroutines for the static build for suite ‘FV3_GFS_v15’ and group ‘physics’.

If the group_name is not specified for a specified suite_name, the suite is called from the autogenerated ccpp_static_api.F90, which calls the init, run and finalize routines for each group. Listing 5.4 is an example of FV3_GFS_v15_init_cap.

function FV3_GFS_v15_init_cap(GFS_Interstitial, &
  cdata,GFS_Control,GFS_Data,CCPP_interstitial) result(ierr)
  use GFS_typedefs, only: GFS_interstitial_type
  use ccpp_types, only: ccpp_t
  use GFS_typedefs, only: GFS_control_type
  use GFS_typedefs, only: GFS_data_type
  use CCPP_typedefs, only: CCPP_interstitial_type

  implicit none

  integer :: ierr
  type(GFS_interstitial_type), intent(inout) :: GFS_Interstitial(:)
  type(ccpp_t), intent(inout) :: cdata
  type(GFS_control_type), intent(inout) :: GFS_Control
  type(GFS_data_type), intent(inout) :: GFS_Data(:)
  type(CCPP_interstitial_type), intent(in) :: CCPP_interstitial

  ierr = 0
  ierr = FV3_GFS_v15_fast_physics_init_cap(cdata=cdata, CCPP_interstitial=CCPP_interstitial)
  if (ierr/=0) return

  ierr = FV3_GFS_v15_time_vary_init_cap (GFS_Interstitial=GFS_Interstitial,cdata=cdata, &
         GFS_Data=GFS_Data,GFS_Control=GFS_Control)
  if (ierr/=0) return

  ierr = FV3_GFS_v15_radiation_init_cap()
  if (ierr/=0) return
  ierr = FV3_GFS_v15_physics_init_cap(cdata=cdata, &
      GFS_Control=GFS_Control)
  if (ierr/=0) return

  ierr = FV3_GFS_v15_stochastics_init_cap()
  if (ierr/=0) return
end function FV3_GFS_v15_init_cap

Listing 5.4: Condensed version of the FV3_GFS_v15_init_cap function contained in the autogenerated file ccpp_FV3_GFS_v15_cap.F90 showing calls to the group caps FV3_GFS_v15_fast_physics_init_cap, FV3_GFS_v15_time_vary_init_cap , etc. for the static build where a group name is not specified.