Milestones

  1. initial steps
    1. port to nag (see below)
    2. ed on still runs (see: frankfurt.cgd.ucar.edu:/scratch/cluster/muszala/ptsChkEdOn/run) - RF to verify results
    3. compare clm4_5_19 points-run with EDMERGE+edOff / I_2000_CLM45_CN  -- BFB 
      [frankfurt.cgd.ucar.edu:/scratch/cluster/muszala]$>> /fs/cgd/csm/tools/cprnc_64/cprnc ptsChkClm4519/run/ptsChkClm4519.clm2.h0.0001-01-01-00000.nc ptsChkEdOff/run/ptsChkEdOff.clm2.h0.0001-01-01-00000.nc | tail
************************************************************************************************************************************

SUMMARY of cprnc:
 A total number of   2436 fields were compared
          of which      0 had non-zero differences
               and      0 had differences in fill patterns
 A total number of     12 fields could not be analyzed
 A total number of      0 fields on file 1 were not found on file2.
  diff_test: the two files seem to be IDENTICAL
    4. get a global res working (f10_f10.I_2000_CLM45_CN) - mpi-serial - 
      PASS  ERS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff
  PASS  ERS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff.memleak
    5. global res multiple processors 
      PASS  SMS_D_Mmpich.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff
  PASS  SMS_D_Mmpich.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff.memleak
  ::::::::::::::
  PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff
  PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOff.memleak
    6. EDMERGE ed-off mpi-serial bfb with clm4_5_19? done 
      PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555.memleak
FAIL  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555.generate.ed4519
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555.compare_hist.4519
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555.memcomp.4519
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.GC.162555.tputcomp.4519
----------------------------------------------------------------------------------
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.memleak
FAIL  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.generate.ed4519
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.compare_hist.4519
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.memcomp.4519
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.tputcomp.4519
    7. EDMERGE ed-off multiple pes bfb with clm4_5_19? done 
      PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519.memleak
FAIL  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519.generate.ed4519
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519.compare_hist.4519
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519.memcomp.4519
FAIL  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.GC.ed4519.tputcomp.4519
COMMENT   tput_decr = 12.92 tput_decr = 4.41 tput_percent_decr = 20.1 tput_percent_decr = 6.89
---------------------------------------------------------------------------
PASS  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519
PASS  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.memleak
FAIL  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.generate.ed4519
PASS  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.compare_hist.4519
PASS  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.memcomp.4519
PASS  ERS_D_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.GC.ed4519.tputcomp.4519
    8. get non-CN compsets working? done, see above
    9. EDMERGE ed-on global res mpi-serial satisfactory for now, retest when merged to clm4_5_35 
      PASS  SMS_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn
PASS  SMS_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn.memleak
----------------------------------------------------------------------
PASS  SMS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn
PASS  SMS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn.memleak
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f19_g16.I_2000_CLM45_CN.frankfurt_nag.edOn
PASS  SMS_Mmpi-serial.f19_g16.I_2000_CLM45_CN.frankfurt_nag.edOn.memleak
----------------------------------------------------------------------
FAIL  SMS_D_Mmpi-serial.f19_g16.I_2000_CLM45_CN.frankfurt_nag.edOn
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn
PASS  SMS_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn.memleak
----------------------------------------------------------------------
PASS  SMS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn
PASS  SMS_D_Mmpi-serial.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn.memleak
----------------------------------------------------------------------
FAIL  SMS_Mmpi-serial.f19_g16.I_2000_CLM45_CN.frankfurt_intel.edOn
----------------------------------------------------------------------
FAIL  SMS_D_Mmpi-serial.f19_g16.I_2000_CLM45_CN.frankfurt_intel.edOn
    10. EDMERGE ed-on global res multiple pes satisfactory for now, retest when merged to clm4_5_35 
      PASS  SMS_Mmpich.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn
PASS  SMS_Mmpich.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn.memleak
----------------------------------------------------------------------
FAIL  SMS_D_Mmpich.f10_f10.I_2000_CLM45_CN.frankfurt_nag.edOn
----------------------------------------------------------------------
PASS  SMS_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.edOn
PASS  SMS_Mmpich.f19_g16.I_2000_CLM45_CN.frankfurt_nag.edOn.memleak
----------------------------------------------------------------------
PASS  SMS.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn
PASS  SMS.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn.memleak
----------------------------------------------------------------------
PASS  SMS_D.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn
PASS  SMS_D.f10_f10.I_2000_CLM45_CN.frankfurt_intel.edOn.memleak
    11. run limited test suite (frankfurt_intel) against clm4_5_18 w/ED off...tests look OK. Enough to move onto merge. 
      >> ./cs.status.ed4519.frankfurt | grep -v tputcomp | grep -v COMMENT | grep -v generate | grep -v nlcomp
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.GC.ed4519
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.GC.ed4519.memleak
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.GC.ed4519
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.GC.ed4519.memleak
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.GC.ed4519
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.GC.ed4519.memleak
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.GC.ed4519
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.GC.ed4519.memleak
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  ERS_Lm3.1x1_numaIA.ICLM45CN.frankfurt_intel.GC.ed4519
PASS  ERS_Lm3.1x1_numaIA.ICLM45CN.frankfurt_intel.GC.ed4519.memleak
PASS  ERS_Lm3.1x1_numaIA.ICLM45CN.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERS_Lm3.1x1_numaIA.ICLM45CN.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  ERS_Lm3.1x1_numaIA.ICNCROP.frankfurt_intel.GC.ed4519
PASS  ERS_Lm3.1x1_numaIA.ICNCROP.frankfurt_intel.GC.ed4519.memleak
PASS  ERS_Lm3.1x1_numaIA.ICNCROP.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  ERS_Lm3.1x1_numaIA.ICNCROP.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  SMS.f10_f10.IRCP26CN.frankfurt_intel.GC.ed4519
PASS  SMS.f10_f10.IRCP26CN.frankfurt_intel.GC.ed4519.memleak
PASS  SMS.f10_f10.IRCP26CN.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  SMS.f10_f10.IRCP26CN.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  SMS.f10_f10.IRCP45CLM45CN.frankfurt_intel.GC.ed4519
PASS  SMS.f10_f10.IRCP45CLM45CN.frankfurt_intel.GC.ed4519.memleak
PASS  SMS.f10_f10.IRCP45CLM45CN.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  SMS.f10_f10.IRCP45CLM45CN.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  SMS_D.1x1_mexicocityMEX.I.frankfurt_intel.GC.ed4519
PASS  SMS_D.1x1_mexicocityMEX.I.frankfurt_intel.GC.ed4519.memleak
PASS  SMS_D.1x1_mexicocityMEX.I.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  SMS_D.1x1_mexicocityMEX.I.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.GC.ed4519
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.GC.ed4519.memleak
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.GC.ed4519.compare_hist.clm4_5_19
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.GC.ed4519.memcomp.clm4_5_19
    12. verify ED results due to changes from nag porting and initial work
  2. Merge to latest branch tag – initial checking/porting done.
    1. merge Branch up to clm4_5_35 trunk tag - verify ED results sill OK
      1. Check results with ED off 
        PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.C.ed4535
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.C.ed4535.memleak
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.C.ed4535.compare_hist.4535
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.C.ed4535.memcomp.4535
PASS  SMS_Ln10_D_Mmpi-serial.f10_f10.ICLM45.frankfurt_nag.C.ed4535.tputcomp.4535
---------------------------------------------------------------------------------
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.C.ed4535
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.C.ed4535.memleak
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.C.ed4535.compare_hist.4535
PASS  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.C.ed4535.memcomp.4535
FAIL  SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.C.ed4535.tputcomp.4535
COMMENT   tput_decr = 0.6620000 tput_percent_decr = 5.33
FAIL SMS_D_Ld5.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-default.nlcomp
---------------------------------------------------------------------------------
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.C.ed4535
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.C.ed4535.memleak
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.C.ed4535.compare_hist.4535
PASS  SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.C.ed4535.memcomp.4535
FAIL  SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.C.ed4535.tputcomp.4535
COMMENT   tput_decr = 0.484 tput_percent_decr = 10.4
FAIL SMS_Ld5_D_Mmpi-serial.f10_f10.ICLM45CNCROP.frankfurt_nag.nlcomp
---------------------------------------------------------------------------------
PASS  ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.C.ed4535
PASS  ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.C.ed4535.memleak
PASS  ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.C.ed4535.compare_hist.4535
PASS  ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.C.ed4535.memcomp.4535
PASS  ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.C.ed4535.tputcomp.4535
FAIL ERS_D_Mmpich.f19_g16.ICLM45CNCROP.frankfurt_nag.nlcomp
    2. merge up to clm4_5_36 - DONE
    3. test frankfurt_nag against clm4_5_36 w/ED off 
      >> ./cs.status.testEdNag.frankfurt | grep -v COMMENT | grep -v tputcomp | grep -v nlcomp
PASS  ERI_D_Mmpich.f19_g16.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  ERI_D_Mmpich.f19_g16.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERI_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  ERI_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERI_Mmpich.f19_g16.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  ERI_Mmpich.f19_g16.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_D_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  ERS_D_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_Ld42_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  ERS_Ld42_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_Lm3_Mmpich.1x1_numaIA.ICLM45BGCCROP.frankfurt_nag.GC.testEdNag
PASS  ERS_Lm3_Mmpich.1x1_numaIA.ICLM45BGCCROP.frankfurt_nag.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_Mmpi-serial.1x1_mexicocityMEX.I1PTCLM45.frankfurt_nag.clm-default.GC.testEdNag
PASS  ERS_Mmpi-serial.1x1_mexicocityMEX.I1PTCLM45.frankfurt_nag.clm-default.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_Mmpich.f10_f10.I1850CLM45BGC.frankfurt_nag.clm-ciso.GC.testEdNag
PASS  ERS_Mmpich.f10_f10.I1850CLM45BGC.frankfurt_nag.clm-ciso.GC.testEdNag.compare_hist.clm4_5_36
PASS  ERS_Mmpich.f10_f10.I20TRCLM45BGC.frankfurt_nag.clm-decStart.GC.testEdNag
PASS  ERS_Mmpich.f10_f10.I20TRCLM45BGC.frankfurt_nag.clm-decStart.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_D_Mmpi-serial.1x1_vancouverCAN.I1PTCLM45.frankfurt_nag.clm-default.GC.testEdNag
PASS  SMS_D_Mmpi-serial.1x1_vancouverCAN.I1PTCLM45.frankfurt_nag.clm-default.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_D_Mmpich.1x1_mexicocityMEX.I.frankfurt_nag.GC.testEdNag
PASS  SMS_D_Mmpich.1x1_mexicocityMEX.I.frankfurt_nag.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_Ln48_D_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-oldhyd.GC.testEdNag
PASS  SMS_Ln48_D_Mmpich.f10_f10.ICLM45BGC.frankfurt_nag.clm-oldhyd.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_Ly1_Mmpich.f19_g16.ICLM45BGCCROP.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  SMS_Ly1_Mmpich.f19_g16.ICLM45BGCCROP.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_Mmpich.1x1_mexicocityMEX.I1PT.frankfurt_nag.GC.testEdNag
PASS  SMS_Mmpich.1x1_mexicocityMEX.I1PT.frankfurt_nag.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_Mmpich.f10_f10.IRCP26CN.frankfurt_nag.clm-reduceOutput.GC.testEdNag
PASS  SMS_Mmpich.f10_f10.IRCP26CN.frankfurt_nag.clm-reduceOutput.GC.testEdNag.compare_hist.clm4_5_36
PASS  SMS_RLA_Mmpi-serial.f45_f45.ICLM45.frankfurt_nag.clm-default.GC.testEdNag
PASS  SMS_RLA_Mmpi-serial.f45_f45.ICLM45.frankfurt_nag.clm-default.GC.testEdNag.compare_hist.clm4_5_36
FAIL  SMS_RLA_Mmpich.f45_f45.ICLM45.frankfurt_nag.clm-default.GC.testEdNag
FAIL  SMS_RLA_Mmpich.f45_f45.ICLM45.frankfurt_nag.clm-default.GC.testEdNag.compare_hist.clm4_5_36
    4. test frankfurt_intel against clm4_5_36 w/ED off 
      PASS  ERI.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERI.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI_D.T31_g37.I1850.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI_D.T31_g37.I1850CLM45.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERI_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERI_D.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERI_D.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERI_D.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERI_D.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERI_D.f19_g16.ICLM45BGC.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.clm-default.GC.testEdIntel
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.clm-default.GC.testEdIntel.memleak
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.clm-default.GC.testEdIntel.generate.ed4536
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.clm-default.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS.f10_f10.I1850CLM45BGC.frankfurt_intel.clm-default.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOff.GC.testEdIntel
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOff.GC.testEdIntel.memleak
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOff.GC.testEdIntel.generate.ed4536
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOff.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOff.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnFloodOnEffvelOn.GC.testEdIntel
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnFloodOnEffvelOn.GC.testEdIntel.memleak
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnFloodOnEffvelOn.GC.testEdIntel.generate.ed4536
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnFloodOnEffvelOn.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnFloodOnEffvelOn.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnIceOff.GC.testEdIntel
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnIceOff.GC.testEdIntel.memleak
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnIceOff.GC.testEdIntel.generate.ed4536
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnIceOff.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS.f19_g16.I1850CLM45BGC.frankfurt_intel.rtm-rtmOnIceOff.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.clm-reduceOutput.GC.testEdIntel
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memleak
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.generate.ed4536
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS.f45_g37.I1850CN.frankfurt_intel.clm-reduceOutput.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS_D.f10_f10.ICLM45.frankfurt_intel.clm-decStart.GC.testEdIntel
PASS  ERS_D.f10_f10.ICLM45.frankfurt_intel.clm-decStart.GC.testEdIntel.memleak
PASS  ERS_D.f10_f10.ICLM45.frankfurt_intel.clm-decStart.GC.testEdIntel.generate.ed4536
PASS  ERS_D.f10_f10.ICLM45.frankfurt_intel.clm-decStart.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS_D.f10_f10.ICLM45.frankfurt_intel.clm-decStart.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-NoVSNoNI.GC.testEdIntel
PASS  ERS_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-NoVSNoNI.GC.testEdIntel.memleak
PASS  ERS_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-NoVSNoNI.GC.testEdIntel.generate.ed4536
PASS  ERS_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-NoVSNoNI.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS_D.f10_f10.ICLM45BGC.frankfurt_intel.clm-NoVSNoNI.GC.testEdIntel.memcomp.clm4_5_36
PASS  ERS_Ly20.1x1_numaIA.ICLM45BGCDVCROP.frankfurt_intel.clm-crop.GC.testEdIntel
PASS  ERS_Ly20.1x1_numaIA.ICLM45BGCDVCROP.frankfurt_intel.clm-crop.GC.testEdIntel.memleak
PASS  ERS_Ly20.1x1_numaIA.ICLM45BGCDVCROP.frankfurt_intel.clm-crop.GC.testEdIntel.generate.ed4536
PASS  ERS_Ly20.1x1_numaIA.ICLM45BGCDVCROP.frankfurt_intel.clm-crop.GC.testEdIntel.compare_hist.clm4_5_36
PASS  ERS_Ly20.1x1_numaIA.ICLM45BGCDVCROP.frankfurt_intel.clm-crop.GC.testEdIntel.memcomp.clm4_5_36
PASS  PEM.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-crop.GC.testEdIntel
PASS  PEM.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-crop.GC.testEdIntel.generate.ed4536
PASS  PEM.f10_f10.ICLM45BGCCROP.frankfurt_intel.clm-crop.GC.testEdIntel.compare_hist.clm4_5_36
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.clm-default.GC.testEdIntel
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.clm-default.GC.testEdIntel.memleak
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.clm-default.GC.testEdIntel.generate.ed4536
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.clm-default.GC.testEdIntel.compare_hist.clm4_5_36
PASS  SMS_D.1x1_mexicocityMEX.ICLM45.frankfurt_intel.clm-default.GC.testEdIntel.memcomp.clm4_5_36
    5. EDMERGE ed-on global res mpi-serial merged to clm4_5_36 BGC not relevant but used for testing – 
      ----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f10_f10.ICLM45BGCED.frankfurt_nag.093354
PASS  SMS_Mmpi-serial.f10_f10.ICLM45BGCED.frankfurt_nag.093354.memleak
----------------------------------------------------------------------
PASS  SMS_D_Mmpi-serial.f10_f10.ICLM45BGCED.frankfurt_nag.101559
PASS  SMS_D_Mmpi-serial.f10_f10.ICLM45BGCED.frankfurt_nag.101559.memleak
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_nag.094013
PASS  SMS_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_nag.094013.memleak
----------------------------------------------------------------------
FAIL  SMS_D_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_nag.154601
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_pgi.094824
PASS  SMS_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_pgi.094824.memleak
----------------------------------------------------------------------
PASS  SMS_D_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_pgi.082458
PASS  SMS_D_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_pgi.082458.memleak
---------------------------------------------=-------------------------
FAIL  SMS_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_intel.103218
----------------------------------------------------------------------
FAIL  SMS_D_Mmpi-serial.f19_g16.ICLM45BGCED.frankfurt_intel.082413
----------------------------------------------------------------------
    6. EDMERGE ed-on global res multiple pes merged to clm4_5_36 BGC not relevant but used for testing 
      ----------------------------------------------------------------------
FAIL  SMS_D_Mmpich.f10_f10.ICLM45BGCED.frankfurt_nag.104736
----------------------------------------------------------------------
FAIL  SMS_Mmpich.f19_g16.ICLM45BGCED.frankfurt_nag.095843
-- I_2000_CLM45_CN passed earlier --
----------------------------------------------------------------------
    7. test CN resolutions:
    8. EDMERGE ed-on global res single proc merged to clm4_5_36 
      ----------------------------------------------------------------------
FAIL  SMS_D_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_intel.104738
----------------------------------------------------------------------
FAIL  SMS_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_intel.103748
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_nag.131519
PASS  SMS_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_nag.131519.memleak
----------------------------------------------------------------------
PASS  SMS_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_pgi.131518
PASS  SMS_Mmpi-serial.f19_g16.ICLM45CNED.frankfurt_pgi.131518.memleak
----------------------------------------------------------------------
    9. EDMERGE ed-on global res multi- proc merged to clm4_5_36 
      ----------------------------------------------------------------------
PASS  SMS_D.f10_f10.ICLM45CNED.frankfurt_pgi.135620
PASS  SMS_D.f10_f10.ICLM45CNED.frankfurt_pgi.135620.memleak
----------------------------------------------------------------------
PASS  SMS_D_Mmpich.f10_f10.ICLM45CNED.frankfurt_nag.142838
PASS  SMS_D_Mmpich.f10_f10.ICLM45CNED.frankfurt_nag.142838.memleak
----------------------------------------------------------------------
PASS  SMS_D.f10_f10.ICLM45CNED.frankfurt_intel.135652
PASS  SMS_D.f10_f10.ICLM45CNED.frankfurt_intel.135652.memleak
----------------------------------------------------------------------
PASS  SMS_Mmpich.f19_g16.ICLM45CNED.frankfurt_nag.clm-reduceOutput.145244
PASS  SMS_Mmpich.f19_g16.ICLM45CNED.frankfurt_nag.clm-reduceOutput.145244.memleak
----------------------------------------------------------------------
  3. infrastructure
    1. add relevant namelists
      1. use_ed and use_ed_spit_fire are now triggered by an ED compset
      2. new ED option for compsets.
      3. make sure that BGC and CROP can't be combined with ED and that RTM is OFF. not done yet
    1. restarts working with ED on
    2. pull paramters out of text file and put into F90 module.
      1. Rosie created a new netcdf file with all params - /fs/cgd/csm/inputdata/lnd/clm2/edParams/CLMPARAMS_ED_1.nc

nag port notes

  • port to nag -
    1. as of Sept. 11 - Nag works with Rosies's original single pts. case.
    2. Port mods introduce non-BFB in GPP, NPP, PFTbiomass and PFTleafbiomass. Some selected mods. below:
      1. Nag was evaluating the entire line and i-1 was causing an out of bounds error when i was equal to 1.
        ED_Mod.F90::canopy_structure
        @@ -1696,21 +1689,23 @@
         write(*,*) 'problem with canopy area', checkarea,cp%area,i
       endif

-      if(i.gt.1.and.arealayer(i).gt.arealayer(i-1))then
+         if ( i .gt. 1) then
+            if ( arealayer(i) .gt. arealayer(i-1) ) then
       	        write(*,*) 'smaller top layer than bottom layer',arealayer(i),arealayer(i-1),cp%area,cp%spread(i-1:i)
             endif
+         endif
     enddo !
      2. How did this work with intel? The use of cp%clm_pno was in place before it was properly binned.
        ED_Mod.F90::clm_ed_link
        total_area = 0._r8
     cp => cs%oldest_patch
-    c = pcolumn(cp%clm_pno)
-    litterc(c) = 0.0_r8
     do while(associated(cp))
        patchn = patchn + 1
        cp%patchno = patchn
@@ -4557,6 +4540,8 @@
        if(patchn.le.numpft - numcft)then !don't expand into crop patches.
           cp%clm_pno = firstsoilpatch(g) + patchn !put it in the second bin, the first is unveg...
           p = cp%clm_pno
+          c = pcolumn(cp%clm_pno)
+          litterc(c) = 0.0_r8
           call root_fraction(cp)
      3. refactor interface for clm_indices. Gets rid of one pointer assignment
        ED_Mod.F90::cindices
        @@ -5441,30 +5424,32 @@
   use shr_kind_mod, only: r8 => shr_kind_r8;
   implicit none

-  type(site), target :: cs_pnt
-  type(site), pointer :: cs
+  type(site), intent(inout) :: cs_pnt
   type(patch), pointer :: cp

   integer i,n,c,t
-  real(r8)  moist !effective fuel moisture
-  real(r8) tau_b(nfsc) !lethal heating rates for each fuel class (min)
-  real(r8) ::  fc_ground(nfsc) !propn of fuel consumed
+  real(r8) :: moist !effective fuel moisture
+  real(r8) :: tau_b(NFSC) !lethal heating rates for each fuel class (min)
+  real(r8) :: fc_ground(NFSC) !propn of fuel consumed

-  cs => cs_pnt
-  cp => cs%oldest_patch;
+  cp => cs_pnt%oldest_patch;
   do while(associated(cp))
     cp%burnt_frac_litter = 1.0_r8
     FC_ground = 0.0_r8

-!---calculate fraction of fuel burnt for each size class.---
+    !---calculate fraction of fuel burnt for each size class.---
     do c = 1,NFSC    !work out the burnt fraction for all pools, even if those pools dont exist.

-      if(c.lt.6.and.cp%fuel_moisture(c).ne.999)then !screen out pools which dont exist and live tissues.
+      if ( ( c .lt. 6) ) then
+         if ( cp%fuel_moisture( c ) .ne. 999 ) then !screen out pools which dont exist and live tissues.
+            ! note, cp%fuel_moisture only as 5 elements
 	moist = cp%fuel_moisture(c)/udata%MEF(c)
+         endif
  • run with f10_f10 (make sure some low resolution works with mpi-serial, then mpi)
    1. not working at all with more than 1 processor.
    • one culprit is in history/restart implementation - comment these out for now, don't worry about restarts
  • verify that science is correct for some basic cases

stuff to keep in mind -

  • get ED infrastructure up to current standards
  • I/0
    • proper restart capability
    • proper history file output
  • refactor parameter handling
  • verify that ED works with openMP
  • verify that science is correct multiple times

final clean-up and refactor

  • clean-up interfaces
  • clean-up associate blocks
  • indentation check
  • review comments

Software Engineering Issues

  • CN code called even when non-CN compset used

Science Issues

  • energy balance errors
  • couple ED to BGC and CN ?
  • resolve competing litter class definition issues (model needs different pools for burning and decomposing litter)
  • Check units and timesteps for reporting of GPP and NPP fluxes from ED. 

Report on running n11_clm4_5_19 (RF, 9/Oct/13)

CONFIG: create_newcase -case f10_multproc -res f10_f10 -mach yellowstone -compset I_2000_CLM45_CN -confopts _D_mpich2

CODE UPDATES:

1. Adjusted the divide by zero error in CanopyFluxesMod.F90. L3780 

-    if(GCcanopy(p).gt.1._r8/rsmax0)then
+    if(GCcanopy(p).gt.1._r8/rsmax0.and.elai(p).gt.0.0_r8)then
-    if(GCcanopy(p).gt.1._r8/rsmax0)then

+    if(GCcanopy(p).gt.1._r8/rsmax0.and.elai(p).gt.0.0_r8)then

2. Added a catch for the DAYRH calculations in clm_driver.F90 L917. 

+ if(cs%dayrh.lt.0.1_r8)then

+            write(,) 'very low dayrh',g,cs%lat,cs%lon,cs%dayrh,clm_a2l%forc_rh(g)

+    cs%dayrh = 0.1_r8

+         endif

3. Turned off the solar balance check in BalanceMod to check for bigger problems (to be debugged).

SCIENCE OBSERVATIONS

- f10 takes 1-2 minutes per month on 15 processors with debug ON. 

- LAI, NPP and GPP appear to be converging reasonably, bar any strange long term impacts. LAI is plateauing because I have turned off the optimizing dynamics. These need to be readded to constitute a proper mechanistic test. 

- There are large crashes in LITTERC that are not precipitated by fire.

- AREA_PLANT appears to enter a slow decline after canopy closure. Is this a recruitment issue?

BALANCE ISSUES

There are currently major problems with radiation balance and carbon balance that are the highest coding priority. 

Issues with the  EDMERGE branch, (as of 14 oct 2013) 

The updated tag (merged to clm4_5_36?) needs a new 'CLM_PARAMS' file.  The script for converting the existing file into the ED specific file is in

/glade/u/home/rfisher/Matlab/pft_files/clm_params_ED.m

and the files are in 

/glade/u/home/rfisher/inputdata/lnd/clm2/pftdata/CLMPARAMS_ED_1.nc

In addition to the change in the parameter file, the new version of CLM has altered comp sets. I want a compset that has

y2000 climate data. CLM4.5 and CN on, ideally no crops or RTM. (i.e. like I_2000_CLM45_CN in the old version). 

The closest match I found was: ICRUCLM45BGC.


This case 

♦create_newcase -case f10_bgc -res f10_f10 -mach yellowstone -compset ICRUCLM45BGC -confopts _D_mpich2

-Runs v.e.r.y. s.l.o.w.l.y. as in, it takes an hour per year. (crashes after 2640 days)

-The LAI is too small. Everything is going too slowly 

whereas this one, in the old version

♦create_newcase -case f10_multproc -res f10_f10 -mach yellowstone -compset I_2000_CLM45_CN -confopts _D_mpich2

-Runs 3 times quicker (10-15 mins per month).  (crashes after 8 years (2783 days), except on Oct 9th it ran all the way to the end?) 

-Science looks to be sensible. 

If this is caused by something to do with BGC or CROP, we can try this compset

♦create_newcase -case f10_mult_CNTEST -res f10_f10 -mach yellowstone -compset ICLM45CNTEST -confopts _D_mpich2

-This runs but v.e.r.y. s.l.o.w.l.y. as well, and with not enough LAI.


Thus, there are two main problems to solve. 

1) The not-enough LAI problem

2) The slow running problem

For 1), we will assume that it is a problem in the new parameter file.

Looking through, it seems that the second PFT is not operating as C4, which will have reduced the psn. I also harmonized a whole other set of parameter values... 

-Runs slowly. The LAI is higher, but not as high as in the f10_bgc runs after 1 year. (high of 0.4 compared to 0.7)


For 2), a key test is to see whether it just generally runs slowly or if ED is the cause. For  this, I have created a test with no extraneous options except for 4.5 and CN: 

♦create_newcase -case f10_1850_45CN -res f10_f10 -mach yellowstone -compset I1850CLM45CN -confopts _D_mpich2

Except, it turns out that all the runs in /EDMERGE/ED_branch/EDMERGE/ have been with ED off ;) 

With ED off this case runs: slowly. about 5 minutes per month... :/ 

With ED on this case runs: not at all... 


There is now another problem 3) which is that the model doesn't run in any of the configurations I have tested it with ED on. 

The issues seem to be to do with processor bounds:

-If I start it in multiprocessor mode, it crashes at the ED_patch statement in CNsetvalues. ED_patch is never set prior to this call, which is probably why. Although it seems to be an issue with the size of the vector, rather than with the content of it...  Setting it to zero in clmtypeinit doesn't actually fix the problem either?

-If I start it in serial mode in totalview (f10_serial), it crashes in get_clump_bounds before the ed_init call. The wrong value of 'nc' is being passed to bounds_clump, and it looks like ED_INIT should in fact be in a loop of:

do nc=1, nclumps

  call get_clump_bounds(nc,bounds_clump)

  call ed_init(bounds_clump)

enddo

Either that, or it should be called with bounds_proc instead of bounds_clump. I don't have a clear idea of how these differ. Is this the issue with 'decomposing' the data structure?  


  • No labels