Child pages
  • Run CAM-Chem on Cheyenne
Skip to end of metadata
Go to start of metadata

This example describes how to to run CAM-chem out-of-the-box with specified dynamics on the NCAR cheyenne HPC. The steps follow closely with the CESM tutorial introductory lecture (see Creating & Running a Case about halfway through these lecture notes, or watch the whole recording). However, this page includes specific instructions for chemistry simulations. The compset required for this setup is named “FCSD”.

  • Get a Cheyenne account for the NCAR High performance Computer system if you don’t have one.

(A) Create a new case

1. Go to the model directory $CESM_ROOT and create a new case called $CASENAME:

$CESM_ROOT = /glade/p/cesm/releases/cesm2_0

>cd $CESM_ROOT/cime/scripts
>./create_newcase --case <your_path+$CASENAME> --mach cheyenne --res f09_f09_mg17 --compset FCSD


You may need to add --run_unsupported to the call line if you are not running with a scientifically validated compset and resolution combination.

Note 1: For additional help and options, type ./create_newcase -h 

Note 2: To find the possible compset types $CESM_ROOT/cime/scripts/query_config --compsets cam. 

A new directory $CASEROOT = <your_path+$CASENAME> is created

Tip for New Users:

Information on the compsets can be found here: http://www.cesm.ucar.edu/models/cesm2/config/compsets.html

(B) Set up your case

2. From within $CASEROOT

>./case.setup 


3. Make changes to the model configuration using the *.xml files; you can edit the files directly or use the xmlchange tool (in your case directory).

Changes to env_build.xml must be made before building, or you will need to re-build.

See advanced pages to change the chemistry source code.

(C) Build the Executable

Tip for New Users:

You can use script xmlquery to query a variable in the xml files before modifying a variable with xmlchange command. For example:

>./xmlquery CALENDAR

4. Compile and build the model in $CASEROOT

>qcmd -- ./case.build

5. Make changes to the model runtime setup: Changes to env_run.xml can be made at any time:

If not starting in the default 2005, change date of RUN_START: see Changing Dates of a Run (also see namelist changes)

Biogeochemistry (bgc) option

<entry id="CLM_BLDNML_OPTS" value="-bgc bgc -crop -irrig=.true."> (Note The -irrig option can ONLY be used with clm4_0 physics)

<entry id="CLM_CONFIG_OPTS" value="-phys clm4_0">

or, use default for “specified phenology” (satellite LAI):

<entry id="CLM_BLDNML_OPTS" value="-bgc sp">

The default is a test run -- 5 days.

Change STOP_OPTION  and STOP_N to desired values.

Check the SST ice file covers the dates you require:

<entry id="SSTICE_DATA_FILENAME" value="$DIN_LOC_ROOT/atm/cam/sst/sst_HadOIBl_bc_0.9x1.25_1850_2016_c170525.nc">

<entry id="SSTICE_YEAR_END" value="2016">

6. Check namelist settings in the namelist files user_nl_cam and user_nl_clm. Most CAM-chem related namelist variables are in CaseDocs/atm_in, but MEGAN and drydep are in CaseDocs/drv_flds_in (these files are created during build).  To modify any of these, copy the appropriate lines to user_nl_cam and edit there.

If RUN_STARTDATE has been changed, you have to also change the date of the initial meteorology file in user_nl_cam to start at the corresponding date.

For other changes see namelist changes or advanced options page.

Tip for New Users:

There are many namelist variables. You can find their definitions at: https://www.cesm.ucar.edu/models/cesm2/component_settings/cam_nml.html

Optionally run:

>./preview_namelists

after changing any user_nl_* files. NOTE: most changes in user_nl_* files do not require re-building. However, during a run (CONTINUE_RUN = TRUE) no changes can be made to history output (fincl lists). If you want to change history output, create a new or branch run.

7. Check the run setup. In the env_batch.xml file make sure to have your project added correctly:  <entry id="PROJECT" value=$YOUR_PROJECT_CODE>

Change your walltime if desired: <entry id="JOB_WALLCLOCK_TIME" value="12:00:00"> (12 hours max)

8. If you make changes to env_build.xml variables or SourceMods after setting up and building, you may have to clean your setup and build again:

>./case.setup --clean
>./case.setup

OR

>./case.setup --reset

(which cleans, then does setup)

followed by

>./case.build --clean
>qcmd -- ./case.build

(D) Run the Model

9. Submit run to queue:

>./case.submit

While running, output is written to <run_dir>: /glade/scratch/<username>/$CASENAME/run

After the run completes successfully, output files are moved to the short term archive: /glade/scratch/<username>/archive/$CASENAME/atm/hist (similar directories exist for other model components: lnd, etc.)

Restart and initial conditions files are written to: /glade/scratch/<username>/archive/$CASENAME/rest

Note: long term archiving is currently not working

10. Useful commands while model is running:

check your run progress: 

>qstat –u <user>

If you find an issue and need to delete your run:

>qdel <JobID

If your model run doesn't complete, try some of the suggestions on the troubleshooting page.

11. To continue a run from restart files, for example after an initial start up, change CONTINUE_RUN to TRUE in the env_run.xml file.

Tip for New Users:

High performance computing systems often have maximum wall times (e.g. JOB_WALLCLOCK_TIME = 12 hours), meaning a long run will need to be split into several smaller runs. In this case, change the “RESUBMIT” value in env_run.xml file to greater than zero. For example, you can simulate 10 years by changing STOP_OPTION=nyears, STOP_N=1 and RESUBMIT=9. This will perform an initial run of 1 years + (9 resubmits x 1 years per job) = 10 years.





  • No labels