This example describes how to to run CAM-chem out-of-the-box with specified dynamics on the NCAR cheyenne HPC. More information on running CESM is available from the CESM Tutorials (e.g., http://www.cesm.ucar.edu/events/tutorials/2019/coursework.html). However, this page includes specific instructions for chemistry simulations. The compset required for this setup is named “FCSD”.
|
Downloading the source code will be completed by using the git clone feature. For example, first download the latest release (e.g. CESM2.2.0, with recent bug fixes):
> cd /glade/work/$USER/ > git clone https://github.com/ESCOMP/CESM.git {my_cesm_sandbox} > cd {my_cesm_sandbox} > git fetch origin > git checkout cesm2.2-rcbranch |
It is recommended to use a descriptive label, including the model version and perhaps the current date, for your local directory instead of "my_cesm_sandbox".
Download the model component source code.
> ./manage_externals/checkout_externals |
And all the code to continue should be in the new sandbox folder. To use previous/other versions or to address errors, see instructions on the CESM page linked above.
2. Use the code in the model directory $CESM_ROOT
to create a new case called $CASENAME
:
> $CESM_ROOT/cime/scripts/create_newcase --case <your_path/$CASENAME> --mach cheyenne --res f09_f09_mg17 --compset FCnudged |
From section A, $CESM_ROOT
would be /glade/work/$USER/my_cesm_sandbox
|
3. From within $CASEROOT
>./case.setup |
|
5. Compile and build the model in $CASEROOT
First address any current bug-fixes: Bugs and Updates |
>qcmd -- ./case.build Note: you cannot run ./case.build interactively from the cheyenne prompt because it uses too much memory: you must use 'qcmd'. |
Note 1: You may need a project number to run qcmd: qcmd -A $PROJECT_NUMBER -- ./case.build
6. (Optional) Make changes to the model runtime setup: Changes to env_run.xml
can be made at any time:
a. If not starting in the default 2005, change dates: see Changing Dates of a Run (also see relevant namelist changes)
b. The default option for biogeochemistry is “specified phenology” (satellite LAI) with CLM5 physics:
<entry id="CLM_BLDNML_OPTS" value="-bgc sp">
<entry id="CLM_PHYSICS_VERSION" value="clm5_0">
Other options are available, such as irrigated crops - please see the CLM documentation.
c. The default simulation is a test run for 5 days. Change STOP_OPTION
and STOP_N
to alter this to desired values.
|
8. Check the run setup. In the env_batch.xml
file make sure to have your project added correctly: <entry id="PROJECT" value=$YOUR_PROJECT_CODE>
Depending on the version of CESM, you may instead find the entry id for PROJECT in env_workflow.xml
Change your walltime if desired: <entry id="JOB_WALLCLOCK_TIME" value="12:00:00"> (12 hours max)
9. If you make changes to env_build.xml
variables or SourceMods after setting up and building, you may have to clean your setup and build again:
>./case.setup --clean >./case.setup |
OR
>./case.setup --reset |
(which cleans, then does setup)
followed by
>./case.build --clean >qcmd -- ./case.build |
10. Submit run to queue:
>./case.submit |
While running, output is written to <run_dir>: /glade/scratch/<username>/$CASENAME/run
|
11. Useful commands while model is running:
check your run progress:
>qstat –u <user> |
If you find an issue and need to delete your run:
>qdel <JobID |
If your model run doesn't complete, try some of the suggestions on the troubleshooting page.
12. To continue a run from restart files, for example after an initial start up, change CONTINUE_RUN to TRUE
in the env_run.xml
file.
High performance computing systems often have maximum wall times (e.g. JOB_WALLCLOCK_TIME = 12 hours), meaning a long run will need to be split into several smaller runs. In this case, change the “RESUBMIT” value in env_run.xml file to greater than zero. For example, you can simulate 10 years by changing STOP_OPTION=nyears, STOP_N=1 and RESUBMIT=9. This will perform an initial run of 1 years + (9 resubmits x 1 years per job) = 10 years. |