The best way to test dart (right now) is to run the developer_tests/test_dart.csh script. The data files needed (large model files, original observation data) are contained in

/glade/p/cisl/dares/DART_test_cases/test_build_datasets.tar

/glade/p/cisl/dares/DART_test_cases/test_build_datasets_preprocess.tar has input.nmls with quantity files. 

  • Simply untar the file in your DART repository and the files should expand into the right place.
  • Make sure you have a valid   build_templates/mkmf.template  and submit or execute test_dart.csh.
  • If you run this without a queueing system, be sure to capture the run-time output for comparison to a known standard (which we don't have yet). Know that at least one of the observation converters (GSI2DART) requires MPI and must run with at least 3 tasks or it will fail (which is correct behavior, surprisingly).

The test at present simply tries to compile as many things as possible and runs what it can. If the model has a workshop.csh, it is run - if it does not, pmo and possibly model_mod_check are run. The observation converters are compiled all executables in those directories are run. Many (most?) of the observation converters do not have input datasets, consequently - they fail to run. I am trying to get test datasets for each of the converters, which also inspires me to update the observation README.[md,rst] file. Be aware that the results are not checked in any way at the moment and the results might not be correct.  With -O0 optimization, the test takes about an hour to run on one cheyenne node. The vast majority of the time is spent compiling, so it is counterproductive to compile with higher levels of optimization unless you are testing something specific.

The /glade/p/cisl/dares/DART_test_cases/bundle_test_data.sh  script can be modified to add new data files. Be sure to rename the existing tar file before creating a new one that has the additional content.

Cleaning Up After a Test

Since the test suite creates directories, unpacks a number of input datasets, and builds executables, cleaning up after the tests is complex. You can run a script created by Nancy: 

/glade/p/cisl/dares/DART_test_cases/clean_test_build_datasets.sh

Tim calls clean_test_build_datasets.sh from an additional script, that also removes executables: 

/glade/u/home/thoar/scripts/cleanup_test.csh

This script, cleanup_test.csh, also removes directories created by test_dart.csh in which input data are stored for the tests. These directories are not needed after the tests complete.

 \rm -rf observations/obs_converters/CONAGUA/work/example_data

 \rm -rf observations/obs_converters/GOES/data

Removing Executables

In order to run cleanup_test.csh, you must define an alias, findx, that finds and lists compiled executables. findx differentiates compiled executables from scripts that have execute permission.

This alias is operating-system-dependent and Tim has specified it both for MacOS and Linux.

Linux

alias findx "find . -type f -print | grep -v '\.svn' | xargs file | grep 'ELF 64-bit LSB executable' | sed -e 's/: .*// '"

Mac OS

alias findx "find . -type f -print | grep -v '\.svn' | xargs file | grep 'Mach-O 64-bit executable' | sed -e 's/: .*//'"

Please help improve this page.


0[1955] cheyenne5:/<3>dares/DART_test_cases > ls -lrt
<snip>
-rwxrwxr-x  1 nancy p86850054        669 Aug 22  2019 clean_test_build_datasets.sh
-rwxrwxr-x  1 thoar dart            8765 Oct 19 08:48 bundle_test_data.sh
drwxrwxr-x  2 nancy p86850054       4096 Oct 19 08:49 old_test_dataset_tarfiles
-rw-r--r--  1 thoar p86850054 4123054080 Oct 19 08:50 test_build_datasets.tar
0[1956] cheyenne5:/<3>dares/DART_test_cases > ls -lt old_test_dataset_tarfiles/
total 43634240
-rw-r--r-- 1 thoar p86850054 4123054080 Aug 26 09:03 test_build_datasets.tar.26Aug2020
-rw-r--r-- 1 thoar p86850054 3868026880 May 29 14:28 test_build_datasets.tar.29May2020
-rw-r--r-- 1 15717 ncar      3868026880 Mar 17  2020 test_build_datasets.tar.17Mar2020
-rw-rw-r-- 1 thoar dart      3868026880 Jan 27  2020 test_build_datasets.tar.27Jan2020
-rw-rw-r-- 1 thoar dart      3843461120 Dec 17  2019 test_build_datasets.tar.17Dec2019
-rw-rw-r-- 1 thoar dart      3837655040 Nov 26  2019 test_build_datasets.tar.26Nov2019
-rw-rw-r-- 1 thoar dart      3734722560 Nov  6  2019 test_build_datasets.tar.06Nov2019
-rw-rw-r-- 1 thoar dart      3734702080 Oct 28  2019 test_build_datasets.tar.28Oct2019
-rw-rw-r-- 1 thoar dart      3734609920 Oct 11  2019 test_build_datasets.tar.18Oct2019
-rw-rw-r-- 1 thoar dart      3679436800 Oct  3  2019 test_build_datasets.tar.03Oct2019
-rw-rw-r-- 1 thoar dart      2132705280 Aug 22  2019 test_build_datasets.tar.22Aug2019
-rw-rw-r-- 1 nancy p86850054 2132695040 Apr  3  2019 test_build_datasets.tar.old.V2
-rw-rw-r-- 1 nancy dart      2124247040 Jan  8  2018 test_build_datasets.tar.old.V1

  • No labels