Page History

...

Documentation: https://github.com/JCSDA-internal/ewok/blob/develop/README.md

EWOK Developer Section

EWOK currently uses the ECMWSF's ecFlow software system for implementing workflows. This section will describe how EWOK and ecFlow interact to run a Skylab experiment. Instructions for setting up your environment to can be found in the JEDI Documentation.

...

As part of EWOK's set up, you will notice two variables that pertain to ecFlow which are needed to run an experiment. They are EWOK_WORKDIR and EWOK_FLOWDIR. The EWOK_WORKDIR is where all of your experiment files will be saved that are generated by the workflow - such as feedback files, background files, observations, and your forecasts. The EWOK_FLOWDIR will contain configuration files and the runtime files that get executed. Tip: for testing small on the fly changes, after kicking off an experiment you can edit the runtime files in EWOK_FLOWDIR and then restart the task. Although the runtime file in the EWOK repository will not be updated, this method is useful if you need to force something to work or if you want to troubleshoot without touching the repo. 

Setting up Cylc

Cylc has many python dependencies, which means it clashes with our ewok-env. Therefore a work around is to install it inside its own virtual environment on your local machine. Instructions are also included for using Discover’s cylc installation. This works with spack-stack 1.9.x modules and GMAO’s SWELL application. 

Cylc Version 8:

• Python requirements: cylc-flow python3.8+; cylc-uiserver python3.8-3.9
• Documentation: Cylc Documentation and Swell Documentation
• Github: Cylc repositories and swell 

Mac via venv:

1. Needs python3.9 for the UI, therefore run: 
   

   Code Block
   language shell
   brew install python@3.9

2. Update PYTHONPATH:
   

   Code Block
   language shell
   module purge unset PYTHONPATH unset PYTHONHOME

3. Created venv without spack-stack:
   

   Code Block
   language shell
   python3.9 -m venv --system-site-packages cylc-venv

4. Activate venv:
   

   Code Block
   language shell
   source cylc-venv/bin/activate

5. Install cylc:
   

   Code Block
   language shell
   pip install cylc-flow pip install cylc-uiserver pip install cylc-rose metomi-rose

6. Install optional:
   

   Code Block
   language shell
   pip install 'cylc-flow[tutorial]' pip install 'cylc-uiserver[hub]'

7. Graphviz:
   

   Code Block
   language shell
   brew install graphviz

8. To test the GUI:
   

   Code Block
   language shell
   cylc gui

Discover via spack-stack:

1. Load spack-stack modules
   

   Code Block
   language shell
   #!/usr/bin/env bash # Initialize modules source $MODULESHOME/init/bash # Load python dependencies echo "Using SLES15 modules" module use /discover/swdev/jcsda/spack-stack/scu17/modulefiles module use /gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-1.9.0/envs/ue-intel-2021.10.0/install/modulefiles/Core module load stack-intel/2021.10.0 module load stack-intel-oneapi-mpi/2021.10.0 module load stack-python/3.11.7 module load py-pip/23.1.2

   
   
2. Load cylc module and test
   

   Code Block
   language shell
   # Load the cylc module module use -a /discover/nobackup/projects/gmao/advda/swell/dev/modulefiles/core/ module load cylc/sles15_8.4.0 # Run cylc command cylc "$@"

   
   
3. You might need to create a file called $HOME/bin/cylc, and make sure it is executable in order to run locally: chmod +x $HOME/bin/cylc
1. Note, I did not have to do this when setting up and running on discover-mil with spack-stack 1.9.0 intel

FAQ

How to remove an experiment from ecflow?

...

While the experiment is running, right-click on the task in ecflow UI, click output, pick the file to see (there would also be a path to that file). In some cases (variational experiments? others?) the stdout/stderr logs can be found in the path: ${EWOK_WORKDIR}/<exp_id>/<date>/. After the experiment has completed, the finishExperiment task will have cleaned up many of these logs. In most cases the yamls, jobs and logs for the latest cycle can still be found in ${EWOK_FLOWDIR}/<exp_id>. To prevent this cleanup of the ewok dir, suspend the finishExperiment task via the GUI or the command line after starting the experiment.

...