...

1. Make sure you are pointing to a python that is version 3.8 or higher

   HPC Platform	python3.8+ environment
   S4	module load miniconda/3.8-s4
   Discover	module load python/GEOSpyD/Min4.10.3_py3.9

   
   

2. Make sure you have checked out the branches you need to test before running the setup.sh script

   1. Typically you will be testing feature branches that could have come from various developers. Each of these developers has their own fork of which will be marked on the PR, and you will need to check those out in your local clone. One way to do this is to add their forks to your remotes in your local clone. This can be done using a series of the "git remote add ..."  command. Here's an example:

      Code Block
      language bash theme Midnight firstline 1 title Add remote repos to your local clone
      # Alex Richert (NOAA) and Dom (NRL) are two of the more active PR contributors git remote add alex https://github.com/AlexanderRichert-NOAA/spack-stack git remote add dom https://github.com/climbfuji/spack-stack ... # Check that you have these entered git remote -v # Update references to the other forks. Running the remote update # command below will pull in all the metadata to the other forks. git remote update -p # Check that you got the references to the other forks (note the use of # the -a option on the git branch command) git branch -avv

      
      

   2. Once you have the necessary remotes added, then you can checkout feature branches from other repos

      Code Block
      language bash theme Midnight firstline 1 title Check out feature branches from other forks
      # Using the remote link entered above, checkout the feature/cool-spack-thing from Dom's spack-stack fork git checkout dom/feature/cool-spack-thing

      
      

...

Note that you don't need jedipara access to do this testing. You can build everything in your user area on S4 or Discover. To help manage that, both HPC platforms supply commands to show how much of your quota (disk space and number of files) is free. 

spack-stack Add-on Environment

Occasionally, there is a need to add a handful of upgraded packages to an existing spack-stack release. spack-stack contains a feature called "chaining environments" which allows the rapid construction of such an "add-on" environment. The idea is to build a new environment that points to an existing environment of which you only have to add in the upgraded packages. The module files in the add-on environment then utilize the base environment's installation and module files for the packages that remain unchanged. 

Recently the need for this came up for spack-stack-1.6.0 with the g2@3.5.1 and g2tmpl@1.13.0 upgraded packages. Note that spack-stack-1.6.0 was shipped with g2@3.4.5 and g2tmpl@1.12.0. Here is the spack-stack issue describing the desired upgrade: https://github.com/JCSDA/spack-stack/issues/1180. Note under this comment, a method for handling this particular request (on S4) was described here: https://github.com/JCSDA/spack-stack/issues/1180#issuecomment-2251378587. In this case, an add-on environment already existed so it wasn't necessary to create it and thus it was possible to simply add in the new packages, concretize, install, and update modules (ie, skip some of the steps in the chaining environments recipe).

Here are the steps taken to accomplish this task.

1. Log into S4 and switch to the jedipara account
   1. sudo -iu jedipara 
   2. We will need to get more of us the jedipara account access
2. module load miniconda/3.8-s4 
   1. This satisfies the requirement to be pointing to python3.8+ before sourcing the spack-stack setup.sh script
3. cd /data/prod/jedi/spack-stack/spack-stack-1.6.0 
4. source setup.sh 
5. cd envs/upp-addon-env 
6. spack env activate . 
7. edit spack.yaml
   1. Make the following modifications in the specs:  section

      Code Block
      language yml theme Midnight firstline 1 title Change From This
      specs: - upp-env %intel ^g2tmpl@1.12.0 ^g2@3.4.5 - prod-util@2.1.1 %intel - ip %intel

      Code Block
      language yml theme Midnight firstline 1 title Change To This
      specs: - upp-env %intel ^g2tmpl@1.12.0 ^g2@3.4.5 - upp-env %intel ^g2tmpl@1.13.0 ^g2@3.5.1 - grib-util@1.3.0 %intel ^g2@3.4.5 - grib-util@1.3.0 %intel ^g2@3.5.1 - prod-util@2.1.1 %intel - ip %intel - ufs-weather-model-env %intel ^g2tmpl@1.12.0 ^g5@3.4.5

   2. Note that
      1. grib-util is dependent on g2, and we want two versions of grib-util: one built with g2@3.4.5 and the other with g2@3.5.1
      2. ufs-weather-model-env is dependent on g2tmpl@1.12.0 and g2@3.4.5, and we want to preserve this
      3. We want two versions of upp-env: one with g2tmpl@1.12.0/g2@3.4.5 and the other with g2tmpl@1.13.0/g2@3.5.1
      4. Hopefully the spec changes indicated above make sense in the context of what we want
8. Update envrepo/packages  following what was done on Orion
   1. The idea here is to add the new versions of g2 and g2tmpl to the packages under envrepo . envrepo  is a special place used by the chaining environment mechanism to hold extra methods for building the new versions we are trying to add in.
   2. There are probably many ways to accomplish this, but since there was a model of what needed to be done already on Orion, I did the following:
      1. Tar'd on Orion the spack-stack-1.6.0/envs/upp-addon-env/envrepo  files.
      2. Unpacked the tar on S4 in a temporary directory
      3. Updated the `/data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/envrepo` directory by comparing it with the unpacked tar files.
9. edit common/modules.yaml 
   1. Under the modules.default.lmod.hierarchy section, add in entries for g2virt  and g2tmplvirt

      Code Block
      language yml theme Midnight firstline 1 title Add in 'g2virt' and 'g2tmplvirt'
      modules: default: ... lmod: ... hierarchy: - mpi - g2virt - g2tmplvirt

10. spack concretize 2>&1 | tee log.concretize 
    1. Check that the only things being concretized are those related to the new specs in the spack.yaml  file above

       Code Block
       language bash theme Midnight firstline 1 title Check what was concretized
       grep "==> Concretized " /data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/upp-addon-env/log.concretize ==> Concretized grib-util@1.3.0%intel ^g2@3.4.5 ==> Concretized grib-util@1.3.0%intel ^g2@3.5.1 ==> Concretized ufs-weather-model-env%intel ^g2@3.4.5 ^g2tmpl@1.12.0 ==> Concretized upp-env%intel ^g2@3.4.5 ^g2tmpl@1.12.0 ==> Concretized upp-env%intel ^g2@3.5.1 ^g2tmpl@1.13.0

11. space install -v 2>&1 | tee log.install 
    1. Again this should run quickly and only build the new package versions and associated environment updates
12. spack module lmod refresh --upstream-modules 
13. spack stack setup-meta-modules 

Pages

• Setting up CI (spack-stack)
• Chaining environments