Getting Started

1. Install Orbstack as described here.
2. In your shell, add the OrbStack directory to your path as instructed when you launch the app. e.g.: export PATH=$PATH:/Users/<user>/.orbstack/bin
3. After launching, I recommend upping the max allowed memory to at least 16GB. This can be done in the Settings in the app. An indication that you are running out of memory when you compile is if you see messages like: “g++: fatal error: Killed signal terminated program cc1plus”
4. You can create a Rocky 8 VM called <name> with the command: orb create rocky:8 <name>. Or an Ubuntu machine with the command: orb create ubuntu:noble <name>. If you don’t specify a version, Create virtual machine(s).  Recommend picking only one, instructions for building spack-stack on Ubuntu Noble are below.  If you don’t specify a version, it will use the latest version of the specified O/S. (As of 2/27/25, the latest version of Ubuntu doesn’t work with spack-stack, because it has gcc@14, which is too new. So use Ubuntu “noble” (v24.04)) See this page for full instructions on creating a variety of different Linux machines. Instructions for building spack-stack on Ubuntu Noble are below.
   1. Rocky 8 VM called <name> with the command:
      orb create rocky:8 <name>
   2. Or an Ubuntu machine with the command: 
      orb create ubuntu:noble <name> 
5. I recommend reading this entire page of the OrbStack docs for the basics of I recommend reading this entire page of the OrbStack docs for the basics of using a Linux machine in OrbStack: https://docs.orbstack.dev/machines/
6. Once your machine is running, the command orb takes you to a shell in the default machine. (Pay attention to what directory orb takes you to, though, as described in detail in #8 below.) You can change the default machine in the app or CLI if you have more than one. Here is a page on the OrbStack CLI.
7. Now you’re in the linux machine and you can install spack-stack and jedi-bundle per the usual instructions.
8. Make sure you know what directory you’re in before installing anything so you don’t accidentally install things on the MacOS side of the filesystem. If you do a “pwd” command and you’re in something that starts with “/Users/*”, you’re probably in the MacOS filesystem. Do a “cd” command to be taken to the Linux home directory.
9. If you’re using Rocky, and you find for some reason your machine isn’t finding your gcc compiler, remember the command: scl enable gcc-toolset-11 bash
10. I was able to get VS Code to work on my OrbStack machine using the VS Code Remote Dev tools and the OrbStack SSH server.

...

This example is for Ubuntu Noble (24.04), using the gcc 13.3.0 compilers and OpenMPI 5.0.5

1. Profile should be established this umask  will be required later in the stack install as will the ulimit  settings
   

   Code Block
   language shell
   # .bash_profile umask 0022 ulimit -S -s unlimited if [[ ~/.bashrc ]]; then . ~/.bashrc fi cd ~

   
   
2. Follow the instructions for Prerequisites for Ubuntu (one-off): 
1. https://spack-stack.readthedocs.io/en/latest/NewSiteConfigs.html#prerequisites-ubuntu-one-off
2. When installing the gcc compilers, make sure you install version 13
1. apt install -y gcc-13 g++-13 gfortran-13 gdb
3. Check that you have the links for the gcc compilers installed, and that the version is 13.x
1. gcc –version
3. Follow the instructions for creating a new environment
4. Follow the instructions for Prerequisites for Ubuntu (one-off):  
5. https://spack-stack.readthedocs.io/en/latest/NewSiteConfigs.
html#newsiteconfigs
6. html#prerequisites-ubuntu-
linux
7. one-
createenv
8. Step 1: For now use the 1.9.0 release branch when cloning spack-stack
1. Place the spack-stack clone in you home directory
2. git clone -b release/1.9.0 --recurse-submodules https://github.com/jcsda/spack-stack.git
9. Step 2: Create a unified-dev environment
1. spack stack create env --site linux.default --template unified-dev --name unified-env.gcc --compiler=gcc
10. Step 5: Check that “spack compiler find --scope system” reports version 13.x
11. Step 7: Use openmpi 5.0.5 instead of mpich
1. In the section where you are adding the compiler and mpi to the config, use (example is based on “gcc –version” returning 13.3.0)
1. spack config add "packages:all:compiler:[gcc@13.3.0]"
2. spack config add "packages:all:providers:mpi:[openmpi@5.0.5]"
12. Step 9: Leave the module configuration as is (tcl), so skip this step

Building jedi-bundle and running skylab

1. off 
   1. When installing the gcc compilers, if you would like to make sure you install version 13
      

      Code Block
      language shell
      # Compilers apt install -y gcc-13 g++-13 gfortran-13 gdb

2. Follow the instructions for creating a new environment:   https://spack-stack.readthedocs.io/en/latest/NewSiteConfigs.html#newsiteconfigs-linux-createenv 
   1. Step 1: For now use the 1.9.0 release branch when cloning spack-stack (Place the spack-stack clone in you home directory.)
      

      Code Block
      language shell
      git clone -b release/1.9.0 --recurse-submodules https://github.com/jcsda/spack-stack.git cd spack-stack # Sources Spack from submodule and sets ${SPACK_STACK_DIR} source setup.sh

   2. Step 2: Create a unified-dev environment - complete step following newsiteconfigs 
      

      Code Block
      language shell
      spack stack create env --site linux.default --template unified-dev --name unified-env.gcc --compiler=gcc cd envs/unified-env.gcc/ spack env activate [-p] .

   3. Step 3: follow newsiteconfigs 
   4. Step 4: follow newsiteconfigs 
   5. Step 5: Check that “spack compiler find --scope system” reports version 13.x)
   6. Step 6: follow newsiteconfigs 
   7. Step 7:  Use openmpi 5.0.5, can skip  add of mpich.  

      1. 1. Code Block
            language shell
            gcc --version # insert your gcc version here (example 13.3.0) spack config add "packages:all:compiler:[gcc@13.3.0]"

         2. Code Block
            language shell
            # openmpi v5.0.5 skip mpich install spack config add "packages:all:providers:mpi:[openmpi@5.0.5]"

   8. Step 8: follow newsiteconfigs 
   9. Step 9: skip - if new Orbstack 
   10. Step 10: skip - if new Orbstack
   11. Step 11: concretize and install: follow newsiteconfigs 
   12. Step 12 - 14: follow newsiteconfigs 

Building jedi-bundle and running skylab

1. Follow instructions in the JEDI documentation
   1. HOST=localhost
   2. COMPILER=gnu
   3. And uncomment/modify the module command lines so they look like
      1. module purge
      2. module use /home/steveherbener/spack-stack/envs/ue-gcc/install/modulefiles/Core
      3. module load stack-gcc/13.3.0
      4. module load stack-openmpi/5.0.5
      5. module load stack-python/3.11.7
   4. https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/using/running_skylab/running_skylab.html
   5. In the jedi-tools provided setup.sh script, set:
2. You will need to setup the mysql and r2d2 localhost. I followed the instructions in the R2D2 repo.
   1. sudo su
   2. mysql -u root
   3.   use mysql;
   4.   alter user ‘root’@’localhost’ identified with mysql_native_password by ‘’;
   5.   flush privileges;
   6.   exit;   # exit mysql
   7. exit  # exit sudo
3. Follow instructions in the JEDI documentation
1. https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/using/running_skylab/running_skylab.html
2. In the jedi-tools provided setup.sh script, set:
1. HOST=localhost
2. COMPILER=gnu
3. And uncomment/modify the module command lines so they look like
1. module purge
2. module use /home/steveherbener/spack-stack/envs/ue-gcc/install/modulefiles/Core
3. module load stack-gcc/13.3.0
4. module load stack-openmpi/5.0.5
5. module load stack-python/3.11.7
4. You will need to setup the mysql and r2d2 localhost. I followed the instructions in the R2D2 repo.
1. https://github.com/JCSDA-internal/r2d2/blob/develop/TUTORIAL.md#prerequisites-for-hpc-macos-and-aws-single-nodes
2. To start the mysql server use: sudo service mysql start
3. I had trouble getting the local database setup script to run due to permission problems with logging into the mysql server as ‘root’@’localhost’. I had to do the following to get this working. What this does is remove the password on the root user in the mysql server.
1. sudo su
2. mysql -u root
3.   use mysql;
4.   alter user ‘root’@’localhost’ identified with mysql_native_password by ‘’;
5.   flush privileges;
6.   exit;   # exit mysql
7. exit  # exit sudo
Initially, I could not get the ecflow UI to connect to the mac display. I had to follow the instructions here to get this working:
5. 2. https://github.com/
orbstack/orbstack/issues/139#issuecomment-1595364746
1. Step 2 in these instructions didn’t work for me. What you want to do is simply copy the contents of the ~/.ssh/id_rsa.pub file on your mac home directory into the ~/.ssh/authorized_keys file on your OrbStack (ubuntu) home directory.
6. 1. JCSDA-internal/r2d2/blob/develop/TUTORIAL.md#prerequisites-for-hpc-macos-and-aws-single-nodes
   2. To start the mysql server use: sudo service mysql start
   3. I had trouble getting the local database setup script to run due to permission problems with logging into the mysql server as ‘root’@’localhost’. I had to do the following to get this working. What this does is remove the password on the root user in the mysql server.
7. Initially, I could not get the ecflow UI to connect to the mac display. I had to follow the instructions here to get this working (https://github.com/orbstack/orbstack/issues/139#issuecomment-1595364746):
   Let's transcribe these specifically for our environment:
   
   
   1. step 0: install Xquartz (desktop app or brew)
   2. step 1: make sure you've created an orb machine (such as the Ubuntu example)
   3. Step 2: copy the contents of the ~/.ssh/id_rsa.pub file on your mac home directory into the ~/.ssh/authorized_keys file on your OrbStack (ubuntu) home directory.
      

      Code Block
      language shell
      cat /mnt/mac/Users/$USER/.ssh/id_rsa.pub > ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys

   4. Step 3: install SSH server
      

      Code Block
      language shell
      sudo apt-get update sudo apt install openssh-server # change ListenStream from 22 to 2222 sudo vim /lib/systemd/system/ssh.socket

      
      
   5. Step 4: restart the machine:
      

      Code Block
      language shell
      # exit orb machine exit # from Mac restart orb machine orbctl restart <name>

      
      
   6. Step 5: Every time you access OrbStack machine and wish to use X11 forwarding you will need to use a terminal on the Mac and run the given ssh command (ssh -Y -p 2222 127.0.0.1).
      

      Code Block
      language shell
      # from Mac access X11 enabled Orb ssh -Y -p 2222 127.0.0.1 # if you want these utilities and to test xclock (optional) sudo apt-get install -y x11-apps

Take note in Step 5 of the manner in which you connect to the OrbStack machine from the mac. Every time you access the OrbStack machine you will need to open a terminal on the mac (not through ObsStack) and run the given ssh command (ssh -Y -p 2222 127.0.0.1).
8. 1. 
      

Changes for running skylab with R2D2 V3

1. The primary change from R2D2 v2 to v3 is the use of a docker container to create the R2D2 server. This docker container contains the r2d2 repo (server code) and the mysql and provides a python API for the r2d2-client code to access the server. Since this is in a docker container and the OrbStack VM itself is a docker container, you need to run the R2D2 server on the native mac (outside of the OrbStack VM).
2. Replace step 2 above (Building jedi-bundle and running skylab) with the following instructions:
3. On the MacOS
   1. Stop the mysql service:
   2. 1. brew services stop mysql
   Create a directory to hold the r2d2 server repos, for example:
   3. 1. mkdir -p $HOME/r2d2-server
      2. cd $HOME/r2d2-server
      3. git clone https://github.com/jcsda-internal/r2d2
      4. git clone https://github.com/jcsda-internal/r2d2-data
      5. Follow instructions here: https://github.com/JCSDA-internal/r2d2/blob/develop/server/README.md#localhost-docker
   -with-database
   4. 1. -with-database
         1. If this works, you’re good on the MacOS side
      1. Skip the check that starts with “mysql -u root …”
      2. Do the check that starts with “curl -X GET …”
   1. Stop the mysql service:
   2. Create a directory to hold the r2d2 server repos, for example:
   If this works, you’re good on the MacOS side
   5. In the ObsStack VM
   Stop the mysql service:
   6. 1. sudo service mysql stop
      2. sudo service --status-all  # check that mysql is not running
   Make sure the link from the r2d2-data repo is in $HOME and not in $JEDI_WORKDIR
   7. 1. ln -s $JEDI_WORKDIR/r2d2-data/r2d2-experiments-localhost $HOME
   In the setup.sh script
   8. 1. API_KEY=localhost
      2. HOST=localhost
      3. USER=localhost
      4. R2D2_SERVER_HOST=”http://host.docker.internal”
      5. Note that the R2D2_SERVER_HOST setting is different from the typical setting for localhost. This is key to getting the ObsStack VM connected to the docker r2d2 server running on the MacOS (step 3 above).
   9. check the connection to the r2d2 server
   10. 1. source setup.sh
       2. curl -X GET $R2D2_SERVER_HOST:$R2D2_SERVER_PORT/health
       1. This should produce a return code of zero (no messages)
   1. Stop the mysql service:
   2. Make sure the link from the r2d2-data repo is in $HOME and not in $JEDI_WORKDIR
   3. In the setup.sh script
   4. check the connection to the r2d2 server
   no messages)

   Running Docker inside Orbstack

   Docker has this feature of “docker inside docker”. Since Orbstack runs docker engine, you can run docker inside Orbstack! To setup docker you should follow the following script for Ubuntu:! To setup docker you should follow the following script for Ubuntu:

   
   # if you want to check for these packages
   

   dpkg-query -l | grep -E 'docker.io|docker-doc|docker-compose|docker-compose-v2|podman-docker|containerd|runc'

   
   

   # if found remove # Remove any apt-based docker

   for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

   ...

   sudo systemctl restart docker
   
   # exit Orbstack

   exitExit

   
   

   From here, activate your orbstack terminal again and then follow the steps here: https://github.com/JCSDA-internal/r2d2/blob/develop/server/README.md#localhost-docker-with-database

   ...

       Port 2222

       User stephen

       IdentityFile ~/.ssh/id_rsaIdentityFile ~/.ssh/id_rsa

   • On the ObsStack
   • On the ObsStack VM
   • edit/create the file: ~/.local/share/CMakeTools/cmake-tools-kits.json
   Add a new kit definition for the OrbStack
   • VM
     • The example below if leveraging my skylab setup in my OrbStack VM:
     • I made a skylab configuration in ~/projects/jedi, so I’m telling vscode to source the setup script from that process to load up all of the spack-stack-1.9.1 modules
   • edit/create the file: ~/.local/share/CMakeTools/cmake-tools-kits.json
   • Add a new kit definition for the OrbStack VM

     {

       "name": "orb-stack GCC 13.3.0",

   ...