Table of Contents

Getting Started

  1. Install Orbstack as described here.
  2. In your shell, add the OrbStack directory to your path as instructed when you launch the app. e.g.: export PATH=$PATH:/Users/<user>/.orbstack/bin
  3. After launching, I recommend upping the max allowed memory to at least 16GB. This can be done in the Settings in the app. An indication that you are running out of memory when you compile is if you see messages like: “g++: fatal error: Killed signal terminated program cc1plus”
  4. You can create a Rocky 8 VM called <name> with the command: orb create rocky:8 <name>. Or an Ubuntu machine with the command: orb create ubuntu:noble <name>. If you don’t specify a version, it will use the latest version of the specified O/S. (As of 2/27/25, the latest version of Ubuntu doesn’t work with spack-stack, because it has gcc@14, which is too new. So use Ubuntu “noble” (v24.04)) See this page for full instructions on creating a variety of different Linux machines. Instructions for building spack-stack on Ubuntu Noble are below.
  5. I recommend reading this entire page of the OrbStack docs for the basics of using a Linux machine in OrbStack: https://docs.orbstack.dev/machines/
  6. Once your machine is running, the command orb takes you to a shell in the default machine. (Pay attention to what directory orb takes you to, though, as described in detail in #8 below.) You can change the default machine in the app or CLI if you have more than one. Here is a page on the OrbStack CLI.
  7. Now you’re in the linux machine and you can install spack-stack and jedi-bundle per the usual instructions.
  8. Make sure you know what directory you’re in before installing anything so you don’t accidentally install things on the MacOS side of the filesystem. If you do a “pwd” command and you’re in something that starts with “/Users/*”, you’re probably in the MacOS filesystem. Do a “cd” command to be taken to the Linux home directory.
  9. If you’re using Rocky, and you find for some reason your machine isn’t finding your gcc compiler, remember the command: scl enable gcc-toolset-11 bash
  10. I was able to get VS Code to work on my OrbStack machine using the VS Code Remote Dev tools and the OrbStack SSH server.

Building spack-stack

This example is for Ubuntu Noble (24.04), using the gcc 13.3.0 compilers and OpenMPI 5.0.5

  1. Follow the instructions for Prerequisites for Ubuntu (one-off): 
    1. https://spack-stack.readthedocs.io/en/latest/NewSiteConfigs.html#prerequisites-ubuntu-one-off
    2. When installing the gcc compilers, make sure you install version 13
      1. apt install -y gcc-13 g++-13 gfortran-13 gdb
    3. Check that you have the links for the gcc compilers installed, and that the version is 13.x
      1. gcc –version
  2. Follow the instructions for creating a new environment
    1. https://spack-stack.readthedocs.io/en/latest/NewSiteConfigs.html#newsiteconfigs-linux-createenv
    2. Step 1: For now use the 1.9.0 release branch when cloning spack-stack
      1. Place the spack-stack clone in you home directory
      2. git clone -b release/1.9.0 --recurse-submodules https://github.com/jcsda/spack-stack.git
    3. Step 2: Create a unified-dev environment
      1. spack stack create env --site linux.default --template unified-dev --name unified-env.gcc --compiler=gcc
    4. Step 5: Check that “spack compiler find --scope system” reports version 13.x
    5. Step 7: Use openmpi 5.0.5 instead of mpich
      1. In the section where you are adding the compiler and mpi to the config, use (example is based on “gcc –version” returning 13.3.0)
        1. spack config add "packages:all:compiler:[gcc@13.3.0]"
        2. spack config add "packages:all:providers:mpi:[openmpi@5.0.5]"
    6. Step 9: Leave the module configuration as is (tcl), so skip this step

 

Building jedi-bundle and running skylab

  1. Follow instructions in the JEDI documentation
    1. https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/using/running_skylab/running_skylab.html
    2. In the jedi-tools provided setup.sh script, set:
      1. HOST=localhost
      2. COMPILER=gnu
      3. And uncomment/modify the module command lines so they look like
        1. module purge
        2. module use /home/steveherbener/spack-stack/envs/ue-gcc/install/modulefiles/Core
        3. module load stack-gcc/13.3.0
        4. module load stack-openmpi/5.0.5
        5. module load stack-python/3.11.7
  2. You will need to setup the mysql and r2d2 localhost. I followed the instructions in the R2D2 repo.
    1. https://github.com/JCSDA-internal/r2d2/blob/develop/TUTORIAL.md#prerequisites-for-hpc-macos-and-aws-single-nodes
    2. To start the mysql server use: sudo service mysql start
    3. I had trouble getting the local database setup script to run due to permission problems with logging into the mysql server as ‘root’@’localhost’. I had to do the following to get this working. What this does is remove the password on the root user in the mysql server.
      1. sudo su
      2. mysql -u root
      3.   use mysql;
      4.   alter user ‘root’@’localhost’ identified with mysql_native_password by ‘’;
      5.   flush privileges;
      6.   exit;   # exit mysql
      7. exit  # exit sudo
  3. Initially, I could not get the ecflow UI to connect to the mac display. I had to follow the instructions here to get this working:
    1. https://github.com/orbstack/orbstack/issues/139#issuecomment-1595364746
      1. Step 2 in these instructions didn’t work for me. What you want to do is simply copy the contents of the ~/.ssh/id_rsa.pub file on your mac home directory into the ~/.ssh/authorized_keys file on your OrbStack (ubuntu) home directory.
    2. Take note in Step 5 of the manner in which you connect to the OrbStack machine from the mac. Every time you access the OrbStack machine you will need to open a terminal on the mac (not through ObsStack) and run the given ssh command (ssh -Y -p 2222 127.0.0.1).

Changes for running skylab with R2D2 V3

  1. The primary change from R2D2 v2 to v3 is the use of a docker container to create the R2D2 server. This docker container contains the r2d2 repo (server code) and the mysql and provides a python API for the r2d2-client code to access the server. Since this is in a docker container and the OrbStack VM itself is a docker container, you need to run the R2D2 server on the native mac (outside of the OrbStack VM).
  2. Replace step 2 above (Building jedi-bundle and running skylab) with the following instructions:
  3. On the MacOS
    1. Stop the mysql service:
      1. brew services stop mysql
    2. Create a directory to hold the r2d2 server repos, for example:
      1. mkdir -p $HOME/r2d2-server
      2. cd $HOME/r2d2-server
      3. git clone https://github.com/jcsda-internal/r2d2
      4. git clone https://github.com/jcsda-internal/r2d2-data
      5. Follow instructions here: https://github.com/JCSDA-internal/r2d2/blob/develop/server/README.md#localhost-docker-with-database
        1. Skip the check that starts with “mysql -u root …”
        2. Do the check that starts with “curl -X GET …”
          1. If this works, you’re good on the MacOS side
  4. In the ObsStack VM
    1. Stop the mysql service:
      1. sudo service mysql stop
      2. sudo service --status-all  # check that mysql is not running
    2. Make sure the link from the r2d2-data repo is in $HOME and not in $JEDI_WORKDIR
      1. ln -s $JEDI_WORKDIR/r2d2-data/r2d2-experiments-localhost $HOME
    3. In the setup.sh script
      1. API_KEY=localhost
      2. HOST=localhost
      3. USER=localhost
      4. R2D2_SERVER_HOST=”http://host.docker.internal”
      5. Note that the R2D2_SERVER_HOST setting is different from the typical setting for localhost. This is key to getting the ObsStack VM connected to the docker r2d2 server running on the MacOS (step 3 above).
    4. check the connection to the r2d2 server
      1. source setup.sh
      2. curl -X GET $R2D2_SERVER_HOST:$R2D2_SERVER_PORT/health
        1. This should produce a return code of zero (no messages)

Running Docker inside Orbstack

Docker has this feature of “docker inside docker”. Since Orbstack runs docker engine, you can run docker inside Orbstack! To setup docker you should follow the following script for Ubuntu:


# Remove any apt-based docker

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done


# Download and install latest docker

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin


# Add user to docker group to remove the need of using sudo

sudo usermod -aG docker $USER

sudo systemctl restart docker

Exit


From here, activate your orbstack terminal again and then follow the steps here: https://github.com/JCSDA-internal/r2d2/blob/develop/server/README.md#localhost-docker-with-database

 

Configuring and running vscode remotely on your OrbStack VM

I’m not sure I fully understand what’s going on, but I am able to run the vscode ssh-remote extension on my Mac which will connect with the OrbStack VM and perform the vscode actions on the ObsStack VM. Here’s what I did:

  • On the Mac
    • edit ~/.ssh/config and add a new host for the OrbStack VM
    • Add a new host definition for the obsstack-vm

Host orbstack-vm

    HostName 127.0.0.1

    Port 2222

    User stephen

    IdentityFile ~/.ssh/id_rsa

  • On the ObsStack VM
    • edit/create the file: ~/.local/share/CMakeTools/cmake-tools-kits.json
    • Add a new kit definition for the OrbStack VM
      • The example below if leveraging my skylab setup in my OrbStack VM:
        • I made a skylab configuration in ~/projects/jedi, so I’m telling vscode to source the setup script from that process to load up all of the spack-stack-1.9.1 modules

  {

    "name": "orb-stack GCC 13.3.0",

    "compilers": {

      "C": "/usr/bin/gcc",

      "CXX": "/usr/bin/g++",

      "FC": "/usr/bin/gfortran",

      "F77": "/usr/bin/gfortran"

    },

    "environmentSetupScript": "/home/stephenh/projects/jedi/setup.sh",

    "isTrusted": true

  }

  • Also, on the OrbStack VM
    • I placed the sourcing of my skylab module environment in my ~/.bashrc file. This gets the terminal mode in vscode to load up my skylab environment. You can then run commands in the vscode terminal such as ncdump, nccmp, etc. Also, I think this also enables the gdb debug mode in vscode when you set up running an app in the launch.json config. Here’s what I have in my .bashrc:

umask 0022

cd ~/projects/jedi

source setup.sh

cd

  • The reason for the cd commands is to properly activate the python venv for the spack-stack-1.9.1 environment.
  • Also, if you are interested in using the Ninja build system (which can be much faster than unix make) then install it using:
    • sudo apt install ninja-build
  • Here is what to do to run a vscode session.
    • On your Mac, start up vscode
      1. Command Palette → Remote-SSH: Connect Current Window to Host
        1. Select the obstack-vm host
        2. Verify you have a connection by checking the lower left corner of your vscode window. There should be a highlighted message containing the “orbstack-vm” name like “SSH: orbstack-vm”
      2. Command Pallette → show remote menu
        1. Open
          1. The browser should be showing you the OrbStack filesystem
          2. Select the workspace you want to develop on
      3. Continue as if running locally
  • Once you’ve done the above, when you start up vscode on your Mac, then you should see a selection option for the remote-ssh obrstack-vm workspace and you can just click on that to get started.
  • In vscode after your workspace is open
    • In your workspace settings.json add some cmake tools configuration. Here’s what I have, which is tailored to my spack-stack-1.9.1 installation and for working on ioda (but hopefully it's helpful for setting up for your needs)

{

    "cmake.sourceDirectory": "${workspaceFolder}/jedi-bundle",

    "cmake.buildDirectory": "${workspaceFolder}/build-debug",

    "cmake.cmakePath": "/home/stephenh/projects/spack-stack/envs/ue-gcc/install/gcc/13.3.0/cmake-3.27.9-gve5oit/bin/cmake",

    "cmake.generator": "Ninja",

    "cmake.ctestDefaultArgs": [ "-R", "ioda" ],

    "cmake.ctest.allowParallelJobs": true,

    "cmake.ctest.parallelJobs": 10

}

  • No labels