...
Singularity containers can be built using a .def file. This allows for easy reproducibility and standardization. You can find more information on this file setup in the Singularity Documentation. Below is a simple .def file used for a bare bones container that is used to run WeatherBenchX.TODO: insert def from Hercules
| Code Block | ||
|---|---|---|
| ||
Bootstrap: docker
From: ubuntu:22.04
%labels
Maintainer AGriffin
Purpose WeatherBenchX
%post
export DEBIAN_FRONTEND=noninteractive
# Install dependencies
apt-get update && apt-get install -y \
software-properties-common \
curl \
wget \
build-essential \
git \
vim \
libffi-dev \
libssl-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
zlib1g-dev \
libncursesw5-dev \
xz-utils \
tk-dev \
libxml2-dev \
libxmlsec1-dev \
liblzma-dev \
ca-certificates \
&& apt-get clean
# Add deadsnakes PPA and install Python 3.11 and venv
add-apt-repository ppa:deadsnakes/ppa -y
apt-get update && apt-get install -y \
python3.11 \
python3.11-venv \
python3.11-dev \
python3.11-distutils \
&& apt-get clean
# Point python3 to python3.11
update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1
# Create the virtual environment
python3 -m venv /venv_wb
# Install WeatherBenchX and dependencies
/venv_wb/bin/pip install --upgrade pip setuptools wheel
git clone https://github.com/google-research/weatherbenchX /weatherbenchX
cd /weatherbenchX
/venv_wb/bin/pip install -e .
/venv_wb/bin/pip install netCDF4
%environment
# Activate virtual environment at runtime
source /venv_wb/bin/activate
export PATH="/venv_wb/bin:$PATH"
%runscript
echo "Singularity container for WeatherBenchX"
exec /bin/bash |
In order to build a container on a remote host or HPC, you need to use the --remote flag since we typically do not have admin privileges. An example build command using the definition file above on Hercules is: singularity build --remote wb_container.sif /work2/noaa/jcsda/agriffin/JEDI_WB/wb_container/wb_container.def
...