This guide is intended to help the user install Singularity on their Mac in preparation for using the JCSDA Singularity image.

It is assumed that the user is using Homebrew for downloading and installing packages on their Mac. In order to install Homebrew itself, execute the following command on your Mac.

Install Homebrew on your Mac 
# The following command uses a bourne shell syntax so if you experience problems, try running it from a bash shell.
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"  # install brew tools


Singularity requires two other pieces of software to put the whole container system together. The first is VirtualBox which extends your Mac to be able to run a large set of different operating systems (windows, linux, etc).  The second is Vagrant which is a manager for configuring a specific operating system using the features of VirtualBox. The combination of VirtualBox and Vagrant running a specific operating system, constitute an object commonly called a "virtual machine".  Once a virtual machine is up and running,  Singularity is used to create a specific environment (software tools, libraries, etc.) within that operating system. The environment that Singularity creates is often called a "container".

 For the JEDI project, we will be using VirtualBox and Vagrant to create an Ubuntu (version 16.04) operating system that is then populated with the specific tools (compilers, ecbuild, etc.) using Singularity.

Steps for installing Singularity

These steps are taken from those found on the Singularity website. 

  1. Install tools required for running the JEDI Singularity container

    1. Install Virtual Box and Vagrant

      brew cask install virtualbox
brew cask install vagrant
brew cask install vagrant-manager

      If you are experiencing problems with the Cask extension, try updating Homebrew and the installed software to the latest version

      brew update    # update Homebrew itself
brew upgrade   # update all the packages that have been installed by Homebrew

    2. Create a home for the Singularity container

      mkdir $HOME/singularity-vm  # Just an example location, you can choose any path you like
cd $HOME/singularity-vm

    3. Configure Vagrant

      vagrant init singularityware/singularity-2.4

      The above command will create a file called "Vagrantfile" in the current directory. This file is used to configure characteristics of the virtual machine that will be running on your Mac. We have noticed two situations that require modifications in the Vagrantfile configuration as outlined below.

      Increase memory for the virtual machine

      We have noticed that the default memory size (1 GB) specified in Vagrantfile is potentially problematic in that it may not be enough memory to support running some simulators (e.g., MPAS), and possibly the cause of corruption of the Singularity image when downloading that image. Increasing the memory size to 4 GB seems to remedy these situations.

      To increase memory, edit the Vagrantfile and find the section for the provider (virtualbox) specific configuration. This is a section that looks like:

        # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  #
  config.vm.provider "virtualbox" do |vb|
    # Display the VirtualBox GUI when booting the machine
    # vb.gui = true

    # Customize the amount of memory on the VM:
    vb.memory = "1028"
  end

      Change the vb.memory value to the desired amount of memory (in MB), which for our purposes will be 4096 (4 GB). The result should look like:

        # Provider-specific configuration so you can fine-tune various
  # backing providers for Vagrant. These expose provider-specific options.
  # Example for VirtualBox:
  #
  config.vm.provider "virtualbox" do |vb|
    # Display the VirtualBox GUI when booting the machine
    # vb.gui = true

    # Customize the amount of memory on the VM:
    vb.memory = "4096"
  end

      Make files accessible to both the host and guest machines

      By default, one cannot exchange files between the host (Mac OS) and guest (Vagrant virtual machine) machines. Fortunately, Vagrant provides a means for this type of access.

      This will enable the passing of files between the host and guest machines. For example, if you download a file from the internet on the host machine, you can simply move that file to the shared directory and the guest machine can then access it.

      Edit the Vagrantfile and find the section for a "synced folder". This is the section that looks like:

        # Share an additional folder to the guest VM. The first argument is
  # the path on the host to the actual folder. The second argument is
  # the path on the guest to mount the folder. And the optional third
  # argument is a set of non-required options.
  #config.vm.synced_folder "../vagrant_data", "/vagrant_data"

      Uncomment the config.vm.synced_folder command and set the paths to the desired locations of the directories on the host and guest machchines. Let's say that you have installed vagrant in $HOME/singularity-vm (which is the directory you will start vagrant from). The example below will make the files in the host machine directory $HOME/singularity-vm/vagrant_data visible to the guest machine in the directory /vagrant_data (and vice-versa). You need to create the $HOME/singularity-vm/vagrant_data directory before starting up vagrant.

        # Share an additional folder to the guest VM. The first argument is
  # the path on the host to the actual folder. The second argument is
  # the path on the guest to mount the folder. And the optional third
  # argument is a set of non-required options.
  config.vm.synced_folder  "./vagrant_data", "/vagrant_data"

  2. Run the JEDI Singularity container

    Note that the following steps can be repeated whenever you want to startup the Singularity container, do some work, and then shutdown the container.

    1. Startup Vagrant

      vagrant up   # start up the virtual machine
vagrant ssh  # start up your user environment within the virtual machine

    2. Startup Singularity

      This step only needs to be done when you are starting for the first time, or when the JCSDA Singularity image gets updated.

      singularity pull shub://JCSDA/singularity

      In the Singularity startup command below, note the --bind option. This tells Singularity to honor the shared directory (between host and guest machines) that was configured in Vagrantfile (see above). If you did not create the shared directory in vagrant, then the --bind option is not needed. Also, if you used a different path for the shared directory for the guest machine in Vagrantfile, the alter the argument to --bind accordingly.

      singularity shell --bind /vagrant_data -e JCSDA-singularity-master.simg

    3. Shutdown Singularity and Vagrant

      exit             # leave the Singularity container
exit             # leave the Vagrant virtual machine
vagrant halt     # shutdown the virtual machine

  3. Customizing your Singularity environment

    Singularity uses a bash shell by default but it does not initialize the shell with .bashrc or .bash_profile scripts.  This is deliberate, to avoid potential conflicts with the host environment.  You can change this default behavior by specifying the SINGULARITY_SHELL environment variable in your vagrant environment (see http://singularity.lbl.gov/docs-shell) or by manually sourcing a script file after you invoke the singulary shell.   Here is an example of such a script file that highlights a few features that can come in handy for JEDI:

    Initialization script
    #!/bin/bash
alias Rm='rm -rf '
alias bb='cd ~/jedi/build'
export FC=mpif90
export DISPLAY=10.0.2.2:0.0
# If DISPLAY=10.0.2.2:0.0 does not work, try the following DISPLAY setting
# export DISPLAY=localhost:10.0

    The alias commands are just sample customizations.  The FC declaration is necessary for JEDI components that require MPI (e.g. fv3).  The DISPLAY declaration is used to enable X forwarding to your host machine, which vagrant assigns the above IP address to by default.  To allow singularity to access the display, you must run this command on your host machine (i.e. on your Mac):

    X forwarding
    xhost + 127.0.0.1

    X forwarding is necessary to run graphical debuggers like kdbg (included in the JCSDA singularity image) and some text editors (e.g. emacs).  If the above commands don't work on your Mac, you may need to install XQuartz.