Amazon Machine Images (AMIs) are similar to Singularity containers - they let you set up EC2 instances that are already provisioned with the software tools you need so you do not have to install them. We will ultimately provide multiple AMIs in order to support different use cases and test performance. We'll start with the simplest to use.
Ubuntu AMI with Singularity
The idea here is to run JEDI within the JCSDA singularity container. Since the singularity container is continuously updated to work with the latest JEDI develop branches, this allows you to work with the latest code developments.
So, this image does not include the usual JEDI tools such as ecbuild, eckit, fckit, boost, etc. However, it does have Singularity installed. So, you can manually pull the latest Singularity image, enter the container, and proceed as you would on other systems. This image is build off of the basic Amazon-provided Ubuntu 16.04 AMI: Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-ba602bc2
To work with this AMI, log into the AWS console and go to your EC2 Dashboard. Then, on the menu on the left, select AMIs. At the top of this page you see a search box. To the left of the search box, select Public Images. Then click in the search box and you'll see a drop down menu listing attributes to search by. Select AMI Name and enter this name:
jedi-ubuntu16.04-singularity
Now select Launch on the upper left. And, proceed as you would with any other EC2 instance. Select the instance type (t2.micro is fine for ufo-bundle) and enter Review and Launch. Then, on the review page, select edit security groups, and select an existing security group. Select oregon-default or virginia-default depending on your region. Then go ahead and launch it.
After the instance is established, ssh into it as before. Remember that this is an ubuntu instance so the user name is ubuntu.
After you log in, you can first mount an existing EFS as described here: (remember that it may be easiest to go to your EFS Dashboard and copy and paste the mount command for the desired file system)
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-xxxxxxxx.efs.us-west-2.amazonaws.com:/ efs
Then download the JCSDA singularity image and enter the container:
singularity pull shub://JCSDA/singularity singularity shell -e JCSDA-singularity-master-latest.simg export FC=mpifort
Then proceed to clone the bundle you want and build it as usual. Remember to place both the src directory and the build directory in your efs directory so you have sufficient storage space.
Standalone Ubuntu and Amazon Linux AMIs
We also have public AMIs available that work outside of the singularity container, with the necessary software (GNU compilers, open-mpi, hdf5, netcdf, boost, eigen3, etc) installed directly on the system. :
jedi-amazon2
Available (public) in the us-west-2 region (Oregon), this image uses the Amazon Linux 2 operating system. In addition to the base software environment, this also includes the latest versions of eckit, fckit, and ecbuild (latest as of September, 2018).
ufo_bundle
Available (public) in the us-east-1 region (N. Virginia), this image uses the Ubuntu 16.04 operating system
ubuntu_18_04_01_nemsfv3gfs
This is a private 18-04 ubuntu AMI available in the us-east-1 region (N. Virginia) that includes the configuration files needed to compile and run the NEMS version of FV3-GFS :
To use any of these images on a single EC2 instance, simply go to your EC2 Dashboard, scroll down the left menu to Images - AMIs, select the AMI you wish to use, and hit Launch. Then proceed as you would with any other EC2 instance. Note - you may wish to mount an EFS file system as described above to build and run jedi in.
Keep an eye out for more JCSDA AMIs and ask Mark (miesch@ucar.edu) if you have any questions about what is in these or how to use them.