Page History

This document is adapted from an original set of instructions in this document. The original is substantially out of date but has been preserved in case it contains relevant historical information.

Images we maintain

• 747101682576.dkr.ecr.us-east-2.amazonaws.com/jedi-gnu-openmpi-dev

  
  

• 747101682576.dkr.ecr.us-east-2.amazonaws.com/jedi-oneapi-impi-dev

Environment Setup

Create a new EC2 instance using a c5n.4xlarge host. Either restore the existing container builder AMI or create a new AMI following the instructions in the spack stack repository.

...

1. ssh into your build environment (described earlier) and start a screen session.

2. Clone the spack stack repo and release

   Code Block
   clone -b release/1.7.09 --recursive "https://github.com/climbfujiJCSDA/spack-stack.git" spack-stack-1.7.0

   
   

3. Setup spack stack

   Code Block
   cd spack-stack-1.7.0 source setup.sh

   
   

4. Create the container:

   Code Block

   export DOCKERBUILD_ENV=clanggnu-mpichopenmpi export DOCKER_CTR_REPO=747101682576.dkr.ecr.us-east-2.amazonaws.com spack stack create ctr --container=docker-ubuntu-${DOCKERBUILD_ENV} \     --specs=jedi-ci | tee log.create.docker-ubuntu-${DOCKERBUILD_ENV}-ci.001 .    Configuring basic directory information ... .      ... script directory: /home/ubuntu/spack-stack/spack-stack-1.7.1/spack-ext/lib/jcsda-emc/spack-stack/stack .     ... base directory: /home/ubuntu/spack-stack/spack-stack-1.7.1/spack-ext/lib/jcsda-emc/spack-stack .      ... spack directory: /home/ubuntu/spack-stack/spack-stack-1.7.1/spack .    ==> Created container /home/ubuntu/spack-stack/spack-stack-1.7.1/envs/docker-ubuntu-clang-mpich

   
   

5. Use spack to create the docker file and build with docker

   Code Block
   cd envs/docker-ubuntu-$DOCKER$BUILD_ENV/ spack containerize > Dockerfile docker build -t $DOCKER_CTR_REPO/jedi-${DOCKERBUILD_ENV}-dev:test1.9 . 2>&1 | tee logdocker.txt

   
   

6. Testing the image once created

   Code Block

   sudo rm -rf $HOME/builds/$BUILD_ENV && mkdir -p $HOME/builds/$BUILD_ENV docker run -v $HOME/builds/$BUILD_ENV:/build -w /build -it \ ${DOCKER_CTR_REPO}/jedi-$BUILD_ENV-dev:1.9 /bin/bash # Now in the container environment. git config --global credential.helper 'cache --timeout=3600' git clone https://github.com/jcsda-internal/jedi-bundle.git wget https://bin.ssec.wisc.edu/pub/s4/CRTM//fix_REL-3.1.1.2.tgz -O /build/fix_REL-3.1.1.2.tgz export CRTM_BINARY_FILES_TARBALL=/build/fix_REL-3.1.1.2.tgz # Remove local esmf rm -vf `find /opt/view/bin -iname '*esmf*'` && \ rm -vf `find /opt/view/lib -iname '*esmf*'` && \ rm -vf `find /opt/view/include -iname '*esmf*'` && \ rm -vf `find /opt/view/cmake -iname '*esmf*'` # Note that new spack stack environments (prior to a skylab release) often # have ctest failures but they should not have build failures. mkdir jedi-bundle/build && cd $_ ecbuild ../ 2>&1 | tee log.configure make -j4 2>&1 | tee log.make ctest

7. push the built image to ECR

   Code Block
   aws ecr get-login-password --profile=jcsda-usaf-aws-us-east-2 \                            --region us-east-2 \    | docker login --username AWS \        --password-stdin $DOCKER_CTR_REPO docker push ${DOCKER_CTR_REPO}/jedi-${DOCKERBUILD_ENV}-dev:test

   
   

Update containers on DockerHub

...