Step-by-step guide

 

Install Docker on your computer

please refer to https://www.docker.com/community-edition for Docker installation on Linux, Mac and Windows.

 

Docker installation

I still have some issues with Dokcer on Windows system regarding to how to access the local host from container. ....Win 10 Pro may work well

 

Pull Docker images from https://hub.docker.com/

I already built two Docker images, which include the all necessary libraries ( Jasper, JPEG, PNG, ZLIB SZIP, NetCDF, HDF5, PNETCDF, UDUNITS2, XERCES, NCO, CDO , GRIP-API, ESMF, MPI)

 

checkout the images

> docker pull xinzhang8noaa/common_libs

Using default tag: latest
latest: Pulling from xinzhang8noaa/common_libs
Digest: sha256:c30d0f571c05e9f8dd9a5eff99e975a47a458dd6c351daa7c2a454ef9740f596
Status: Image is up to date for xinzhang8noaa/common_libs:latest

#List the images you have cached

> docker images

REPOSITORY TAG IMAGE ID CREATED SIZE
xinzhang8noaa/common_libs latest 085872e2c814 3 days ago 794 MB

 

Checkout the GSI code from Bitbucket

 

checkout GSI from bitbucket

> git clone https://username@bitbucket.org/jcsda/gsi.git

> cd gsi/
> git submodule init
> git submodule update
> git fetch && git checkout gnu_pgi_port


Run the Docker


Run the docker

#Suppose you have a GSI directory, which includes the GSI codes

> pwd

/home/xinzhang/jedi/gsi

#we are going to start the docker and mount the GSI directory into the container

#Firstly, we need to check the uid and gid of your working directory will be mounted in container

>id

uid=1000(xinzhang) gid=1001(xinzhang) .......

#Start the docker container

> docker run -it --rm --name GSI -v `pwd`:/gsi -w /gsi --user 1000:1001 -h gsi xinzhang8noaa/common_libs

groups: cannot find name for group ID 1001
I have no name!@gsi:/gsi$

#this command will start a isolated linux session with all the libraries, tools needed to compile, run GSI. please ignore "I have no name!",as the container linux does not have your account.

  • -it : iteractive session
  • --rm: the container will be delete once you exit from the session
  • --name GSI : session name
  • -v `pwd:/gsi : mount the current directory as /gsi in container
  • --user 1000:1001 : file access permission in containner, it sets the same as the host machine.
  • -h gsi : set the hostname

Inside the container

inside the container

> pwd

/gsi

> which mpif90

/usr/local/bin/mpif90

> cd /usr/local/lib ; ls

cmake libhdf5_hl_cpp.so.11 libmpi_usempi_ignore_tkr.la libpng.a
esmf.mk libhdf5_hl_cpp.so.11.1.0 libmpi_usempi_ignore_tkr.so libpng.so
libblas.so libhdf5hl_fortran.a libmpi_usempi_ignore_tkr.so.20 libpng14.a
libblas.so.3 libhdf5hl_fortran.la libmpi_usempi_ignore_tkr.so.20.10.0 libpng14.so
libblas.so.3.7.0 libhdf5hl_fortran.so libmpi_usempif08.la libpng14.so.14
libesmf.a libhdf5hl_fortran.so.10 libmpi_usempif08.so libpng14.so.14.19.0
libesmf.so libhdf5hl_fortran.so.10.0.3 libmpi_usempif08.so.20 libsz.a
libesmf_fullylinked.so libjasper.a libmpi_usempif08.so.20.10.0 libsz.so
libfreetype.a libjasper.la libnetcdf.settings libszip.a
libgrib_api.so libjasper.so libnetcdf.so libszip.so
libgrib_api_f77.so libjasper.so.1 libnetcdf.so.11 libszip.so.2.1
libgrib_api_f90.so libjasper.so.1.0.0 libnetcdf.so.11.4.0 libtmglib.so
libhdf5.a libjpeg.a libnetcdff.a libudunits2.so
libhdf5.la libjpeg.la libnetcdff.la libxerces-c-3.1.so
libhdf5.settings libjpeg.so libnetcdff.so libxerces-c.a
libhdf5.so libjpeg.so.9 libnetcdff.so.6 libxerces-c.la
libhdf5.so.10 libjpeg.so.9.2.0 libnetcdff.so.6.1.1 libxerces-c.so
libhdf5.so.10.2.1 liblapack.so libompitrace.la libz.a
libhdf5_cpp.a liblapack.so.3 libompitrace.so libz.so
libhdf5_cpp.la liblapack.so.3.7.0 libompitrace.so.20 libz.so.1
libhdf5_cpp.so libmca_common_sm.la libompitrace.so.20.10.0 libz.so.1.2.11
libhdf5_cpp.so.13 libmca_common_sm.so libopen-pal.la mpi.mod
libhdf5_cpp.so.13.0.0 libmca_common_sm.so.20 libopen-pal.so mpi_ext.mod
libhdf5_fortran.a libmca_common_sm.so.20.10.0 libopen-pal.so.20 mpi_f08.mod
libhdf5_fortran.la libmpi.la libopen-pal.so.20.10.0 mpi_f08_callbacks.mod
libhdf5_fortran.so libmpi.so libopen-rte.la mpi_f08_ext.mod
libhdf5_fortran.so.10 libmpi.so.20 libopen-rte.so mpi_f08_interfaces.mod
libhdf5_fortran.so.10.0.4 libmpi.so.20.10.0 libopen-rte.so.20 mpi_f08_interfaces_callbacks.mod
libhdf5_hl.a libmpi_cxx.la libopen-rte.so.20.10.0 mpi_f08_types.mod
libhdf5_hl.la libmpi_cxx.so liboshmem.la openmpi
libhdf5_hl.so libmpi_cxx.so.20 liboshmem.so pkgconfig
libhdf5_hl.so.10 libmpi_cxx.so.20.10.0 liboshmem.so.20 pmpi_f08_interfaces.mod
libhdf5_hl.so.10.1.1 libmpi_mpifh.la liboshmem.so.20.10.0 python3.5
libhdf5_hl_cpp.a libmpi_mpifh.so libpnetcdf.a
libhdf5_hl_cpp.la libmpi_mpifh.so.20 libpnetcdf.so
libhdf5_hl_cpp.so libmpi_mpifh.so.20.10.0 libpng

Compiling GSI inside the container:

 

compiling GSI inside the container

> mkdir build

> cd build/

> export FC=mpifort

> cmake -DBUILD_CORELIBS=ON -DUSE_WRF=OFF -DBUILD_GLOBAL=ON ..

-- The C compiler identification is GNU 6.2.0
-- The CXX compiler identification is GNU 6.2.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 6.2.0
-- Check for working Fortran compiler: /usr/bin/gfortran
-- Check for working Fortran compiler: /usr/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran supports Fortran 90
-- Checking whether /usr/bin/gfortran supports Fortran 90 -- yes
Setting paths for Generic System
/gsi
Setting GNU flags
Setting GNU Compiler Flags
-- Found MPI_C: /usr/local/lib/libmpi.so 
-- Found MPI_CXX: /usr/local/lib/libmpi_cxx.so;/usr/local/lib/libmpi.so 
-- Found MPI_Fortran: /usr/local/lib/libmpi_usempif08.so;/usr/local/lib/libmpi_usempi_ignore_tkr.so;/usr/local/lib/libmpi_mpifh.so;/usr/local/lib/libmpi.so 
-- Found NetCDF: /usr/local/lib/libnetcdff.so;/usr/local/lib/libnetcdf.so 
-- Found HDF5: /usr/local/lib/libhdf5.so;/usr/local/lib/libsz.so;/usr/local/lib/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/local/lib/libhdf5_hl.so;/usr/local/lib/libhdf5.so;/usr/local/lib/libsz.so;/usr/local/lib/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/local/lib/libhdf5hl_fortran.so;/usr/local/lib/libhdf5_hl.so;/usr/local/lib/libhdf5_fortran.so;/usr/local/lib/libhdf5.so;/usr/local/lib/libsz.so;/usr/local/lib/libz.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.8.17")
-- Found CURL: /usr/lib/x86_64-linux-gnu/libcurl.so (found version "7.47.0")
 trying to find lapack
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE 
-- A library with BLAS API found.
-- Looking for Fortran cheev
-- Looking for Fortran cheev - found
-- A library with LAPACK API found.
setting values for corelibs
in core-libs
in enkf
-- A library with BLAS API found.
-- A library with LAPACK API found.
-- Configuring done
-- Generating done
-- Build files have been written to: /gsi/build
> nproc
8
 
> make -j4
Scanning dependencies of target bufrlib_prm
Scanning dependencies of target bacio_v2.0.1
Scanning dependencies of target copy
[  0%] Generating ../../include/bufrlib.prm
Scanning dependencies of target crtm_v2.2.3
[  0%] Built target copy
[  0%] Building Fortran object core-libs/bacio/CMakeFiles/bacio_v2.0.1.dir/__/__/libsrc/bacio/baciof.f.o
[  0%] Building Fortran object core-libs/bacio/CMakeFiles/bacio_v2.0.1.dir/__/__/libsrc/bacio/bafrio.f.o
[  0%] Built target bufrlib_prm
[  0%] Building Fortran object core-libs/bacio/CMak..................
......................
......................
[100%] Linking Fortran executable ../../bin/enkf_gfs.x
[100%] Built target enkf_gfs.x
 
ls -al bin/gsi_global.x
-rwxr-xr-x. 1 1000 1001 38298632 Jun 19 18:03 bin/gsi_global.x