...
Cheng is working on aerosol component.
OBS - Hui
- UFO code sprint for naming convention: up to now, 91% ctests passed with new naming convention in the sprint branch. Volunteers continue to be needed to finish ctest for the code naming convention changes but also to excercise UFO tests for full-volume data
- NOAA-21 (JPSS-2) is being monitored in JCSDA obs monitoring pagehttps://demo.jcsda.org/: NOAA-21 ATMS. JCSA obtained data 11 days after launch and started invetigating on this new data. Images shown on the webpage received approval from JPSS-2 office.
- Developing precipitation verification tools using Imerge and JEDI cycling runs (Lindsey Hayden will present over AGU)
- Preparing talks for JCSDA symposium in Jan
Greg Thompson
20+ participants in code sprint
...
- R2D2 - Eric is now able to ingest the existing r2d2-data repository into mysql and run all the toy model experiments in skylab, with a PR for fv3-jedi also some of the others?
- coming weeks to fine tune, test on other platforms, figure out details on communication, improve error handling, ...
- R2D2 - Maryam working on automatic CI testing using Tariq's testing harness
- IODA
- memory issue reported for ioda reader where the size of the data in memory is ten times larger than that in (uncompressed) netCDF files. Increase in memory is somewhere in the compute phase (constructing obs frame?) - Steve looking at it with help from Sergey
- ask Steve H. about the issue for SOCA
- work on improving reader speed using parallel reads will start after these two issues have been addressed
- reminder: parallel write issue has been identified, fix (remove colons from filenames and filepaths) will be introduced gradually over the next weeks/months to avoid breaking existing capabilities (because we can turn of parallel writes in the yamls)
- AWS
- skylab-atm-land experiments now run reliably on parallelcluster, current setup (with room for improvement) costs about 400-500$ per one day (4 cycles) with all 25 members
- Anna now a beta tester of the system
- currently no user administration - just a single ubuntu user - Dom working on setting up ldap on the head node for proper user administration
- long-term plan is to have two clusters up and running at all times (head nodes; compute nodes shut down automatically if unused), a near real-time system and an R&D system, with access for JCSDA core and in-kinds (details to be determined)
- UFS-jedi
- UFS-jedi model and forecast run on AWS parallelcluster and on macOS, code base for JEDI and ufs-weather-model is as of Nov 17
- need updates for several JEDI and UFS repositories, this won't be easy or quick to integrate
- need new test case for ng-godas app (due to cice updates)
- in particular: UFS has newer version of FV3 dycore that is incompatible with the code in fv3-jedi-linearmodel (will discuss in a meeting on Fri Dec 2)
- spack-stack
- nearing the release of spack-stack 1.2.0 with skylab-3.0.0 environment
- updates to many libraries, especially eckit, fckit, atlas
- a test release was installed on S4 in Dom's space, can be used for testing
- we created a spack-stack slack channel in the JCSDA slack workspace, open for everyone - please ask Dom to be added if not already invited
Algorithms - Anna
At the last JEDI Algo meeting , Steve V. Presented on the VADER code spring, and Sergey presented on the (lack of) scalability of JEDI GETKF.
Memory optimization PR today
New interpolator post-processor has been created that can output on lat-lon grid
PR for Saber-last Monday Steve Vahl made a presentation on VADER, and Sergey Frolov talked about the current results and issues with scalability with fv3-jedi LETKF.
Generic development PRs recently merged or under review:
- Memory optimization for the IncrementEnsemble in EnVar
- Generalization of the Interpolators in oops; added new postprocessor to write out fields on lat-lon grids
- SABER block for localized ensemble covariances
PR - Option to use control - member
PR for Var BC
PR for custom cookbook for VADERin LETKF to center the ensemble
- VarBC preconditioning for primal minimizers
- Capability to specify custom cookbooks for VADER in yaml
Partner updates
None.