Francois H led a discussion about upcoming model interface changes related to Atlas. Francois started with a summary of the changes and impacts on the model interfaces in this presentation:

Highlights from the presentation include:

  • Atlas helps by presenting model grids and data in a generic fashion
  • Updates impact the Geometry, State and Increment classes in the model interfaces.
  • For models that use a grid that does not have a corresponding built-in grid in Atlas, a new application called MeshBuilder has been added that will help with the conversion. 
    • The primary effort when using MeshBuilder is the construction of the connectivity data structure.
    • See slides for details and examples
  • A coordinated effort will be required to prepare and merge the PRs for this effort (likely to take place in October, 2023)
    • Please see the branches feature/atlas in oops/saber/fv3-jedi for the changes in OOPS generic code and model interfaces. These branches are in a mature state but not entirely complete yet.

Moving forward we can hold discussions in this GitHub discussion: https://github.com/orgs/JCSDA-internal/discussions/62. Note that there is a link to a design document included in the discussion page.

A great discussion ensued and the meeting was recorded. Contact Phillip Gibbs for gaining access to the recording. Here are highlights of the discussion:

  • spack-stack versions of atlas, eckit, fckit, fiat and ectrans will need be upgraded to newer versions (this is being tracked in ZenHub)
  • There may be some cases where the new Atlas interface will enable reducing current memory usage (eg, reduction of fieldsets)
  • In cases where it is necessary to provide a mapping between x,y space and lat,lon space, the requirement for this mapping may be eliminated.
    • This mapping will be covered by the specification of the connectivity
  • If you utilize MeshBuilder, for now the only way to pass those results to a subsequent process is to dump the information into a file and have the subsequent process read it out of the file.
  • Francois demonstrated using the MeshBuilder with a walk through of a couple cases
    • For now, supported polygons include quadrilaterals and triangles.
      • Pentagons exist in Atlas but are not utilized in the MeshBuilder
      • For now hexagons are not supported in Atlas
    • There exists an option to have Atlas compute the connectivity which utilizes Delauney Triangulation
      • However, it's not generally recommended to use this option
      • Creates a lot of triangles
      • Creates rough boundary edges and halo regions
      • Not repeatable when the number of MPI tasks changes
  • Atlas has recently been updated with a fix that follows conservation rules when computing areas
  • A fair amount of time was spent on the topic of staggered grids
    • For the goal of keeping the algorithms generic in JEDI, the winds need to be at the grid centers.
    • Note that the models do not need to change their grids, the moving of the winds to/from the grid centers can be done in the model interface.
    • The group thought this was reasonable and do-able, but some concerns were raised about having to do too many interpolations to meet the winds at the grid center requirement
  • The updates in the new Atlas interface should be do-able for regional models
    • Be careful to treat the lateral boundary conditions differently than the "ghost" points. Ghost points are related to MPI partitioning.
      • Francois did an interface for fv3 regional model, and he ended up creating a halo outside of Atlas containing the boundary conditions and then passed in that halo to Atlas.
  • Once the new Atlas interface is in place, there are plans to do optimization work in JEDI which should benefit everyone.
  • It was noted that the grid going from Atlas to Saber is currently a point cloud so this will need to be addressed.
  • A handy technique for verifying the correctness of a new model Atlas interface is to plot the grid
    • Francois used a technique of inserting a call to plot the grid in the JEDI code which can be visually inspected after the run.
      • Used a library called gmsh
      • See slides for details
    • Dan added that we may want to add a scatter plot to the verification methods. This could be useful to see if the right MPI task is picking up the right points.
    • It might not be possible to use a more quantitative approach (than plotting with visual inspection) to verify correctness. It's quite difficult to know what correct means.
  • No labels