Where we are

Access to observations data is very specific to each DA system.

Where do we want to go?

Use generic API for accessing observation related data.

What are the high level goals of IODA?

  1. In memory vs. disk vs. long term storage API?
    • IODA as input and output of the DA system
    • IODA for accessing data and visualizing it, doing statistics, verification, etc... (possibly interactive, running on desktop or similar)
    • IODA inside JEDI (running on HPC: MPI, threads, GPUs...)
    • Are we expecting IODA to hold long term archives or to live only within an "experiment"?
  2. Are we expecting the workstation and HPC APIs to be the same?
    • Seems nice but will add complexity to the interface for workstation use
  3. Will all systems in JEDI be forced to use IODA?
    • IODA is embedded inside a very limited number of JEDI objects (y, H, R and BiasCorrection). IODA as such it is not visible from JEDI.
    • Users coming with new observations will appreciate using an existing flexible (and efficient?) solution.
    • Users moving an existing DA system into the framework already have an observation data structure. Should we force them to re-write their codes? Would that be considered a low entry ticket?
    • Should the decisions to adopt JEDI and IODA be independent or a "all or nothing"?
  4. Should all observation types within a JEDI application be forced to use the same observation data structure?
    • Not a requirement from a JEDI design point of view
  5. Is IODA an interface definition or a specific implementation?

Proposed steps

 

Step 1: Implement common file format

Task 1.1: Implement common file format for diagnostic output

Task 1.2: Implement converter from current GSI binary files to common format

Task 1.3: Implement common file format for UFO and GSI input

Description of the work: Work on output for diagnostics has started but with slight variations between centers, this step implements a common definition of the output file format. That same format can then be used as input to the UFO (and GSI) instead of the current binary files.

What it gives us: Ability to share data files between centers.

Step 2: Get started

Task 2.1: Define where we want to go in the longer term.

Task 2.2: Write job description, advertise and recruit software engineer.

Step 3: Organize IODA meeting (Boulder, 1 or 2 days?)

Invite:

  • ???
  • ???

Goals:

  • Define requirements for IODA API
  • Define first steps of prototype implementation

Step 4: Design and implement IODA API

Task 4.1: TBD after meeting in step 3

Description of the work: This step...

What it gives us: Ability to...

Comments

Throughout the work, targets for the next steps will be identified and discussed within the team.

  • No labels