Yannick: Introduction
Phil: starts the presentation
Summary of Phil's presentation.
This is on-going work.
Data are populated in Memory (IODA) and passed to UFO.
We will focus on IODA and more precisely on ObsSpace
Changing the internal structure of the ObsSpace. New design.
2 prototypes candidates: Column Priority or Raw priority
Row priority occurs only once
Decided to go with both
Cons: Column prio: what is now. Raw will be better. Avoid boolean (for C++) and unsigned typpe
Flexible new features: able to slice and dice the ObsSPace. Imagine the ObsSpace as a spreadsheet
New code will be wrapped in the existing ObsSpace. All existing functions will still be applicable.Coding will start soon. We are just finishing the design.
Then the wrapper will be removed, if this is not too much work
Integration: Will start soon to move the code into IODA repository (see slides for schedule). There will also be optimization work.
Discussions.
Steve: Will not change the IODA files. The interface (yaml) will stay the same. The switch will be transparent.
Yannick: That will be nov
??: Can you give details about the raw approach.
Phil:
Steve: Coplumns corresponds to variables. Raws correspond to locations. Currently we are manipulating variables (raw). So raw priority is advantageous
Phil:
Zhiquan (Jake) Liu: do we expect new implementation will speed up something and/or less memory use?
Phil: I am optimistic it will be.
Steve: Slice the pieces that we want to pick and pass only what is needed to UFO
Phil: will drop stl maps, and keep only vectors. Searches are faster with vectors rather than maps.
Christian: What about time dimension.
Steve: Time window filtering will remain. Time binning will also stay. We are only touching the storage. Datetime is like a column. There will be move, remove and write columns
Phil: There will be permissions, ie some columns will be read only
Steve: original obs can be put read only (in a different column) to be used later as verification. Today, we don't have the read only check.
Jake Liu: What about the I/O?
Steve: we currently have read an write of IODA files and ODB. That will stay. Working on a BUFR read, should be available in the next months.
Yannick: Anymore questions. We welcome visitors from all partners. See you next week.