The main idea is to create a work plan that follows the main Agile recommendations. Among those, we aim for code that will be working at all times and we prefer to break the plan in more smaller units of work that can be tackled independently rather than fewer large units of work.
Where we are
The first stage of the polymorphic observation operator from GMAO has been merged into the GSI trunk. The main base class for the observation operators has the following methods (data members not shown):
m_obsNode interface
type, abstract:: obsNode procedure, nopass:: header_read ! read a header procedure, nopass:: header_write ! write a header procedure:: init ! initialize a node procedure:: clean ! clean a node procedure(intrfc_mytype_ ), deferred:: mytype ! return my type name procedure(intrfc_setHop_ ), deferred:: setHop ! re-construct H procedure(intrfc_xread_ ), deferred:: xread ! read extensions procedure(intrfc_xwrite_ ), deferred:: xwrite ! write extensions procedure(intrfc_isvalid_), deferred:: isvalid ! validate extensions procedure(intrfc_gettlddp_), deferred:: gettlddp ! (tlddp,nob)=(sum(%tld*%tld),sum(1) end type obsNode
Where we want to go
The long term goal of the refactoring is that the base types for observation related classes will have the following additional methods for:
- Computing observation equivalent values from interpolated model values, accounting for bias correction
- Apply quality control to obervations
- Create the observation error covariance matrix
- Apply the observation error covariance matrix
- Create the observation operators TL/AD and/or Jacobians
- Apply the observation operators TL/AD and/or Jacobians
These methods might not all belong to the same class, for example, it is very likely that there would be a class to represent the observation error covariance matrix and that would include the methods related to that aspect. However, for the short term planning and refactoring, it is enough to consider what methods will be needed that relate to observations.
These methods will be:
- Independent of the horizontal model grid
- Independent of all global variables
Unit tests will be developed and implemented with each method.
Proposed steps for GSI evolution
The steps below are a path towards the final goal described above, not an end in themselves.
Step 1: Mild touch up/improvements to initial version
Task 1.1: Reconcile changes with latest trunk
Task 1.2: Reduce obs-types by using extensions of base types
Task 1.3: Remove transitional aliases to yobs, obsHeadBundle, thead, ttail, etc
Step 2: Add existing methods to polymorphic structure
Task 2.1: Add int & stp methods:
- layer out link list operations
- implement %TLop & %ADop
- cast into array to reduce procedure-call, type-casting and de-referencing
Task 2.2: Add setup method
- layer out link list operations
Description of the work: This step comprises moving the existing setup*, int* and step* into the polymorphic structure. The interfaces of the routines will need adapting but the contents of the routines is unchanged.
What it gives us: A starting point for the rest of the refactoring work.
Step 3: Add intermediate methods
Task 3.1: Add interpolate method and associated tests
Task 3.2: Add observation operator method and associated tests
Task 3.3: Add quality control method and associated tests
Task 3.4: Add bias correction methods and associated tests
Task 3.5: Add observation error covariance methods and associated tests
Description of the work: This step involves splitting the methods created in step 1 into the components that will be needed to build the UFO and JEDI. The most important aspect will be the design of the interfaces for the new methods. The data structures that are passed through the interfaces will be designed and encapsulated. Only minimum required changes inside the methods will happen in this step. The focus is on what operations are to be performed, not how.
What it gives us: Interfaces so that work can continue in parallel on both sides of the interfaces in the next steps.
Step 4: Create hierarchy of observation types and factorize code between sub-classes
Task 4.1: TBD
Description of the work: This step focuses on identifying commonalities in the methods for different observation types and moving common code up in the class hierarchy.
What it gives us: Source with minimal code duplication.
Step 5: Eliminate methods that can be replaced by generic code outside the hierarchy
Task 5.1: Use generic ESMF-based interpolation and remove the interpolate method from the polymorphic structure.
Task 5.2: TBD
Description of the work: Modify the higher level code to call generic implementations of algorithms that can be implemented at that level.
What it gives us: GSI observation-related code inside the UFO structure.
Comments
Steps 3 and 4 can happen at different speeds for different targets that will have been identified.
It is very important that at every step the code is integrated in the GSI and works. Merging in scientific developments happening in the GSI outside the project will be a continuous task throughout the refactoring. Refactored code will be delivered often in small bites.
Scientific developements that require significant code restructuring should be discussed with the project team ahead of time and coordinated.
Throughout the work, targets for the next steps will be identified and discussed within the team.
Work on IODA will interact with this part of the project. As I/O have already been moved into the polymorphic observation operator most of the work will be fairly isolated from what is proposed here. However, communications between the two parts of the project will be important.