The MPAS-O team discussed the outline for KPP that Mike sent along previously. 

1. Passing a single column at a time is OK, as long as 2. can be accommodated.

2. The attributes to vmix_input_type and vmix_output_type are all pointers. The reason for this is the following: First, if the attributes are allocatable arrays and are required intents in/out, we should just pass the native fields directly and thereby eliminate a copy. Copies could turn out to be a large overhead. Also, pointers can accommodate models regardless of array ordering (i.e. regardless of whether the vertical index is first or last). There will likely be performance issues depending on whether the k-index is first or last, but this will be an issue with copies or pointers.

3. We should be careful about namespace conflict. While we probably intend for all of the module to be private except for a few interfaces, we want to make sure that these interfaces are not so generic as to conflict with routines that already exist in ocean models.

Let us know your thoughts on this.

  • No labels

5 Comments

  1. Michael Levy

    Conceptually, it doesn't make sense to me to pass pointers -- I want to do a full copy because then I know exactly how the data is stored. I talked with Michael Duda a few months ago, and he mentioned what I think he said was the indexing scheme MPAS uses: rather than forcing users to use either (i,j) or (j,i), we can use (i1,j1,j2, i2) and then either set i1=j1=1 for (j,i) or i2=j2=1 for (i,j). I think it would be smart to incorporate something like that eventually, but the first goal is to just get something working so for that I'll be using the POP indexing. Once we have a working prototype, I want to go back and play with the indices, but still leave the memory copy in place. While I see where you are coming from regarding the overhead, hopefully the ever-increasing computer size will let us scale up to a point where the memory isn't a bottleneck.

    Regarding point #3, I completely agree. I'll do my best to avoid generic public names, and if I do inadvertently create one I'll resolve it on the vmix side rather than expecting one of the models to change any of its names.

  2. Todd Ringler

    I think that we should talk about this on the phone. I will be the first to admit that I am not an expert wrt to pointers, but our two resident experts are confident that pointers will minimize the burden on both the calling model and the vertical mixing module. The piece that I am more confident with is the memory issue. On a per flop basis, memory is getting increasingly smaller --- thus making duplicate arrays for the sake of convenience increasingly expensive. The flip side of this is that if we end with hard arrays, then we should just keep them "bare", pass the directly and not confuse the compiler by packaging them into a derived data type simply to make the interface look clean.

    To reiterate my concern from the Breckenridge meeting, we have to have a design process here. That in no way prohibits the prototyping that you are now pushing on, but it does suggest that we have to willing to "throw away" the prototype code in order to produce software that meets the design requirements. With a document in hand, we then have a "meeting place" to discuss various approaches while fleshing out requirements.

    We are willing to put together some pseudo code to illustrate how the pointer approach would work. Should we do this?

    1. Michael Levy

      I would definitely be interested in seeing pseudo-code. If it's any help for the generalizing, POP uses

       viscosity(nx, ny, nlev, nblocks)

      and MOM apparently uses something similar:

       viscosity(nx, ny, nlev)

      they just don't use OMP blocks. In fact, given the column-major nature of Fortran, each block in POP would be contiguous memory so pointers would be identical in these two cases. But what if a model indexed with

       viscosity(nlev, ny, nx)

      or

       viscosity(ny, nx, nlev, nblock)

      or something else structurally different?

      As for the prototyping / design process comment, I think we're on the same page. I suspect the final version of the code will look very different from what I have now, but I'm hoping that having the prototype during the design process will help highlight things that should or shouldn't be done a certain way (more likely "shouldn't"). Maybe we should try to schedule a telecon for early next week?

  3. Todd Ringler

    I wanted to check to make sure that we are still intending to have vmix work on one column at at time? It looks like the POP/vmix code is working on a whole block at one time.

    1. Michael Levy

      I thought we had decided to send vmix an array of columns, but now I can't find any mention of it in my notes. Yet another argument for having the design document already, but I have no problem (outside of what Steve talks about in horizontal smoothing and such) with moving all the column loops to the model / stand-alone driver.