2020-10-05

Agenda Item 1:

General Update from Ben (tests and documentation)

Discussion:

Update by Ben:

I had a meeting with Daryl this morning to improve our coordination to improve the scattering tables. A year ago, Penn State delivered some scattering tables that were never implemented. It’s called CRTM-DS. We will take this work where they extended the tables to larger particle sizes.

For CRTM 2.4 I will find out this week when the code freeze will be. The final release will be on about October 21^st. I have some regression tests now. Some pass, some fail. This is good, we want them to fail. Ctest is really just for JEDI and UFO but we also have standalone tests and I will merge that tonight. And the only thing that is missing are the scattering tables. I didn’t get to that yet because we found a bug. The bug was we found some differences in the test cases in the seventh digit below zero. For now, we can get bitwise identical results for the same compiler if we switch on the IEEE compliance compiler flag. However, we still get different results for different compilers. It seems like the solution that was accepted in the past was to just print fewer digits, so I don’t know what to do there.

So, the missing pieces are scattering properties and documentation. We will spend some time next week on documentation in Latex. The JEDI team is using this “Read the Docs” website for their documentation, which I don’t like. We have a doxygen-to-latex converter that will grab comments from the source code and automatically generate documentation.

Any questions or comments on the 2.4 release?

Alright, why don’t we go around the room and have a brief update from everybody?

Result:

-       Regression test suite for the CRTM has been re-enabled on Github.

-       Both ctest and standalone tests are available.

Tasks:

-       Add Doxygen + Latex documentation to the CRTM.

-       Add Penn State extended scattering properties.

Responsible People:

-       CRTM core team

Deadline:

October 21^st

Agenda Item 2:

Cheng’s Update (Aerosols)

Discussion:

Cheng:

I am working on Yingtao’s branch trying to add netCDF I/O for CMAQ aerosols properties. To summarize, I think we need to make some changes to Yingtao’s code to make it backwards compatible. Is that okay with you Yingtao?

Yingtao: Sure, we can have a code review afterwards.

Cheng: So that’s what I’m working on right now. The other question is for Hongli. I’m wondering, for DA, will you be using both GOCART and CMAQ together? With the current setup when you initialize the aerosol table you can only initialize either GOCART or CMAQ. So, I am wondering if both tables would be necessary at runtime.

Hongli: I have never used the CRTM GOCART and I think in the GSI we never use GOCART or CMAQ at the same time.

Cheng: Ok, so I will only ensure that one aerosol table will be initialized for each GSI run.

Hongli: You mentioned you will update the code by the end of this week. Is your code compatible to the GSI?

Cheng: What do you mean, compatible?

Hongli: I am not sure if GSI can handle the netCDF aerosol files.

Cheng: You don’t need to worry about that, Yingtao has worked on creating binary files.

Ben: Any questions? I don’t think we need to be too concerned about the netCDF files. We will still support binary files until some unspecified point in the future.

Result:

N/A

Tasks:

-       Add netCDF files for CMAQ aerosols

Responsible People:

-       Cheng Dang

Deadline:

October 21^st

Agenda Item 3:

Daniel’s Update (GPU CRTM)

Discussion:

Daniel:

In the last couple of weeks, I was working on getting more kernels on the GPU, still using the check_crtm test case and this week I was switching to a better UFO test case. I can’t get it to compile with the existing code but I am trying to switch some of the routines into check_crtm. I have been talking to some of my colleagues here at NOAA and if I can run it in GSI, that would be easier for me because I am already familiar with it. But in the meantime, I will continue using the existing test cases.

Ben: You can have a deeper look at how the CRTM works and if you can find a different way to sequence things that would be beneficial. Your goal is to have a test program that runs over multiple profiles. I have a code that I haven’t made available to run test cases with a large number of profiles. Essentially it is based on an existing test but with the difference that some random changes are applied to an initial set of atmospheric profiles. I think Patrick also has a code that can read in different profiles.

Patrick: Yes, I have written a generic test case that can read in netCDF profiles, such as the ECMWF83 and UMBC48.

Daniel: In the check_crtm are the arrays big enough to get some benefit from GPUs?

Ben: Yes, because they are allocatable arrays. You can make them as big as you want.

Daniel: I would really need a really big test to fully occupy the GPU.

Ben: I am not really sure but there has been more of a real-world example that has been written for UFO.

Have you installed the fv3_bundle or just the UFO?

Daniel: Just the UFO.

Ben: I’ll find out something for you and take that as an action. I’ll contact Dan Holdaway, he’s the expert on fv3_jedi.

Hongli: I have questions for Yingtao. When I look at the GSI code there is no interface for the CRTM CMAQ. Can Yingtao provide me some driver for the CRTM CMAQ, so I can provide the aerosols then I can develop the GSI interface for CRTM.

Yingtao: I have a question: the current GSI interface to the CRTM, the driver is only calculating the radiance and Jacobian only, not the AOD.

Hongli: Yes.

Yingtao: What parameters do you retain from the CRTM? If it’s AOD then it would be simpler to do. If not then we would need to modify the current GSI driver.

Hongli: I just looked at the draft.

Yingtao: So, the CRTM resolution, there is an RTSolution. They already do have an AOD. Do you use that?

Hongli: Yeah, I will need to check. For old GSI I ran one case but I had some issues with the AOD part.

Ben: Seems to be a GSI issue specifically inside the CRMT_Interface.f90.

Hongli: Yes, I saw the aerosol table there, you may need to list 15 aerosol species there. I think another difference is that for the CMAQ even for one species there are 3 modes, accumulation, coarse mode etc. .

Ben: So, you will probably have to modify the GSI.

Yingtao: Hongli can you forward the driver program to me?

Hongli: Yes, that would be great. Another issue from my side, the CRTM only has 8 aerosol species but we need 45. My idea is to have a meeting with a CMAQ aerosol expert so we can discuss the mapping.

Ben: My recommendation is to make sure that all the technical elements in the CRTM work first and I’d be hesitant to call a meeting before that.

Hongli: Ok, I understand.

Ben: In general, our future plan is to avoid mapping as much as we can. This mapping stuff is just a real pain to manage and support.

Hongli: Sure. If we start to talk about the mapping we have some idea if the current 8 species is good enough for the new CMAQ versions.

Ben: It’s pretty obvious the answer is no. So, it is up to us in the group to expand the available species with a generic way of aerosol representation. Right now, we just get some pipes running with CMAQ, but modifying the GSI is outside of the expertise of the CRTM core team. But we are happy to support that.

Hongli: Sure, thank you.

Result:

N/A

Tasks:

-       Set up a suitable GPU test case for the CRTM with a large number of profiles.

-       Create a CRTM CMAQ GSI driver

Responsible People:

-       Daniel Abdi, Ben Johnson

-       Yingtao Ma

Deadline:

N/A

Agenda Item 5:

Edward Hyer’s Update (pyCRTM tests)

Discussion:

Edward Hyer:

I was able to do some work this summer using the pyCRTM from Bryan Karpowicz and I did some single column experiments to investigate the aerosol sensitivity in the CRTM. The tools are in a really great shape and once Cheng Dang can finish the aerosol tables in netCDF the single column experiments are essential for science.

Ben: have you modified pyCRTM?

Hyer: No, it is running as it was. I would point out that the pyCRTM installer includes a patch to the CRTM code. When you install pyCRTM it actually patches the CRTM.

Ben: You saw my email for the quarterly report. Maybe you can write a few slides about the pyCRTM workflow?

Hyer: No problem, I’ll provide an example.

Ben: Another update: Bryan has given me permission to add pyCRTM to the public CRTM repository. Any changes coming in through public will also mirrored in the private repository. That

Result:

-       pyCRTM is very well suited for sensitivity studies and scientific tests with single-column atmospheric profiles

Tasks:

-       Perform scientific studies with pyCRTM

Responsible People:

-       Edward Hyer

Deadline:

N/A

Agenda Item 5:

Ming Chen’s Update (CSEM)

Discussion:

Ming:

I think that the CSEM repository is already available. I didn’t hear any updates from Kevin about the STAR Github availability. But I think I can push a little bit more. Your cmake configuration is based on macOS?

Ben: There’s a UFO cmake build system. Yes, there will be cmake availability.

Ming: Is it necessary for me to provide another cmake build system? Currently I am just wrapping up the ocean <unintelligible> L-band and also the RSS model. I think there should be several very interesting results.

Ben: Are you gonna give a presentation on these results?

Ming: Probably at the next monthly meeting.

Ben: I think there are some problems at the L-Band with Jacobians.

Ming: Yes, I agree. It’s a lot about experience. It depends on the efficiency and the correctness and consistency in the code design.

Ben: Right, Patrick started to right some generic tests for TL. I think AD is a little bit more difficult.

Ming: Yeah, if I am required I can do it.

Ben: Just as a general principle as scientific programmers we should write tests for our code. My hope is to come up with some generic guidelines as to how a good regression and unit test should look like for the CRTM.

Ming: I think that in a computer system it’s good not to test everything at the end point to have scalability. In that regard unit tests are very good.

Ben: I have a series of regression tests and today I will finish the unit tests.

Ming: That’s okay I can ship some tests.

Ben: It would be a help for me if you can send me some of the tests you have written offline.

Result:

N/A

Tasks:

-       Make CSEM available on Github

-       Present CSEM results at CRTM monthly meeting

Responsible People:

-       Ming Chen

Deadline:

October 21^st

Agenda Item 5:

Patrick’s Update (Transmittance Coefficient Generation

Discussion:

Patrick:

So far I have computed apodized layer-to-space transmittances for IASI and the remaining part is to compute the CRTM regression coefficients.

Result:

-       Apodized layer-to-space transmittances for IASI have been computed.

Tasks:

-       Compute the CRTM regression coefficients

Responsible People:

-       Patrick Stegmann

Deadline:

October 21^st

Agenda Item 5:

Yingtao’s Update (CMAQ Aerosols)

Discussion:

Yingtao:

I am trying to upload those coefficients for ABI and I am trying to learn how to use git-LSF.

When I am trying to checkout a branch I get an error. It says it should be an LSF-link but it’s an actual file.

Ben: That happened to me a couple of times. There’s a file called .git_attributes. It’s not a big problem. It just means this file should have been added to LSF.

Yingtao: I cannot change branch with this error.

Ben: You can just stash it. If you want to fix it, just type ‘git add’ and commit.

Yingtao: Okay.

Ben: I don’t see any issues right now. On which branch are you?

Yingtao: I am in my CMAQ branch right now.

Ben: Cheng, did you get all of your questions answered with Yingtao?

Cheng: We have one major question: How did you test your CMAQ branch?

Yingtao: I have one big messy test suite which is not suitable for publishing but I can forward it to you.

Cheng: Another question, I know that you combine the CMAQ and GOCART properties into one file?

Yingtao: No, I have two separate binary files. I shared one for CMAQ and the GOCART one is in the repo.

Cheng: Another question, I know that you combine the CMAQ and GOCART properties into one file?

Yingtao: No, I have two separate binary files. I shared one for CMAQ and the GOCART one is in the repo.

Ben: I remember that in the cloud table there is an effective variance. Is something like that in the aerosol table as well?

Yingtao: I don’t know.

Cheng: I believe that is a new property in the CMAQ table.

Ben: There is also an effective variance for aerosols as well but it may be old. It shows up in some older code, but not in new code.

Cheng: If you check the LUT, effective variance is not a dimension.

Yingtao: Probably that dimension was used when they generated the coefficients.

Ben: I see it in some structures of some test codes. Maybe it doesn’t exist anymore. But if I recall, even in the cloud table it doesn’t do anything.

Yingtao: You mentioned at the beginning of this meeting that the guy from Penn state provided a new cloud table. How is that different from the current cloud table in the CRTM?

Ben: I am gonna share the slides real quick. This is a delivery from their group. They provided these ice properties. They have six different binary files and they expanded the range of effective sizes and they tried to improve the coordination with the GFS. Using these expanded tables, they improve the results in heavy precipitation. They call it the CRTM-DS, but they only use spheres. Part of the things that we promised for CRTM-2.4 is increased particle size. We would also like to have different shapes for the same size dimensions. I am meeting with Greg Thompson to see how we can do that with the Thompson microphysics.

Yingtao: Are you gonna include that in REL-2.4?

Ben: Yes, we will provide at least three extra tables from Ping Yang’s group, the CRTM-DS tables and the default tables.

Yingtao: How can you let the user chose?

Ben: Good question. Right now, the option is just to rename the binary file. I could probably come up with something.

Yingtao: The aerosol part has the same manual selection right now.

Ben: Yes, but there are only three of us right now to change that. You can help if you want.

Yingtao: No, I don’t have time.

Result:

N/A

Tasks:

-       Learn how to use git-LFS

-       Upload ABI coefficients

Responsible People:

-       Yingtao Ma

Deadline:

October 21^st