Issue 2 - Comparability of CHAMP and COSMIC and other GPS satellite systems

The CHAMP-COSMIC comparison (or CHAMP-SACC) should not give results similar to the early COSMIC-COSMIC comparisons because the separations in time and space are much larger, thus introducing significant real differences, which vanish only when those separations become very small, as shown by the attached UCAR figure. That's what makes the early COSMIC data so unique. (If I had any say in the matter, I'd keep two satellites close together for the remainder of the mission simply to exploit this unique power to learn all we can about the properties of the measurement.) Also, we expect the CHAMP and SACC data to be somewhat noisier due to lower antenna gains and other limitations (e.g., no open loop). As to whether GPSRO is close to being a benchmark, that would seem to depend on how slightly different those differences are between groups. I would tend to look at the three groups most closely in agreement to judge that. Not everyone is at the same level yet in their analysis but some may indeed be approaching benchmark quality. What is the standard? Biases consistently <0.05K?

The principal reason for the larger differences is the larger separation between the observation pairs when CHAMP is considered. Subsetting by separation distance is needed, accounting also for the relative orientation of the occultation ray paths. The analysis suggested in this issue should be performed to set an upper bound on mean differences due solely to satellite system, independent of pair separation.

Response from Ben Ho

My response and several figures for my response to this issue are linked below.

In general, results here agree with comments from Tom, Tony, Seth, and Jim.

Here I present
1. Differences between CHAMP-COSMIC and COSMIC-COSMIC comparison
2. The comparability of CHAMP to each COSMIC receiver
3. The comparability of CHAMP to COSMIC data with different operational modes

To best reduce the CHAMP-COSMIC temporal and spatial mismatch, I use CHAMP vs. AMSU
and COSMIC vs. AMSU to indicate the compatibility of COSMIC and CHAMP.

Figs. of Detail Response
Detail Response

Results here show that the mean dry temperature difference between CHAMP and COSMIC is, in general,
less than 0.1 K. By using much tighter (in terms of time and location) CHAMP-AMSU pairs and COSMIC-
AMSU pairs,we found that the mean difference between CHAMP and COSMIC is around 0.07 K.

Consolidated Comments from Gutman, Yoe and Reale

Seth Gutman's Response
One reason the mean differences and spread seem larger when we compare CHAMP and COSMIC is because the CHAMP and COSMIC spacecraft almost never observe the same air mass at the same time and in exactly the same way. This is because CHAMP's orbital inclination and altitude is about 87o and 450 km respectively, while the COSMIC satellite orbits are inclined 72o and their altitude is about 800 km. Even if the occultation of the same GPS satellite occurred simultaneously, the different geometries should present a different cut through the atmosphere, and the path integrated refractivity should be well correlated but still different. The fact that we get excellent agreement from two dissimilar spacecraft separated somewhat in time and space, and looking at the same (or even a different) GPS satellite from a different azimuth and elevation is to be expected if the observing system is functioning as anticipated, the assumptions used to make the retrievals are reasonable, and the atmosphere observed during one occultation is similar to the one observed by the other spacecraft.

Jim Yoe's Response
Right - these are comparisons of limited opportunity, with arbitrary tolerances prescribed for closeness in space, time, and orientation. Since these tolerance criteria are looser than those for the 3000 early COSMIC pairs, the expectation is that the differences between the sampled atmospheres for each pair is likely to be larger.

Tony Reale's Response
Similar capability for NPIVS to collocate with historical CHAMP, COSMIC "retrievals" versus each other and collocated ground truth (sondes, ARM site profiles ...)

Response from Jens Wickert

CHAMP has some specifics due to the fact, that the satellite clock is adjusted every 1s (necessary for the magnetic and gravity measurements). This leads to some disturbing effects compared to data from other missions. The different handling of these effects also is probably one of the causes for the observed differences between retrievals from different groups at higher altitudes, where the signal is small and disturbing effects are more significant. I would expect better agreement at higher altitudes of results from different groups for e.g. COSMIC.
Nevertheless, even for CHAMP we have very good agreement between different centers (0.001% in refractivity). This shows that we are already very close (or are already) to be a benchmark systems at least for the refractivity. I'm sure that the differences between the centers can be understood in detail when investigate this carefully.

Comments from Rob Kursinski

I agree with the comments above about the differences between CHAMP and COSMIC in terms of both temporal and spatial proximity and air masses and differences in the GPS receiver hardware and software making the measurements.

CHAMP is very nice in that it is easily the longest of all GPSRO records to date. CHAMP is hurt by the lack of open loop signal tracking. In working with Chi AO, we stumbled onto some systematic regional and seasonal differences at low latitudes between CHAMP and COSMIC that are larger than the formal uncertainties in each. The bias is apparently associated with CHAMP systematically not tracking the occulted signals as deep into the lower troposphere as COSMIC for certain types of profiles. Interestingly sometimes the CHAMP biases relative to COSMIC are sometimes higher and sometimes they are lower. This results in regional and seasonal systematic biases in the CHAMP results at low altitudes caused by the deficiencies in the closed loop signal tracking. They don't appear to be present or at least they are very much subtler in COSMIC.

I'll try to post some examples.