Issue 1: Comparability of data from different COSMIC satellites
Claims of 0.02-0.05 K precision in retrieved T at all vertical levels (surface to 30 km) seem based on average differences between 2 satellites, based on samples of ~3,000 occultation pairs. These pairs are presumably from different regions, times of day, seasons. Rather than averaging the differences, some indication of the spatial and temporal patterns of mean differences is needed, as well as the statistical distribution of differences. Mean differences may be <0.1 K, but what about those 0.5 to 1.5 K difference values? And how do these statistics look for retrieved water vapor? Basically, is the averaging process masking important information on measurement or retrieval uncertainty? Analyses subsetting the data into latitude bands as well as land and ocean subsets could show that this reproducibility is not regionally dependent.
Response from Kevin Trenberth:
In Fig 8 of the BAMS paper the mean and s.d of the differences is given, From about 8 to 22 km altitude standard deviations are small <0.2% (about 0.5C), but at lower altitudes and higher levels the values are larger. Presumably one reason for larger differences at lower levels is the water vapor contribution and thus heterogeneity of clouds etc come into play, and multipath tracks come into play, especially in the tropics. I agree that a closer examination of the larger differences is warranted. Retrieved water vapor depends on an assumed temperature from elsewhere and I don't think that is a strength of GPSRO.
6 Comments
Tom Yunck
Since the close separations during the 1st ~6 mos of COSMIC provide a unique dataset for performance analysis, all independent pairs should be examined, not just the one pair described in the BAMS paper. That should give several thousand more samples and reveal instrument-specific dependencies, if any. The means (<0.05K) and the sigmas are of course different statistical properties, both well characterized in the analysis, and no-one is likely to confuse the two, so I'm not sure what the issue is here. No-one is claiming individual retrievals are good to <0.05K, just the biases, which alone is a REALLY big deal. If that holds up, the value of RO as a climate benchmark and long-term calibrator is assured. Individual retrieval sigmas of <1K from ~8 to 25 km, if confirmed, also lead the pack among space techniques. By all means let's examine this further with exhaustive analysis of the unique early COSMIC pairs. If those numbers hold up they make for quite a potent combination.
Tony Mannucci
The claim of 0.02-0.05 K precision in retrieved T from the surface up to 30 km is a statement about so-called "dry temperature", not actual temperature. More precisely stated, it is a statement about refractivity precision.
As a first step in our discussions, I suggest we focus on benchmarking refractivity. Many of the issues raised here apply to refractivity. Focusing on refractivity simplifies the initial discussions. In my opinion, refractivity is a useful climate benchmark. I am not prepared to comment on suitability of GPS water retrievals as a climate benchmark.
Subsetting the pairs by region, time of day, season etc. is an excellent suggestion. Statistics should be calculated for various subsets. Careful attention must be paid to statistical significance as the number of pairs in each subset becomes smaller. Analyzing the distribution of differences and understanding the outliers is important also, as suggested in this issue. A concrete program of additional analysis should be an outcome of the workshop. NOAA experiences will be appreciated in guiding further analysis.
Ben Ho
Response from Ben Ho
To address this issue more specifically, here I provide several extra figures to quantify:
1. The comparability of COSMIC FM3 to COSMIC FM4 at different latitudinal bands
2. The comparability of COSMIC FM3 to COSMIC FM4 at different latitudinal bands over land and over ocean
3. The comparability of COSMIC FM3 to COSMIC FM4 at different local times
Figures to quantify the comparability of water vapor from different COSMIC satellites are also attached.
More explanation is in "Detail Response".
You may point to the link and click the right button of your mouse and select "open link in new window" to view it.
Figs. of Detail Response
Detail Response
In general, temporal and spatial comparability of dry temperature from COSMIC FM3 and FM4 satellites are quantified here where we have more FM3-FM4 pairs than other pair combination. The global (both lands and oceans) mean FM3-FM4 differences in dry temperature are between -0.1 K to 0.1 K from the surface to around 30 km.
Seth Gutman
The following are Consolidated Responses from Gutman, Yoe and Reale
Seth Gutman's Response
I did not attend the COSMIC breakfast meeting in New Orleans, but I assume that the results of "0.02-0.05 K precision in retrieved T at all vertical levels (surface to 30 km) for about 3000 occultation pairs" were made soon after the constellation was launched and a number of the spacecraft in the constellation were in close proximity. This result is significant because it indicates that the results of the physical retrieval process are (for the satellites used in this study) essentially platform independent. This provides direct empirical support for the notion that GPS-RO results are statistically reliable and, in theory, repeatable. Since we're talking about the precision of the retrieval, the physical locations of the measurements or when the measurements were made are not important. This is not to say that the statistical outliers are not interesting or important, since they may tell us something about how the observation is being made. For example, did the outliers occur when the pointing angles of the spacecraft differed significantly? If so, this could tell us something about how the orientation of the spacecraft affects the measurement and may point (no pun intended) to a way to correct the retrievals for multipath or antenna pattern differences. With respect to the precision statistics for retrieved water vapor, the results should not look very different from the temperature results. This is because in order to retrieve water vapor you would have to fix temperature and pressure for the retrievals from both satellites presumably in the same way.
Jim Yoe's Response
I agree with Seth, the key here is precision and reproducibility that are unique among remote sensing systems. The question of geographic/seasonal distribution is not particularly relevant to determining the significance of these results -any stratification of the results should be based on the extent to which the sampling volumes of the paired measurements differ in time, location, and orientation. This is analogous to the attempts to characterize the internal consistency of radiosonde soundings by putting paired sensors on single balloons, and to the use of the Simultaneous Nadir Overpass (SNO) method developed by Cao and other NESDIS/STAR scientists to inter-calibrate comparable radiometric sounders on different polar satellite platforms. Even though all SNO match-ups occur in a narrow, high-latitude band, they provide a remarkable insight on the reproducibility of radiometric soundings, by having two sensors view the (nearly) same scene from the same perspective (through the same atmospheric volume) at the same time. I don't think the mean or rms differences determined for any IR or MW channels for which SNO has been applied demonstrate reproducibility comparable to that of these COSMIC data.
Tony Reale's Response
With respect to tropospheric T and H20 retrieval from COSMIC (their relevance for climate must be clarified), the averaging process certainly masks errors. For example, "preliminary" result show that a simple breakdown of collocated radiosonde and COSMIC moisture (and also T) profiles by "moist" versus "dry" reveal characteristics that are 'hidden" in the non-segregated set (see attached analysis as provide by my colleague Bomin Sun using 10-days of Collocated COSMIC and radiosonde profiles).
Jens Wickert
Response from Jens Wickert
I agree, that the early COSMIC measurements form a unique data set to learn on the accuracy of GPS RO. In addition we will see in near future a GPS RO tandem configuration flying. TerraSAR-X (launched mid 2007) and the hardware-identical TANDEM-X (foreseen for 2009/2010, German X-SAR satellites) both equipped with IGOR receiver and for and aft looking occultation antennas (identical with COSMIC antennas) will fly with distance of max. 30 km to get "stereo radar pictures" and will provide the opportunity for "parallel" occultations, recorded by both receivers. The data will cover several years and could be used for detailed investigations on the accuracy of GPS RO (Ionosphere, Solar cycle, day/night, seasonal and regional influence etc).
Rob Kursinski
The results Ben has posted at Mar 12, 2008 17:29 are very interesting and address many of the specific issues raised in Issue 1. I will also add as I mentioned to Jay Fein this morning that a serious effort needs to be made to understand the results Ben is showing in terms of their implications on our theoretical understanding of the GPSRO errors. It is very important that we refine/update our understanding of both the systematic and random errors based on these results and better understand what errors will be present in estimating the evolution of our climate. As I noted at OPAC-3, systematic errors are supposedly common and therefore canceling in the FM3 vs. FM4 differences.
I will try to comment a bit more on this at the meeting next week.