Remote Sensing Radial Data Exchange Format
(Proposed for Consideration in the ABR process for FY2008)
There is a real need for a next-generation Remote Sensing Radial Data Exhange Format. The older formats in use ("Universal Format" and
DORADE) are either inadequate to address user needs, or never received
wide community acceptance. New sensor development efforts by EOL make
examination of Exchange Format issues very relevant, as timely
development would greatly enhance the utility of data from the new
sensors. There would be direct benefit to the User Community,
particularly University PIs who are granted use of EOL Facilities.
The Remote Sensing Radial Data Exchange Format (RS-RDEF) was a topic
for discussion at a special "town meeting" at the October, 2005 Radar
Conference, and received good community support. Preliminary work
began on the Format in mid-FY06, and resulted in very preliminary code
(the FORAY code and associated API development by Dennis
Flanigan). The RS-RDEF was submitted for consideration to the SoO
process in Summer 2006, but was returned to the Facility for action,
and failed to gain traction.
Action really needs to be taken on the RS-RDEF. In developing the
CAPRIS proposal, it was obvious that we need to define a data format
to be used by future lidar instruments. The HCR development will
require a data format. Changes to S-Pol and cooperation with CHILL
will benefit from a clearly defined Exchange Format. The REAL
platform would benefit from establishment of such a format, as would
MAPR. Our current remote sensing data analysis software (solo, the
translators) is aging and becoming unsupportable (remaining life on
the order of 2 years); it makes sense to define a new format prior to
creating the follow-ons to those software packages. The VOC
development will be simplified with a new exchange format.
Indications are that Community support remains strong. Jason Fritz of
CSU-CHILL has obtained preliminary FORAY code from Dennis Flanigan and
used it to produce CHILL NetCDF data that meets specific EOL conventions.
Jason also incorporated components of FORAY into CSU's version of
the Fabry Refractive Index code. NCAR/RAL has used FORAY in
S-Pol's RVP8 sweepfile generation and in the auxiliary product
generation software used in the KFTG 88D radar.
While most of the platforms that would directly benefit are RSF
platforms, the development of RS-RDEF (wonderful acronym, eh?) is
properly the job of EOL's CDS. CDS has been set-up to address
cross-facility issues, encourage commonalities in data management
between the Facilities, and bring expertise to bear on data and data
management issues. External users do not view EOL as a sum of
Facilities – when HCR is requested from RAF, EOL will better be able
to produce usable data from the platform if there is a clearly defined
format.
More detail is available. Cost estimates are provided for FY08. It
is expected that some work will be required in FY07 in order to take
advantage of further user input at the EOL Users' Workshop.
Cost estimates (preliminary, through Dennis Flanigan):
– work on precise format convention specifications
6 weeks SE
2 weeks AS or Sci
– work to complete FORAY and an initial API
10 weeks SE
2 weeks AS/Sci