CSV Header Definition Proposal Sandbox
We have currently agreed that XML would be the format of choice.
Three existing schemas have been suggested:
- Transducer Markup Language (TML) is a language for capturing and characterizing not only data from transducers, but information necessary for the processing and understanding of that data by the eventual recipient of the transducer data. Both sensors and transmitters can be captured and characterized within TML, leading to the use of the term “transducer” rather than “sensor”. TML handles not only static but also streaming transducer data. TML permits the data stream to handle live transducer data both being added to the stream and being deleted from the stream.
- SensorML provides standard models and an XML encoding for describing any process, including the process of measurement by sensors and instructions for deriving higher-level information from observations. Processes described in SensorML are discoverable and executable. All processes define their inputs, outputs, parameters, and method, as well as provide relevant metadata. SensorML models detectors and sensors as processes that convert real phenomena to data.
- netCDML NcML
- Excellent for time series data and metadata. Could be done real-time over the wire, but not designed for such.
Seems if we want to go with TransducerML, then we should replace our entire CSV scheme with that. I have a difficult time locating good examples, and ones that are a minimal set for both TransducerML and SensorML (better examples for SensorXML exist). Both TransducerML and SensorML seem heavy-weight, NcML is light-weight.
Chris currently proposes using the netCDF NcML schema with some slight modifications.
Based on this CSV string:
Pure NcML example:
- The netCDF attribute should probably be replaced.
- The global attributes in this example, ProjectName and Platform, would be optional.
- Other variable attributes could optionally be included (e.g. missing_value, valid_range, standard_name).
- This could also be used for ASCII data distribution post-flight.
A post-flight header might look like: