Crossing a new frontier in hydrometeorological data management with an Integrated Sensor Data Management System (ISDMS)

Image credit: IAHS


Recent advances in the electronics and software engineering industries have positively impacted the development of sensor networks for sensor data acquisition. However, the data management aspect of the process has failed to keep pace with those technological advances. Current hydrometeorological sensing solutions often suffer from a lack of homogenous metadata descriptions for both the instrumentation and the data collected. These shortcomings may be attributed to the fact that the current hardware and software solutions are not vertically integrated (i.e., they do not mesh seamlessly). Typically, software solutions to support the data management aspect are developed to meet the constraints of the hardware (i.e., data loggers) which are not designed nor programmed to capture, operate on, and serve observations and accompanying metadata in a network or over the internet. This makes sensors and sensor networks agnostic to each other and thus inherently creates the conditions for a heterogeneous mix of different schematic, semantic and syntactic systems. Such inconsistencies increase considerably the workload in the sensor post-deployment phase and human error in the data management system.

This paper introduces a fully integrated data collection, data management and data curation solution named Integrated Sensor Data Management System (ISDMS) designed to a) overcome the sensor-and-data management schism in hydrometeorological observation systems and b) involve minimal human effort and expertise for its operation. It is composed of a) a software framework that supports metadata capture at the outset of the sensor deployment phase, data management, access, analysis and publication, to name a few and b) a sensor platform (hardware) named TranscodX. This latter raises itself above dataloggers, because of its capability to capture, process, and stream sensor measurements and corresponding metadata to an embedded standard data management framework. TranscodX further supports the capture, integration and processing of community-defined semantics and syntax using controlled vocabularies needed to enable information systems interoperability. Because of its extended capabilities, it is labeled a Transcoder rather than a data logger.

New York City, NY, USA
Dr. Paul Celicourt
Co-Founder & Research Scientist

My research interest is in Hydroinformatics, particularly the development and application of hardware and software technologies that resolve the chronic schematic, syntactic and semantic heterogeneities in earth sciences data sources.