Data Pipeline

Data Quality Control for Continuous Oxygen Sensor Data

Our team developed a standardized data quality control (QC) pipeline designed to ensure we provide high-quality environmental data to regulators and stakeholders. This pipeline processes calibrated dissolved oxygen, temperature and salinity data from continuous loggers deployed in embayments throughout Buzzards Bay, MA and applies consistent, site-specific flags to streamline the quality control workflow. Users can customize flag thresholds to optimize the process for unique sites. This ensures flexibility without compromising consistency.

The QC pipeline provides interactive graphical interfaces that we use for data review and flagging. These tools ensure that high-quality and consistent QC-ed data are produced, even with multiple handlers of dataloggers and data.
Our team also developed an analysis pipeline that helps to detect water quality issues, calculates relevant statistics and formats yearly datasets for submission to the Massachusetts Department of Environmental Protection (DEP). This module will further improve the workflow, facilitate comparisons of water quality around Buzzards Bay, and simplify data submission. These QC and analysis pipelines will help deliver accurate and reliable data from environmental monitoring for decision-making.