Ticket #14 (task)
Opened 6 years ago
Last modified 6 years ago
Define specific QAQC tests
Status: closed (fixed)
| Reported by: | cbc | Assigned to: | haines |
|---|---|---|---|
| Priority: | major | Milestone: | 2006 SECOORA documentation contract |
| Component: | qaqc | Version: | 3.0 |
| Keywords: | CDL | Cc: | |
e.g., document definitions of specific tests (two levels here, general and detailed plus all references)
Change History
07/12/06 11:40:39: Modified by cbc
- summary changed from Defein specific QAQC tests to Define specific QAQC tests.
07/12/06 14:53:05: Modified by cbc
- status changed from new to closed.
- resolution set to fixed.
07/12/06 16:40:14: Modified by cbc
- cc deleted.
The is documented in our sprint notes:
Definitions of QC tests
Data availability: Unreported data are usually caused by communication failures or bad or partial data transmission or problems related to data acquisition. These include parity bit errors or parsing errors. This includes what NDBC describes as Transmission Errors (Section 2.2.2).
Sensor range (widest range) is the sensor manufacturer’s stated measurement bounds (minimum and maximum) and is range of possible valid measurement. This should be performed prior to any time continuity or rate of change test. This is intended to represent the physical limits of the sensor. The main issue has been with relative humidity because it is derived.
Gross range (middle range) Regional sanity check that a measurement falls within an annual range based on knowledge of the general location of the platform.
Climatological range (narrowest and applied to specific time or geographic range). Historically derived site specific data range. These include seasonal or monthly ranges derived at a specific platform for several years. Range is monthly mean climatology, compute std dev, and range +/- n*std dev.
Rate of change test is intended to check for physically impossible changes with time but can be refined for general areas and include frontal passage limits. NDBC refers to this test as the Time Continuity (Section 4.1.2) and does not account for very rapidly changing conditions such as during the passage of tropical cyclones and severe extratropical storms.
Duplicate sensor test if possible using NDBC algorithm (Section 4.1.4).
Nearest neighbor check is comparison with nearby observations over an extended period of time. “Nearby” can be related to error covariance. This provides a flagging or notification service for persistence of erroneous observations relative nearby stations. Need to retain the difference and how long a large difference has persisted to provide information back to data provider that a sensor or package has degraded. For the code sprint, provide the mechanism to perform check with the ability to change what “nearby” stations the check uses to start evaluating error covariance.
Model comparison is absolute difference of interpolated observation with model. For quality assessment of the observation is not very useful. This is more useful for a model validation and model skill assessment. We will not try to tackle this one for the code sprint.