Introduction

In the following the official NO2 data product using processor 1.7, rcode ‘nvs1’ is compared to the rcode ‘nvs3’ of processor 1.8.

1.8 data might differ to the existing 1.7 data in several ways, wherefore a direct comparison can be difficult. Most importantly, the 1.8 calibration already revealed that there might be additional validity periods needed, which have not been accounted for with 1.7 calibrations. Moreover, the calibration process itself is different. Using 1.8, the calibration team can now use the E-MLE approach in which we have more confidence that the obtained slant column amount in the reference spectra is more closer to the truth, and the uncertainty during the calibration is reduced. Additionally, the splitting into tropospheric and stratospheric NO2 also allows a better estimation of the the air mass factor and uses a more realistic effective temperature instead of a fixed one at 254.K.

A more detailed description on the fitting setups for the rcodes can be found in the PGN Data Products ReadMe.

Data overview and filtering

The data are first filtered by the overlapping period, since the 1.7 timeseries has been closed from a processing point-of-view when we switched to 1.8 processing. The data are further filtered by the flag of the 1.8 data being 0, 10, 1, 11.

The following figures give an overview of the available timeseries for the overlapping/filtered period, where the vertical red line indicates the validity periods/CF’s used for processing.

Differences

The following plots show the difference between 1.7-1.8 data as a function of several parameters. 1.7 data are used as reference, wherefore negative values imply 1.7 data to be smaller.

  • summarizing boxplot and histogram
  • vertical column change with time
  • vertical column change with solar zenith angle

Sample quantiles for vertical column difference of p1-7 minus p1-8
0% 5% 50% 95% 100%
-0.03389 -0.0101 -0.00437 2e-05 0.0048

PIT histograms

The probability integral transform (PIT) is a useful evaluation tool to evaluate probabilistic forecast. In the evaluation context here, we do not have forecasts, but the 1.8 processor gives a total uncertainty estimation. Therefore, the PIT can evaluate if and where the 1.7 dataset can be found.

The major improvement of 1.8 datasets over 1.7 datasets is a rigorous uncertainty estimation which is reported as ‘total uncertainty’. This uncertainty can be seperated into three uncertainties, where the independent uncertainty is given by the measured uncertainty. The independent uncertainty can be seen as a precision estimation, and is given for both 1.7 and 1.8 datasets. But in addition to the independent uncertainty, 1.8 datasets report the common uncertainty which results from the calibration process, and the structured uncertainty. The combination of these three uncertainties provide a reliable uncertainty estimation which can be seen as an accuracy. More information about the uncertainty can be found in the PGN Data Products ReadMe.

The following timeseries illustrate example days for the overlapping period with at least 200 high quality data. The black dots illustrate the p1-8 timeseries and the red dots the p1-7 timeseries. Additionally, the 2-sigma level of the p1-8 total uncertainty is shown in thin black lines.

The following PIT histogram shows the occurrences of the p1-7 for the given p1-8 distribution, binned by 2.5%.

62.57 % of the p1-7 NO2 data points are within the 95% uncertainty interval of the p1-8 timeseries, and 37.43% outside.