Detecting abrupt changes in missing time series data

When time series data contain missing values, it is common practice to substitute them using missing value imputation. However, if there is an unobserved abrupt change in the missing values, then standard imputation techniques are insufficient since they are biased towards normal data. Likewise, standard detectors cannot find abrupt changes that “hide” in missing data. To address these shortcomings, we propose Interval Forecast Imputation (IFI), which is a simple and intuitive combination of uncertainty intervals, forecasting, and anomaly detection that detects abrupt changes in missing time series data. A further advantage of IFI is that it is compatible with every state of the art forecasting technique—ranging from simple exponential smoothing over neural network-assisted forecasts to the popular Prophet library—while requiring only O(1) additional time and space. In our experiments, we observe that IFI can detect abrupt changes in missing data and improves the imputation accuracy of all forecasting methods it is combined with.