使用自适应估计的非平稳流数据的估计、预测与异常检测

Estimation, Forecasting, and Anomaly Detection for Nonstationary Streams Using Adaptive Estimation

IEEE Transactions on Cybernetics · 2021
被引 20
ABS 3

中文导读

提出实时自适应组件(RAC)框架,结合自适应滤波与惩罚回归,处理流数据中的概念漂移和结构变化,并集成异常检测功能,实验表明其在模拟和真实网络安全数据上表现良好。

Abstract

Streaming data provides substantial challenges for data analysis. From a computational standpoint, these challenges arise from constraints related to computer memory and processing speed. Statistically, the challenges relate to constructing procedures that can handle the so-called concept drift-the tendency of future data to have different underlying properties to current and historic data. The issue of handling structure, such as trend and periodicity, remains a difficult problem for streaming estimation. We propose the real-time adaptive component (RAC), a penalized-regression modeling framework that satisfies the computational constraints of streaming data, and provides the capability for dealing with concept drift. At the core of the estimation process are techniques from adaptive filtering. The RAC procedure adopts a specified basis to handle local structure, along with a least absolute shrinkage operator-like penalty procedure to handle over fitting. We enhance the RAC estimation procedure with a streaming anomaly detection capability. The experiments with simulated data suggest the procedure can be considered as a competitive tool for a variety of scenarios, and an illustration with real cyber-security data further demonstrates the promise of the method.

数据流挖掘异常检测概念漂移自适应滤波时间序列分析