Optimal Hybrid Transmission Strategy for Remote State Estimation With Deep Reinforcement Learning
针对远程状态估计中高频链路带宽高但可靠性低、低频链路可靠性高但能耗低的特点，提出一种混合传输策略，并用深度强化学习算法求解最优调度方案，实现高可靠低能耗传输。
This work delves into an optimal hybrid transmission strategy for remote state estimation (RSE), where some smart sensors observe various systems and transmit their local state estimates to a remote estimator. Inspired by the high bandwidth of a high-frequency (HF) link alongside the robust reliability and low energy consumption of a low-frequency (LF) link, a novel hybrid transmission strategy is proposed. This strategy allows smart sensors to utilize either the HF link or the LF link dynamically, each with distinct channel characteristics and energy consumption. To achieve highly reliable and energy-efficient transmission, it is imperative to devise an optimal transmission scheduling strategy that dictates the selection of the link and channel allocation. Formulating this challenge as a Markov decision process (MDP), a sufficient condition ensuring the existence of an optimal deterministic and stationary (ODS) policy is established. The structural characteristics of the optimal policy are derived. Furthermore, a deep reinforcement learning (DRL) approach, specifically the dueling double deep <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i>-network (D3QN) algorithm, is employed to approximate the optimal policy. Finally, the structural results and the effectiveness of the DRL algorithm are verified by simulation examples.