Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control
研究了分布式无线网络化控制系统中上下行联合调度问题，推导了稳定性条件，并用深度强化学习框架求解最优调度策略，提出动作空间缩减和嵌入方法以应对大规模动作空间挑战。
We consider a joint uplink and downlink scheduling problem of a fully distributed wireless networked control system (WNCS) with a limited number of frequency channels. Using elements of stochastic systems theory, we derive a sufficient stability condition of the WNCS, which is stated in terms of both the control and communication system parameters. Once the condition is satisfied, there exists a stationary and deterministic scheduling policy that can stabilize all plants of the WNCS. By analyzing and representing the per-step cost function of the WNCS in terms of a finite-length countable vector state, we formulate the optimal transmission scheduling problem into a Markov decision process and develop a deep reinforcement learning (DRL)-based framework for solving it. To tackle the challenges of a large action space in DRL, we propose novel action space reduction and action embedding methods for the DRL framework that can be applied to various algorithms, including deep Q-network (DQN), deep deterministic policy gradient (DDPG), and twin delayed DDPG (TD3). Numerical results show that the proposed algorithm significantly outperforms benchmark policies.