Markov Decision Processes with Observation Costs: Framework and Computation with a Penalty Scheme
研究了在观测状态需付出成本时，如何优化观测时机和后续行动，通过将时间间隔纳入增广系统，利用拟变分不等式和惩罚方法求解，并通过三个数值实验验证了框架的有效性。
We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimization of observation times as well as the subsequent action values. We consider the finite horizon and discounted infinite horizon problems as well as an extension with parameter uncertainty. By including the time elapsed from observations as part of the augmented Markov system, the value function satisfies a system of quasivariational inequalities (QVIs). Such a class of QVIs can be seen as an extension to the interconnected obstacle problem. We prove a comparison principle for this class of QVIs, which implies the uniqueness of solutions to our proposed problem. Penalty methods are then utilized to obtain arbitrarily accurate solutions. Finally, we perform numerical experiments on three applications that illustrate our framework. Funding: J. Tam is supported by the Engineering and Physical Sciences Research Council [Grant 2269738].