Quantifying the benefits of increasing decision-making frequency for health applications with regular decision epochs
研究在医疗场景中增加决策频率带来的价值，通过马尔可夫决策过程框架分析最优策略的结构性质，并以肝脏移植和慢性肾病治疗为例进行数值验证。
Sequential decision-making problems in the context of uncertainty naturally arise in healthcare settings. In general, the frequency at which decisions can be made or changed is determined by physical limitations, such as the frequency of doctor’s visits or transplantation offers. Quantifying the benefits of increasing the frequency of decision-making allows us to quantify the value of changing these physical constraints and thus improve the quality of care. In this article, we study the value provided by having additional decision-making opportunities in each epoch. We model this problem using a Markov Decision Process (MDP) framework. We provide structural properties of the optimal policies and quantify the difference in optimal values between MDP problems of different decision-making frequencies. We analyze numerical examples using liver transplantation in high-risk patients and treatment initiation in chronic kidney disease to illustrate our findings.