Analytical Solution to a Discrete-Time Model for Dynamic Learning and Decision Making
研究了一个无限时域离散时间模型，该模型统一了序贯假设检验、动态定价和多臂老虎机等问题，通过构建有效前沿的差分方程得到解析解，为离散时间模型的新发现提供了工具。
Problems concerning dynamic learning and decision making are difficult to solve analytically. We study an infinite-horizon discrete-time model with a constant unknown state that may take two possible values. As a special partially observable Markov decision process (POMDP), this model unifies several types of learning-and-doing problems such as sequential hypothesis testing, dynamic pricing with demand learning, and multiarmed bandits. We adopt a relatively new solution framework from the POMDP literature based on the backward construction of the efficient frontier(s) of continuation-value vectors. This framework accommodates different optimality criteria simultaneously. In the infinite-horizon setting, with the aid of a set of signal quality indices, the extreme points on the efficient frontier can be linked through a set of difference equations and solved analytically. The solution carries structural properties analogous to those obtained under continuous-time models, and it provides a useful tool for making new discoveries through discrete-time models. This paper was accepted by Baris Ata, stochastic models and simulation.