Predictive Inference, Sufficiency, Entropy and an Asymptotic Likelihood Principle
本文从重复抽样实验出发，论证了用熵或KL散度衡量预测分布近似真实分布的合理性，为模型选择与参数估计提供了小样本理论依据。
The objective of inferring stochastic models from a set of data is to obtain the best description, by using a probability model, of the statistical behaviour of future samples of the process. A conceptual repeated sampling experiment is considered for evaluating a predictive distribution used to describe such future observations and leads to an asymptotic likelihood principle. Considerations of likelihood and sufficiency lead to the use of entropy or the Kullback–Leibler information as the natural measure of approximation to the actual distribution by a predictive distribution in repeated samples. This gives a small-sample justification for the use of entropy for evaluating parameter estimation as well as model order and structure determination procedures.