Stochastic complexity and the mdl principle
提出随机复杂度作为数据编码的下界,用于比较不同参数数量的模型,并给出高斯和多项分布下的新模型选择准则,通过最小描述长度原则计算近似值。
A Search for the stochastic complexity of the observed data, as the greates lower bound with which the data can be encoded, represents a global maximum likelihood principle, which permits comparison of models regardless of the number of parameters in them. For important special classes, such as the guassian and the multinomial models, formulas for the stochastic complexity give new and powerful model selection criteria,while in the general case approximations can be computed with the MDL principle. Once a model is found with which the stochastic complexity is reached, there is nothing further to learn from the data with the proposed models. The basic notions are reviewed and numerical examples are given.