Least Squares and Entropy: A Penalty Function Perspective
从惩罚函数角度重新解释熵度量，并与最小二乘估计进行对比，展示两者在一般线性模型中的应用，分析参数估计差异的原因，为基于熵的计量问题提供建议。
Abstract Mathematical measures of entropy as defined by Shannon and cross entropy as defined by Kullback and Leibler are currently in vogue in the field of econometrics, primarily due to the comprehensive work of Golan, Judge, and Miller. An alternative interpretation of the entropy measure as a penalty function over deviations is presented, and a number of parallels are drawn with least squares estimators. It is demonstrated that both approaches may be applied to the general linear model. The causes of differences in estimated parameter values are described, and some suggestions for the formulation of entropy‐based econometric problems are presented.