Performance evaluation of semantic segmentation models: a cross meta-frontier DEA approach
提出一种交叉元前沿数据包络分析方法，综合评估语义分割模型的准确性、硬件负担和模型结构，并分解效率找出低效来源，为模型改进提供方向。
Performance evaluation of semantic segmentation models is an essential task because it helps to identify the best-performing model. Traditional methods, however, are generally concerned with the improvement of a single quality or quantity. Moreover, what causes low performance usually goes unnoticed. To address these issues, a new cross meta-frontier data envelopment analysis (DEA) approach is proposed in this article. For evaluating model performance comprehensively, not only accuracy metrics, but also hardware burden and model structure factors, are taken as DEA outputs and inputs, separately. In addition, the potential inefficiency is attributed to architectures and backbones via efficiency decomposition, so that it can find the sources of inefficiency and provides a direction for performance improvement. Finally, based on the proposed approach, the performance of 16 classical semantic segmentation models on the PASCAL VOC dataset are re-evaluated and explained. The results verify that the proposed approach can be considered as a comprehensive and interpretable performance evaluation technique, which expands the traditional accuracy-based measurement.