Optimizing Evaluation Metrics for Multitask Learning via the Alternating Direction Method of Multipliers
提出一种直接优化多任务学习评估指标的方法，利用交替方向乘子法分解非光滑优化问题，在分类和回归任务上优于基线方法。
Multitask learning (MTL) aims to improve the generalization performance of multiple tasks by exploiting the shared factors among them. Various metrics (e.g., -score, area under the ROC curve) are used to evaluate the performances of MTL methods. Most existing MTL methods try to minimize either the misclassified errors for classification or the mean squared errors for regression. In this paper, we propose a method to directly optimize the evaluation metrics for a large family of MTL problems. The formulation of MTL that directly optimizes evaluation metrics is the combination of two parts: 1) a regularizer defined on the weight matrix over all tasks, in order to capture the relatedness of these tasks and 2) a sum of multiple structured hinge losses, each corresponding to a surrogate of some evaluation metric on one task. This formulation is challenging in optimization because both of its parts are nonsmooth. To tackle this issue, we propose a novel optimization procedure based on the alternating direction scheme of multipliers, where we decompose the whole optimization problem into a subproblem corresponding to the regularizer and another subproblem corresponding to the structured hinge losses. For a large family of MTL problems, the first subproblem has closed-form solutions. To solve the second subproblem, we propose an efficient primal-dual algorithm via coordinate ascent. Extensive evaluation results demonstrate that, in a large family of MTL problems, the proposed MTL method of directly optimization evaluation metrics has superior performance gains against the corresponding baseline methods.