UNITE: Multitask Learning With Sufficient Feature for Dense Prediction
提出一种即插即用模块，同时利用跨任务一致性和互补信息来捕获充分特征，在NYUD-v2和PASCAL-Context数据集上显著提升多任务密集预测性能。
Existing multitask dense prediction methods typically rely on either global shared neural architecture or cross-task fusion strategy. However, these approaches tend to overlook either potential cross-task complementary or consistent information, resulting in suboptimal results. Motivated by this observation, we propose a novel plug-and-play module to concurrently leverage cross-task consistent and complementary information, thereby capturing a sufficient feature. Specifically, for a given pair of tasks, we compute a cross-task similarity matrix that extracts cross-task consistent features bidirectionally. To integrate the complementary signals from different tasks, we fuse the cross-task consistent features with the corresponding task-specific features using an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1\times 1$</tex-math> </inline-formula> convolution. Extensive experimental results demonstrate the remarkable performance gain of our method on two challenging datasets w.r.t different task sets, compared with seven approaches. Under the two-task setting, our method has achieved 1.63% and 8.32% improvements on NYUD-v2 and PASCAL-Context, respectively. On the three-task setting, we obtain an additional 7.7% multitask performance gain.