为了提高语义分割精度,解决模型在不同数据域上泛化性差的问题,提出基于深度信息的无监督领域自适应语义分割方法.首先,深度感知自适应框架通过捕捉深度信息和语义信息的内在联系,减小不同域之间的差异;然后,设计了一个轻量级深度估计网络来提供深度信息,通过跨任务交互策略融合深度和语义信息,并在深度感知空间对齐源域和目标域的分布差距;最后,提出基于深度信息的域内自适应策略弥合目标域内部的分布差异,将目标域分为子源域和子目标域,并缩小子源域和子目标域分布差距.实验结果表明,所提方法在SYNTHIA-2-Cityscapes和SYNTHIA-2-Mapillary跨域任务上的平均交并比分别为46.7%和73.3%,与同类方法相比,该方法在语义分割和深度估计精度上均有显著提升.
Abstract:
To improve the segmentation performance and solve the problem of poor generalization of the model in different data domains, we propose a method based on depth information for semantic segmentation in the context of unsupervised domain adaptation. It includes a Depth-aware Adaptation Framework(DAF) and a Intra-domain Adaptation(IDA) strategy. Firstly, DAF is proposed to adapt domains by capitalizing on the inherent correlations of semantic and depth information. Then a novel light-weight depth estimation network is designed provide additional depth information, and we fuse semantic and depth information by cross-task interaction, then align the distribution in depth-aware space between source and target domains. Finally, IDA strategy is proposed to bridge the distribution gap inside the target domain. To this end, a depth-aware ranking strategy is presented to separate target domain into sub-source domain and sub-target domain, and then we perform the alignment between sub-source domain and sub-target domain. Experiments on SYNTHIA-2-Cityscapes and SYNTHIA-2-Mapillary cross-domain tasks show that our method achieves significant improvement(46.7% mIoU and 73.3% mIoU, respectively) compared with the similar methods.
