标题: MDTNet: Partial transformer with degradation-aware module for restoring old photos with multiple degradations

作者: Cao, LQ (Cao, Liqin); Zhao, Y (Zhao, Yuan); Zhang, F (Zhang, Fan); Zhang, X (Zhang, Xuan); Liu, JH (Liu, Ju Hua); Zhong, YF (Zhong, Yanfei)

来源出版物: NEUROCOMPUTING 卷: 616 文献号: 128866 DOI: 10.1016/j.neucom.2024.128866 Early Access Date: NOV 2024 Published Date: 2025 FEB 1

摘要: Old photos preserve the memories of the old days and have special meanings in life. However, they often suffer from severe and complex degradations, such as cracks, dirt, and noise. Old photo restoration is thus a meaningful but challenging task. Most of the current deep learning methods to repair corrupted old photos focus on one single type of degradation or limited number of degradations. The lack of proper old photo datasets makes the digital restoration problem even more challenging. In this paper, we propose MDTNet, which is a novel transformer-based method for restoring old photos with multiple degradations. MDTNet is an efficient end-to-end transformer architecture with only a single encoder and a decoder to complete the restoration. Specifically, a novel partial transformer encoder is designed with partial multi-head self-attention to extract local-global context information from the valid masked area, which represents the unbroken area. In addition, a degradation-aware module is proposed in the decoder to automatically learn different degradation information, to improve the effect of multiple degradation restoration. Moreover, a synthetic old photo dataset - the SynOld dataset - was built, which simulates the multiple degradations of real old photos. Experimental results show that the proposed method outperforms the state-of-the-art methods on both public datasets and the custom-developed dataset. The code is available at https://github.com/YuanZhaoc/MDTNet.

作者关键词: Old photo restoration; Partial transformer; Degradation-aware module

地址: [Cao, Liqin; Zhao, Yuan] Wuhan Univ, Sch Resource & Environm Sci, Wuhan, Peoples R China.

[Zhang, Fan; Zhong, Yanfei] Wuhan Univ, State Key Lab Informat Engn Surveying Mapping & Re, Wuhan, Peoples R China.

[Zhang, Xuan] Norwegian Res Ctr AS, Grimstad, Norway.

[Liu, Ju Hua] Wuhan Univ, Sch Comp, Wuhan, Peoples R China.

通讯作者地址: Zhao, Y (通讯作者)，Wuhan Univ, Sch Resource & Environm Sci, Wuhan, Peoples R China.

电子邮件地址: 2018302170005@whu.edu.cn

影响因子：5.5