Light field image super-resolution based on dual learning and deep Fourier channel attention

Jian Ma; Jian Ma; Zhipeng Li; Jin Cheng; Ping An; Dong Liang; Linsheng Huang

doi:10.1364/OL.522701

Optics Letters
Vol. 49,
Issue 11,
pp. 2886-2889
(2024)
•https://doi.org/10.1364/OL.522701

Light field image super-resolution based on dual learning and deep Fourier channel attention

Jian Ma, Zhipeng Li, Jin Cheng, Ping An, Dong Liang, and Linsheng Huang

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Jian Ma, Zhipeng Li, Jin Cheng, Ping An, Dong Liang, and Linsheng Huang, "Light field image super-resolution based on dual learning and deep Fourier channel attention," Opt. Lett. 49, 2886-2889 (2024)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Abstract

Light field (LF) imaging has gained significant attention in the field of computational imaging due to its unique capability to capture both spatial and angular information of a scene. In recent years, super-resolution (SR) techniques based on deep learning have shown considerable advantages in enhancing LF image resolution. However, the inherent challenges of obtaining rich structural information and reconstructing complex texture details persist, particularly in scenarios where spatial and angular information are intricately interwoven. This Letter introduces a novel, to the best of our knowledge, approach for Disentangling LF Image SR Network (DLISN) by leveraging the synergy of dual learning and Fourier channel attention (FCA) mechanisms. Dual learning strategies are employed to enhance reconstruction results, addressing limitations in model generalization caused by the difficulty in acquiring paired datasets in real-world LF scenarios. The integration of FCA facilitates the extraction of high-frequency information associated with different structures, contributing to improved spatial resolution. Experimental results consistently demonstrate superior performance in enhancing the resolution of LF images.

Full Article | PDF Article

More Like This

Super-resolution reconstruction of terahertz images based on a deep-learning network with a residual channel attention mechanism

Xiuwei Yang, Dehai Zhang, Zhongmin Wang, Yanbo Zhang, Jun Wu, Biyuan Wu, and Xiaohu Wu
Appl. Opt. 61(12) 3363-3370 (2022)

Deep learning empowers photothermal microscopy with super-resolution capabilities

Yonghui Wang, Zhuoyan Yue, Fei Wang, Peng Song, and Junyan Liu
Opt. Lett. 49(8) 1957-1960 (2024)

Blind light field image quality assessment based on deep meta-learning

Jian Ma, Xiaoyin Zhang, and Junbo Wang
Opt. Lett. 48(23) 6184-6187 (2023)

Previous Article Next Article

Data availability

Data underlying the results presented in this paper are not publicly available but may be obtained from the authors upon reasonable request.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (3)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (3)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (9)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Method	Scale	EPFL	HCInew	HCIold	INRIA	STFgantry
Bicubic	2 $\times$	29.50/0.935	31.69/0.933	37.46/0.977	31.10/0.956	30.82/0.947
VDSR [22]	2 $\times$	32.50/0.959	34.37/0.956	40.61/0.986	34.43/0.974	35.54/0.979
EDSR [23]	2 $\times$	33.09/0.963	34.83/0.959	41.01/0.987	34.97/0.976	36.29/0.981
RCAN [12]	2 $\times$	33.16/0.963	34.98/0.960	41.05/0.987	35.01/0.976	36.33/0.982
resLF [5]	2 $\times$	32.75/0.967	36.07/0.971	42.61/0.992	34.57/0.978	36.89/0.987
LFSSR [24]	2 $\times$	33.69/0.974	36.86/0.975	43.75/0.993	35.27/0.983	38.07/0.990
LF-ATO [25]	2 $\times$	34.27/0.976	37.24/0.977	44.20/0.994	36.15/0.984	39.64/0.993
LF-InterNet [26]	2 $\times$	34.14/0.972	37.28/0.977	44.45/0.995	35.80/0.985	38.72/0.992
LF-DFnet [27]	2 $\times$	34.44/0.977	37.44/0.979	44.23/0.994	36.36/0.984	39.61/0.993
DistgSSR [9]	2 $\times$	34.80/0.979	37.95/0.980	44.92/0.995	36.58/0.986	40.37/0.994
DLISN	2 $\times$	35.11/0.979	37.99/0.978	44.87/0.995	36.88/0.986	40.72/0.995
Bicubic	4 $\times$	25.14/0.831	27.61/0.851	32.42/0.934	26.82/0.886	25.93/0.843
VDSR [22]	4 $\times$	26.82/0.869	29.12/0.876	34.01/0.943	28.87/0.914	28.31/0.893
EDSR [23]	4 $\times$	27.82/0.892	29.94/0.893	35.53/0.957	29.86/0.931	29.43/0.921
RCAN [12]	4 $\times$	28.31/0.899	30.25/0.896	35.89/0.959	30.36/0.936	30.25/0.934
resLF [5]	4 $\times$	27.86/0.899	30.37/0.907	36.12/0.966	29.72/0.936	29.64/0.927
LFSSR [24]	4 $\times$	29.16/0.915	30.88/0.913	36.90/0.970	31.03/0.944	30.14/0.937
LF-ATO [25]	4 $\times$	28.52/0.912	30.88/0.914	37.00/0.970	30.71/0.949	30.61/0.943
LF-InterNet [26]	4 $\times$	28.67/0.914	30.98/0.917	37.11/0.972	30.64/0.949	30.53/0.943
LF-DFnet [27]	4 $\times$	28.77/0.917	31.23/0.920	37.32/0.972	30.83/0.950	31.15/0.949
DistgSSR [9]	4 $\times$	28.98/0.919	31.38/0.922	37.55/0.973	30.99/0.952	31.63/0.953
DLISN	4 $\times$	29.12/0.922	31.46/0.923	37.72/0.974	31.12/0.953	31.81/0.955

Methods	2 $\times$			4 $\times$
Methods	Param.	FLOPs	PSNR	Param.	FLOPs	PSNR
EDSR [23]	38.6M	988.9G	36.04	38.9M	1017G	30.20
RCAN [12]	15.3M	392.8G	36.11	15.4M	408.5G	30.27
resLF [5]	6.35M	37.06G	37.50	6.79M	39.70G	31.25
LF-InterNet [26]	4.80M	47.46G	38.08	5.23M	50.10G	31.59
LF-DFnet [27]	3.94M	57.22G	38.42	3.99M	57.31G	31.86
DLISN(ours)	7.5M	165.1G	39.11	7.8M	169.3G	32.25

	Dual learning	Distg	DF-block	Param.	EPFL	HCInew	HCIold	INRIA	STFgt	Average
0		$✓$		3.53M	34.80	37.95	44.92	36.58	40.37	38.92
1		$✓$	$✓$	7.5M	35.00	37.92	44.85	36.75	40.52	39.01
2	$✓$	$✓$		3.54M	34.66	37.96	44.86	36.52	40.63	38.93
3	$✓$	$✓$	$✓$	7.8M	35.11	37.99	44.87	36.88	40.72	39.11

Method	Scale	EPFL	HCInew	HCIold	INRIA	STFgantry
Bicubic	2 $\times$	29.50/0.935	31.69/0.933	37.46/0.977	31.10/0.956	30.82/0.947
VDSR [22]	2 $\times$	32.50/0.959	34.37/0.956	40.61/0.986	34.43/0.974	35.54/0.979
EDSR [23]	2 $\times$	33.09/0.963	34.83/0.959	41.01/0.987	34.97/0.976	36.29/0.981
RCAN [12]	2 $\times$	33.16/0.963	34.98/0.960	41.05/0.987	35.01/0.976	36.33/0.982
resLF [5]	2 $\times$	32.75/0.967	36.07/0.971	42.61/0.992	34.57/0.978	36.89/0.987
LFSSR [24]	2 $\times$	33.69/0.974	36.86/0.975	43.75/0.993	35.27/0.983	38.07/0.990
LF-ATO [25]	2 $\times$	34.27/0.976	37.24/0.977	44.20/0.994	36.15/0.984	39.64/0.993
LF-InterNet [26]	2 $\times$	34.14/0.972	37.28/0.977	44.45/0.995	35.80/0.985	38.72/0.992
LF-DFnet [27]	2 $\times$	34.44/0.977	37.44/0.979	44.23/0.994	36.36/0.984	39.61/0.993
DistgSSR [9]	2 $\times$	34.80/0.979	37.95/0.980	44.92/0.995	36.58/0.986	40.37/0.994
DLISN	2 $\times$	35.11/0.979	37.99/0.978	44.87/0.995	36.88/0.986	40.72/0.995
Bicubic	4 $\times$	25.14/0.831	27.61/0.851	32.42/0.934	26.82/0.886	25.93/0.843
VDSR [22]	4 $\times$	26.82/0.869	29.12/0.876	34.01/0.943	28.87/0.914	28.31/0.893
EDSR [23]	4 $\times$	27.82/0.892	29.94/0.893	35.53/0.957	29.86/0.931	29.43/0.921
RCAN [12]	4 $\times$	28.31/0.899	30.25/0.896	35.89/0.959	30.36/0.936	30.25/0.934
resLF [5]	4 $\times$	27.86/0.899	30.37/0.907	36.12/0.966	29.72/0.936	29.64/0.927
LFSSR [24]	4 $\times$	29.16/0.915	30.88/0.913	36.90/0.970	31.03/0.944	30.14/0.937
LF-ATO [25]	4 $\times$	28.52/0.912	30.88/0.914	37.00/0.970	30.71/0.949	30.61/0.943
LF-InterNet [26]	4 $\times$	28.67/0.914	30.98/0.917	37.11/0.972	30.64/0.949	30.53/0.943
LF-DFnet [27]	4 $\times$	28.77/0.917	31.23/0.920	37.32/0.972	30.83/0.950	31.15/0.949
DistgSSR [9]	4 $\times$	28.98/0.919	31.38/0.922	37.55/0.973	30.99/0.952	31.63/0.953
DLISN	4 $\times$	29.12/0.922	31.46/0.923	37.72/0.974	31.12/0.953	31.81/0.955

Methods	2 $\times$			4 $\times$
Methods	Param.	FLOPs	PSNR	Param.	FLOPs	PSNR
EDSR [23]	38.6M	988.9G	36.04	38.9M	1017G	30.20
RCAN [12]	15.3M	392.8G	36.11	15.4M	408.5G	30.27
resLF [5]	6.35M	37.06G	37.50	6.79M	39.70G	31.25
LF-InterNet [26]	4.80M	47.46G	38.08	5.23M	50.10G	31.59
LF-DFnet [27]	3.94M	57.22G	38.42	3.99M	57.31G	31.86
DLISN(ours)	7.5M	165.1G	39.11	7.8M	169.3G	32.25

Abstract

Data availability

Cited By

Figures (3)

Tables (3)

Equations (9)

Optics Letters