Four-direction global matching with cost volume update for stereovision

Yusheng Jin; Hong Zhao; Penghui Bu

doi:10.1364/AO.422798

Applied Optics
Vol. 60,
Issue 19,
pp. 5471-5479
(2021)
•https://doi.org/10.1364/AO.422798

Four-direction global matching with cost volume update for stereovision

Yusheng Jin, Hong Zhao, and Penghui Bu

Not Accessible

Your library or personal account may give you access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Yusheng Jin, Hong Zhao, and Penghui Bu, "Four-direction global matching with cost volume update for stereovision," Appl. Opt. 60, 5471-5479 (2021)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Related Topics
Table of Contents Category
- Imaging Systems, Image Processing, and Displays
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: February 16, 2021
- Revised Manuscript: May 11, 2021
- Manuscript Accepted: May 26, 2021
- Published: June 21, 2021

Abstract

The accuracy and speed of semi-global matching (SGM) make it widely used in many computer vision problems. However, SGM often struggles in dealing with pixels in the homogeneous regions and also suffers from streak artefacts for weak smoothness constraints. Meanwhile, we observe that the global method usually fails in occluded areas. The disparities for occluded pixels are typically the average of the disparity of nearby pixels. The local method can propagate the information into occluded pixels with a similar color. In this paper, we propose a novel, to the best of our knowledge, four-direction global matching with a cost volume update scheme to cope with textureless regions and occlusion. The proposed method makes two changes in the recursive formula: a) the computation process considers four visited nodes to enforce more smooth constraints; b) the recursive formula integrates cost filtering to propagate reliable information farther in nontextured regions. Thus, our method can inherit the speed of SGM, properly avoid streaking artefacts, and deal with the occluded pixel. Extensive experiments in stereo matching on Middlebury demonstrate that our method outperforms typical SGM-based cost aggregation approaches and other state-of-the-art local methods.

Full Article | PDF Article

More Like This

Collaborative semi-global stereo matching

Penghui Bu, Hong Zhao, Jiaxing Yan, and Yusheng Jin
Appl. Opt. 60(31) 9757-9768 (2021)

Occlusion disparity refinement for stereo matching through the geometric prior-based adaptive label search

Junwei Wang, Wei Zhou, Ziheng Qian, Jiaqi Shen, and Hanming Guo
J. Opt. Soc. Am. A 38(11) 1619-1630 (2021)

Adaptive epipolar distance transform and adaptive matching cost for stereo matching

Yakun Zhang, Haibin Li, and Wenming Zhang
J. Opt. Soc. Am. A 38(8) 1170-1177 (2021)

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (6)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (4)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (14)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

	GF [4]	NL [5]	ST [29]	GC [11]	SGM [14]	MGM [15]	FGM	FGMU
Tsukuba	$2 {.40}_{3}$	$2 {.48}_{5}$	$2 {.95}_{8}$	${2.85}_{7}$	$2 {.64}_{6}$	$2 {.41}_{4}$	$2 {.39}_{2}$	$2 {.17}_{1}$
Venus	$1 {.24}_{4}$	$1 {.37}_{6}$	$1 {.34}_{5}$	${1.54}_{7}$	${1.37}_{6}$	${1.07}_{3}$	$1.0 3_{2}$	$0. 52_{1}$
Teddy	$7 {.06}_{4}$	$6 {.85}_{3}$	$8 {.26}_{8}$	${7.35}_{5}$	${8.04}_{7}$	${7.46}_{6}$	$6. 12_{2}$	$4 .9 8_{1}$
Cones	$2 {.45}_{1}$	$4 {.11}_{5}$	$4 {.55}_{7}$	${4.45}_{6}$	${4.45}_{6}$	${3.79}_{3}$	$3. 80_{4}$	$2. 63_{2}$
Aloe	$4 {.48}_{7}$	$4 {.82}_{8}$	$4 {.46}_{6}$	$3 {.30}_{5}$	${3.27}_{4}$	${2.99}_{2}$	$2 {.85}_{1}$	$3. 01_{3}$
Baby1	$3 {.90}_{6}$	$7 {.99}_{8}$	$5 {.23}_{7}$	$3 {.28}_{5}$	${2.48}_{4}$	${2.31}_{3}$	$2 {.19}_{1}$	$2. 2 1_{2}$
Baby2	$4 {.48}_{6}$	$13 {.48}_{7}$	$14 {.53}_{8}$	${2.76}_{5}$	${1.52}_{4}$	${1.22}_{3}$	$1. 10_{1}$	$1.1 7_{2}$
Baby3	$5 {.45}_{7}$	$5 {.18}_{6}$	$5 {.57}_{8}$	${3.32}_{3}$	${3.41}_{5}$	${3.40}_{4}$	$2 {.96}_{2}$	${2.78}_{1}$
Bowling1	$14 {.78}_{6}$	$18 {.52}_{8}$	$15 {.80}_{7}$	$13 {.01}_{5}$	${9.63}_{3}$	${6.97}_{2}$	$5 {.67}_{1}$	$11 {.21}_{4}$
Bowling2	$6 {.16}_{6}$	$8 {.83}_{7}$	$9 {.26}_{8}$	${5.92}_{5}$	${3.43}_{4}$	${3.32}_{3}$	$3. 21_{1}$	$3. 27_{2}$
Cloth1	$0 {.73}_{7}$	$0 {.71}_{6}$	$0. 69_{5}$	$0 {.80}_{8}$	${0.56}_{4}$	${0.46}_{3}$	${0.45}_{2}$	$0. 2 4_{1}$
Cloth2	$2 {.76}_{6}$	$3 {.90}_{8}$	$2. 90_{7}$	${1.60}_{3}$	${1.77}_{5}$	${1.46}_{2}$	${1.64}_{4}$	$1. 14_{1}$
Cloth3	$1 .6 8_{5}$	$2 {.38}_{8}$	$2 {.09}_{7}$	${1.70}_{6}$	${1.37}_{4}$	${1.09}_{2}$	$1. 28_{3}$	$0 {.98}_{1}$
Cloth4	$1. 32_{5}$	$1. 95_{8}$	$1. 61_{7}$	${1.23}_{4}$	${1.49}_{6}$	${0.99}_{2}$	$1 {.05}_{3}$	$0. 78_{1}$
Flowerpots	$11 {.56}_{6}$	$14 {.14}_{8}$	$12 {.16}_{7}$	${8.45}_{5}$	${7.95}_{4}$	${7.27}_{2}$	$7. 75_{3}$	$6 {.12}_{1}$
Lampshade1	$11 {.91}_{8}$	$9 {.16}_{6}$	$9 {.96}_{7}$	${7.25}_{5}$	${6.45}_{4}$	${5.07}_{3}$	$4 {.86}_{2}$	$3 {.49}_{1}$
Lampshade2	$21 {.73}_{8}$	${13.69}_{6}$	$22 {.31}_{7}$	${7.38}_{5}$	${4.54}_{4}$	${4.11}_{3}$	$3 {.93}_{2}$	$2 {.59}_{1}$
Rocks1	$1 {.92}_{7}$	$2 {.88}_{8}$	$1 {.80}_{6}$	${1.45}_{4}$	${1.56}_{5}$	${1.19}_{2}$	$1.2 6_{3}$	$1. 15_{1}$
Rocks2	$1 {.11}_{3}$	$1 {.79}_{8}$	$1. 36_{7}$	$1. 13_{4}$	${1.26}_{6}$	${0.99}_{2}$	$1. 1 5_{5}$	$0. 83_{1}$
Wood1	$3. 80_{5}$	$10 {.15}_{8}$	$3 {.92}_{7}$	$3 {.91}_{6}$	${1.70}_{2}$	${2.68}_{3}$	$1 {.45}_{1}$	$3 {.12}_{4}$
Wood2	$2. 26_{6}$	$3 {.65}_{8}$	$2 {.50}_{7}$	$1. 41_{5}$	${1.37}_{4}$	${1.07}_{3}$	$0 {.79}_{2}$	${0.45}_{1}$
Art	$8 {.33}_{6}$	$9 {.92}_{8}$	$9 {.03}_{7}$	${8.24}_{5}$	${5.88}_{3}$	${6.28}_{4}$	$5 {.27}_{1}$	$5. 74_{2}$
Books	$8 {.58}_{5}$	$11 {.57}_{8}$	$9 {.76}_{7}$	${8.72}_{6}$	${6.74}_{4}$	${6.13}_{3}$	$5 {.63}_{1}$	$5.6 8_{2}$
Dolls	$4 {.21}_{5}$	$5 {.05}_{8}$	$4 {.26}_{7}$	${3.19}_{3}$	${3.42}_{6}$	${3.04}_{2}$	$3. 20_{4}$	$2 {.91}_{1}$
Laundry	$15 {.26}_{8}$	$12 {.35}_{4}$	$13 {.54}_{7}$	${11.61}_{3}$	${12.63}_{6}$	${12.62}_{5}$	$11 {.36}_{2}$	$10 {.85}_{1}$
Moebius	$8 {.87}_{8}$	$7 {.45}_{5}$	$7 {.92}_{7}$	${7.27}_{3}$	${7.83}_{6}$	${7.07}_{2}$	$6 {.67}_{1}$	$7 {.41}_{4}$
Reindeer	$7 {.01}_{6}$	$10 {.81}_{8}$	$7 {.77}_{7}$	${4.62}_{5}$	${4.32}_{4}$	${4.09}_{2}$	${4.17}_{3}$	$2 {.78}_{1}$
Avg. error	$5 {.42}_{6}$	$7 {.23}_{8}$	$6 {.87}_{7}$	$4 {.73}_{5}$	$4 {.11}_{4}$	$3 {.72}_{3}$	$3 {.43}_{2}$	$3 {.34}_{1}$

	Initial Disparity		Final Disparity
Method	Noc(%)	All(%)	Noc(%)	All(%)
GF [4]	5.42	17.09	5.07	11.05
NL [5]	7.23	17.99	6.89	12.08
ST [29]	6.87	17.16	6.08	11.90
GC [11]	4.73	16.85	4.78	10.74
SGM [14]	4.11	16.86	3.58	9.81
MGM [15]	3.72	16.44	3.30	9.58
FMG	3.43	16.18	3.03	9.32
FMGU	3.34	12.68	3.01	9.24

	Initial Disparity		Final Disparity
Method	Noc(%)	All(%)	Noc(%)	All(%)
GC [11]	13.24	25.33	9.95	18.26
SGM [14]	13.69	25.88	10.43	18.76
MGM [15]	13.60	25.86	10.29	18.68
FMG	13.31	25.61	10.08	18.55
FMGU	11.23	22.38	9.95	17.98

Algorithm	Tsukuba	Venus	Teddy
SGM [14]	1.64 s	3.08 s	8.84 s
MGM [15]	3.60 s	6.56 s	18.72 s
FMG	3.52 s	6.42 s	17.95 s
FMGU	4.01 s	7.12 s	19.21 s

	GF [4]	NL [5]	ST [29]	GC [11]	SGM [14]	MGM [15]	FGM	FGMU
Tsukuba	$2 {.40}_{3}$	$2 {.48}_{5}$	$2 {.95}_{8}$	${2.85}_{7}$	$2 {.64}_{6}$	$2 {.41}_{4}$	$2 {.39}_{2}$	$2 {.17}_{1}$
Venus	$1 {.24}_{4}$	$1 {.37}_{6}$	$1 {.34}_{5}$	${1.54}_{7}$	${1.37}_{6}$	${1.07}_{3}$	$1.0 3_{2}$	$0. 52_{1}$
Teddy	$7 {.06}_{4}$	$6 {.85}_{3}$	$8 {.26}_{8}$	${7.35}_{5}$	${8.04}_{7}$	${7.46}_{6}$	$6. 12_{2}$	$4 .9 8_{1}$
Cones	$2 {.45}_{1}$	$4 {.11}_{5}$	$4 {.55}_{7}$	${4.45}_{6}$	${4.45}_{6}$	${3.79}_{3}$	$3. 80_{4}$	$2. 63_{2}$
Aloe	$4 {.48}_{7}$	$4 {.82}_{8}$	$4 {.46}_{6}$	$3 {.30}_{5}$	${3.27}_{4}$	${2.99}_{2}$	$2 {.85}_{1}$	$3. 01_{3}$
Baby1	$3 {.90}_{6}$	$7 {.99}_{8}$	$5 {.23}_{7}$	$3 {.28}_{5}$	${2.48}_{4}$	${2.31}_{3}$	$2 {.19}_{1}$	$2. 2 1_{2}$
Baby2	$4 {.48}_{6}$	$13 {.48}_{7}$	$14 {.53}_{8}$	${2.76}_{5}$	${1.52}_{4}$	${1.22}_{3}$	$1. 10_{1}$	$1.1 7_{2}$
Baby3	$5 {.45}_{7}$	$5 {.18}_{6}$	$5 {.57}_{8}$	${3.32}_{3}$	${3.41}_{5}$	${3.40}_{4}$	$2 {.96}_{2}$	${2.78}_{1}$
Bowling1	$14 {.78}_{6}$	$18 {.52}_{8}$	$15 {.80}_{7}$	$13 {.01}_{5}$	${9.63}_{3}$	${6.97}_{2}$	$5 {.67}_{1}$	$11 {.21}_{4}$
Bowling2	$6 {.16}_{6}$	$8 {.83}_{7}$	$9 {.26}_{8}$	${5.92}_{5}$	${3.43}_{4}$	${3.32}_{3}$	$3. 21_{1}$	$3. 27_{2}$
Cloth1	$0 {.73}_{7}$	$0 {.71}_{6}$	$0. 69_{5}$	$0 {.80}_{8}$	${0.56}_{4}$	${0.46}_{3}$	${0.45}_{2}$	$0. 2 4_{1}$
Cloth2	$2 {.76}_{6}$	$3 {.90}_{8}$	$2. 90_{7}$	${1.60}_{3}$	${1.77}_{5}$	${1.46}_{2}$	${1.64}_{4}$	$1. 14_{1}$
Cloth3	$1 .6 8_{5}$	$2 {.38}_{8}$	$2 {.09}_{7}$	${1.70}_{6}$	${1.37}_{4}$	${1.09}_{2}$	$1. 28_{3}$	$0 {.98}_{1}$
Cloth4	$1. 32_{5}$	$1. 95_{8}$	$1. 61_{7}$	${1.23}_{4}$	${1.49}_{6}$	${0.99}_{2}$	$1 {.05}_{3}$	$0. 78_{1}$
Flowerpots	$11 {.56}_{6}$	$14 {.14}_{8}$	$12 {.16}_{7}$	${8.45}_{5}$	${7.95}_{4}$	${7.27}_{2}$	$7. 75_{3}$	$6 {.12}_{1}$
Lampshade1	$11 {.91}_{8}$	$9 {.16}_{6}$	$9 {.96}_{7}$	${7.25}_{5}$	${6.45}_{4}$	${5.07}_{3}$	$4 {.86}_{2}$	$3 {.49}_{1}$
Lampshade2	$21 {.73}_{8}$	${13.69}_{6}$	$22 {.31}_{7}$	${7.38}_{5}$	${4.54}_{4}$	${4.11}_{3}$	$3 {.93}_{2}$	$2 {.59}_{1}$
Rocks1	$1 {.92}_{7}$	$2 {.88}_{8}$	$1 {.80}_{6}$	${1.45}_{4}$	${1.56}_{5}$	${1.19}_{2}$	$1.2 6_{3}$	$1. 15_{1}$
Rocks2	$1 {.11}_{3}$	$1 {.79}_{8}$	$1. 36_{7}$	$1. 13_{4}$	${1.26}_{6}$	${0.99}_{2}$	$1. 1 5_{5}$	$0. 83_{1}$
Wood1	$3. 80_{5}$	$10 {.15}_{8}$	$3 {.92}_{7}$	$3 {.91}_{6}$	${1.70}_{2}$	${2.68}_{3}$	$1 {.45}_{1}$	$3 {.12}_{4}$
Wood2	$2. 26_{6}$	$3 {.65}_{8}$	$2 {.50}_{7}$	$1. 41_{5}$	${1.37}_{4}$	${1.07}_{3}$	$0 {.79}_{2}$	${0.45}_{1}$
Art	$8 {.33}_{6}$	$9 {.92}_{8}$	$9 {.03}_{7}$	${8.24}_{5}$	${5.88}_{3}$	${6.28}_{4}$	$5 {.27}_{1}$	$5. 74_{2}$
Books	$8 {.58}_{5}$	$11 {.57}_{8}$	$9 {.76}_{7}$	${8.72}_{6}$	${6.74}_{4}$	${6.13}_{3}$	$5 {.63}_{1}$	$5.6 8_{2}$
Dolls	$4 {.21}_{5}$	$5 {.05}_{8}$	$4 {.26}_{7}$	${3.19}_{3}$	${3.42}_{6}$	${3.04}_{2}$	$3. 20_{4}$	$2 {.91}_{1}$
Laundry	$15 {.26}_{8}$	$12 {.35}_{4}$	$13 {.54}_{7}$	${11.61}_{3}$	${12.63}_{6}$	${12.62}_{5}$	$11 {.36}_{2}$	$10 {.85}_{1}$
Moebius	$8 {.87}_{8}$	$7 {.45}_{5}$	$7 {.92}_{7}$	${7.27}_{3}$	${7.83}_{6}$	${7.07}_{2}$	$6 {.67}_{1}$	$7 {.41}_{4}$
Reindeer	$7 {.01}_{6}$	$10 {.81}_{8}$	$7 {.77}_{7}$	${4.62}_{5}$	${4.32}_{4}$	${4.09}_{2}$	${4.17}_{3}$	$2 {.78}_{1}$
Avg. error	$5 {.42}_{6}$	$7 {.23}_{8}$	$6 {.87}_{7}$	$4 {.73}_{5}$	$4 {.11}_{4}$	$3 {.72}_{3}$	$3 {.43}_{2}$	$3 {.34}_{1}$

Abstract

Data Availability

Cited By

Figures (6)

Tables (4)

Equations (14)

Applied Optics