Abstract

We want to use a large-scale camera array system in which each camera is placed at the desired position to photograph a subject and later render images of the subject viewed from various directions or render images for a three-dimensional display. The homography matrix for each camera should be calculated in advance to correct the captured images. In the case that each camera is physically facing toward the subject as precisely as possible but the captured image still includes geometrical distortion, if the expected error in the deviations from the ideal directions is assumed to be the zero vector, the homography matrix of each camera can be easily obtained.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
    [CrossRef]
  2. K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).
  3. V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.
  4. M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.
  5. T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).
  6. Y.-S. Kang, C. Lee, and Y.-S. Ho, “An efficient rectification algorithm for multi-view images in parallel camera array,” in Proceedings of 3DTV Conference (IEEE, 2008), pp. 61–64.
  7. Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000).
    [CrossRef]

2011 (1)

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

2005 (1)

T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).

2003 (1)

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

2000 (1)

Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000).
[CrossRef]

Deng, K.

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

Deng, Z.

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

Feng, T.

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

Fujii, T.

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Fukushima, N.

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Ho, Y.-S.

Y.-S. Kang, C. Lee, and Y.-S. Ho, “An efficient rectification algorithm for multi-view images in parallel camera array,” in Proceedings of 3DTV Conference (IEEE, 2008), pp. 61–64.

Joshi, N.

V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.

Kang, Y.-S.

Y.-S. Kang, C. Lee, and Y.-S. Ho, “An efficient rectification algorithm for multi-view images in parallel camera array,” in Proceedings of 3DTV Conference (IEEE, 2008), pp. 61–64.

Lee, C.

Y.-S. Kang, C. Lee, and Y.-S. Ho, “An efficient rectification algorithm for multi-view images in parallel camera array,” in Proceedings of 3DTV Conference (IEEE, 2008), pp. 61–64.

Levoy, M.

V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.

Lin, Z.

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

Martinec, D.

T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).

Ota, M.

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Pajdla, T.

T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).

Svoboda, T.

T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).

Tanimoto, M.

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Tehrani, M. P.

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

Vaish, V.

V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.

Wang, L.

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

Wilburn, B.

V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.

Yendo, T.

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Zhang, Z.

Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000).
[CrossRef]

Comput. Graphics (1)

K. Deng, L. Wang, Z. Lin, T. Feng, and Z. Deng, “Correction and rectification of light fields,” Comput. Graphics 27, 169–177 (2003).

IEEE Signal Process. Mag. (1)

M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-viewpoint TV,” IEEE Signal Process. Mag. 28, 67–76 (2011).
[CrossRef]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000).
[CrossRef]

Presence: Teleoper. Virtual Environ. (1)

T. Svoboda, D. Martinec, and T. Pajdla, “A convenient multi-camera self-calibration for virtual environments,” Presence: Teleoper. Virtual Environ. 14, 407–422 (2005).

Other (3)

Y.-S. Kang, C. Lee, and Y.-S. Ho, “An efficient rectification algorithm for multi-view images in parallel camera array,” in Proceedings of 3DTV Conference (IEEE, 2008), pp. 61–64.

V. Vaish, B. Wilburn, N. Joshi, and M. Levoy, “Using plane + parallax for calibrating dense camera arrays,” in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (IEEE, 2004), pp. 2–9.

M. Ota, N. Fukushima, T. Yendo, M. Tanimoto, and T. Fujii, “Rectification of pure translation 2D camera array,” in Proceedings of the International Workshop on Advanced Image Technology (IWAIT, 2009), paper 0044.

Supplementary Material (1)

» Media 1: MP4 (3968 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (4)

Fig. 1.
Fig. 1.

Processing flow for calculating the homography matrix.

Fig. 2.
Fig. 2.

Placement of cameras in the simulation.

Fig. 3.
Fig. 3.

Simulation results.

Fig. 4.
Fig. 4.

Images of cameras 15, 16, and 17 (k=15,16,17) (Media 1).

Tables (1)

Tables Icon

Table 1. Camera Array System Setup

Equations (45)

Equations on this page are rendered with MathJax. Learn more.

fk=f+Δfk,f is a fixed value;
E[Δfk]=0,Δfk0.
[ku0kv0]=[u0v0]+[Δu0kΔv0k],u0andv0are fixed values;
E[Δu0k]=0,Δu0k0;
E[Δv0k]=0,Δv0k0.
θXk=ϑXk+ΔθXk,ϑXkis a fixed value;
θYk=ϑYk+ΔθYk,ϑYkis a fixed value;
θZk=ϑZk+ΔθZk,ϑZkis a fixed value;
E[ΔθXk]=0,ΔθXk0;
E[ΔθYk]=0,ΔθYk0;
E[ΔθZk]=0,ΔθZk0.
λx=A˙[R˙X+T],
x=[uv1],
A˙=[(f+Δf)ku0u0+Δu00(f+Δf)kvv0+Δv0001],
R˙=R˙XR˙YR˙Z,
R˙X=[1000cos(θX+ΔθX)sin(θX+ΔθX)0sin(θX+ΔθX)cos(θX+ΔθX)],
R˙Y=[cos(θY+ΔθY)0sin(θY+ΔθY)010sin(θY+ΔθY)0cos(θY+ΔθY)],
R˙Z=[cos(θZ+ΔθZ)sin(θZ+ΔθZ)0sin(θZ+ΔθZ)cos(θZ+ΔθZ)0001],
X=[XYZ],
T=[TXTYTZ].
X=λR.1A.1xR.1T.
Xk=λk(B0k+ΔθXkBXk+ΔθYkBYk+ΔθZkBZk+ΔfkBfk+Δu0kBu0k+Δv0kBv0k)+C0k+ΔθXkCXk+ΔθYkCYk+ΔθZkCZk.
argminX^{k=1N[(λkB0Xk+C0XkX^)2+(λkB0Yk+C0YkY^)2+(λkB0Zk+C0ZkZ^)2]+k=1NDXkΔθXk+k=1NDYkΔθYk+k=1NDZkΔθZk+k=1NDfkΔfk+k=1NDu0kΔu0k+k=1NDv0kΔv0k}.
argminX^{ k=1N[(λkB0Xk+C0XkX^)2+(λkB0Yk+C0YkY^)2+(λkB0Zk+C0ZkZ^)2]}.
A=[fku0u00fkvv0001],
RX(θX)=[1000cosθXsinθX0sinθXcosθX],
RY(θY)=[cosθY0sinθY010sinθY0cosθY],
RZ(θZ)=[cosθZsinθZ0sinθZcosθZ0001],
A1=[1/fku0u0/fku01/fkvv0/fkv001].
RX(θX)1=RX(θX);RY(θY)1andRZ(θZ)1are defined similarly.
R˙X1=RX(θXΔθX)RX(θX)ΔθXRX(π/2θX);R˙Y1andR˙Z1are defined similarly.
A˙1=[1/(f+Δf)ku0(u0+Δu0)/(f+Δf)ku01/(f+Δf)kv(v0+Δv0)/(f+Δf)kv001]
[(1Δf/f)/fku0(u0+Δu0)(1Δf/f)/fku0(1Δf/f)/fkv(v0+Δv0)(1Δf/f)/fkv001]
(1Δff)A1+Δu0fku[001000000]+Δv0fkv[000001000],
R˙1=R˙Z1R˙Y1R˙X1
=[RZ(θZ)ΔθZRZ(π/2θZ)][RY(θY)ΔθYRY(π/2θY)][RX(θX)ΔθXRX(π/2θX)]
=RZ(θZ)RY(θY)RX(θX)ΔθXRZ(θZ)RY(θY)RX(π/2θX)ΔθYRZ(θZ)RY(π/2θY)RX(θX)ΔθZRZ(π/2θZ)RY(θY)RX(θX).
X=λR˙1A˙1xR˙1T
=λ{RZ(θZ)RY(θY)RX(θX)ΔθXRZ(θZ)RY(θY)RX(π/2θX)ΔθYRZ(θZ)RY(π/2θY)RX(θX)ΔθZRZ(π/2θZ)RY(θY)RX(θX)}{(1Δff)A1+Δu0fku[001000000]+Δv0fkv[000001000]}x{RZ(θZ)RY(θY)RX(θX)ΔθXRZ(θZ)RY(θY)RX(π/2θX)ΔθYRZ(θZ)RY(π/2θY)RX(θX)ΔθZRZ(π/2θZ)RY(θY)RX(θX)}T.
X=λ(B0+ΔθXBX+ΔθYBY+ΔθZBZ+ΔfBf+Δu0Bu0+Δv0Bv0)+C0+ΔθXCX+ΔθYCY+ΔθZCZ.
argminX^k=1N{[XkX^]T[XkX^]}.
[XkX^]T[XkX^]=[λk(B0Xk+ΔθXkBXXk++Δv0kBv0Xk)+C0Xk++ΔθZkCZXkX^]2+[λk(B0Yk+)+C0Yk+Y^]2+[λk(B0Zk+)+C0Zk+Z^]2
(λkB0Xk+C0XkX^)2+(λkB0Yk+C0YkY^)2+(λkB0Zk+C0ZkZ^)2+2(λkB0Xk+C0XkX^)(λkΔθXkBXXk++ΔθZkCZXk)+2(λkB0Yk+C0YkY^)(λkΔθYkBXYk+)+2(λkB0Zk+C0ZkZ^)(λkΔθZkBXZk+)
=(λkB0Xk+C0XkX^)2+(λkB0Yk+C0YkY^)2+(λkB0Zk+C0ZkZ^)2+DXkΔθXk+DYkΔθYk+DZkΔθZk+DfkΔfk+Du0kΔu0k+Dv0kΔv0k.
argminX^k=1N{[XkX^]T[XkX^]}=argminX^{k=1N[(λkB0Xk+C0XkX^)2+(λkB0Yk+C0YkY^)2+(λkB0Zk+C0ZkZ^)2 ]+k=1NDXkΔθXk+k=1NDYkΔθYk+k=1NDZkΔθZk+k=1NDfkΔfk+k=1NDu0kΔu0k+k=1NDv0kΔv0k}.

Metrics