Abstract

In this paper, a novel optimization-based stitching method is presented. It minimizes an energy function defined with derivatives up to the second order. We have identified some appropriate choices for its parameters, allowing it to reduce artifacts such as ghosting, color inconsistency, and misalignment. To accelerate the computation, a multi-resolution technique is introduced. The significant speedup and memory saving make it possible for use in hand-held capturing devices.

© 2007 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. F. P. Junior and N. J. Leite, "A Morphological Algorithm for Photomosaicking," 8th European Signal Processing Conference 3, 1881-1884 (1996).
  2. F. Meyer and S. Beucher, "Morphological Segmentation," J. Visual Comm. and Image Representation 1, 21-46 (1990).
    [CrossRef]
  3. M. Uyttendaele, A. Eden, and R. Szeliski, "Eliminating Ghosting and Exposure Artifacts in Image Mosaics," IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2, 509-516 (2001).
  4. P. Soille, "Morphological Image Compositing," IEEE Trans. Pattern Analysis and Machine Intelligence 28, 673- 683 (2006).
    [CrossRef]
  5. A. Levin, A. Zomet, S. Peleg, and Y. Weiss, "Seamless image stitching in the Gradient Domain," 8th European Conference on Computer Vision 4, 377-389 (2004).
  6. C. T. Hsu and J. L. Wu, "Multiresolution Mosaic," IEEE Trans. Consumer Electronics 42, 981-990 (1996).
    [CrossRef]
  7. M. S. Su, W. L. Hwang, and K. Y. Cheng, "Variational calculus approach to multiresolution image mosaic," International Conference on Image Processing 2, 245 - 245 (2001).
  8. M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
    [CrossRef]
  9. A. A. Efros andW. T. Freeman, "Image quilting for texture synthesis and transfer," SIGGRAPH ’01: 28th Annual Conference on Computer Graphics and Interactive Techniques, 341-346 (2001).
  10. S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).
  11. S. T. Suen, E. Y. Lam, and K. K. Wong, "Photographic mosaic for camera phones based on minimization of curvature value variations," Tech. rep., Department of Electrical and Electronic Engineering, the University of Hong Kong (2006), http://www.eee.hku.hk/research/research_reports.htm.
  12. G. Strang and T. Nguyen, Wavelets and Filter Banks (MA: Wellesley-Cambridge Press, 1996).
  13. "The Panorama Factory V4.4 for Windows XP," http://www.panoramafactory.com/.

2006 (2)

P. Soille, "Morphological Image Compositing," IEEE Trans. Pattern Analysis and Machine Intelligence 28, 673- 683 (2006).
[CrossRef]

S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).

2004 (1)

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
[CrossRef]

1996 (1)

C. T. Hsu and J. L. Wu, "Multiresolution Mosaic," IEEE Trans. Consumer Electronics 42, 981-990 (1996).
[CrossRef]

1990 (1)

F. Meyer and S. Beucher, "Morphological Segmentation," J. Visual Comm. and Image Representation 1, 21-46 (1990).
[CrossRef]

Beucher, S.

F. Meyer and S. Beucher, "Morphological Segmentation," J. Visual Comm. and Image Representation 1, 21-46 (1990).
[CrossRef]

Cheng, K. Y.

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
[CrossRef]

Hsu, C. T.

C. T. Hsu and J. L. Wu, "Multiresolution Mosaic," IEEE Trans. Consumer Electronics 42, 981-990 (1996).
[CrossRef]

Hwang, W. L.

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
[CrossRef]

Lam, E. Y.

S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).

Meyer, F.

F. Meyer and S. Beucher, "Morphological Segmentation," J. Visual Comm. and Image Representation 1, 21-46 (1990).
[CrossRef]

Soille, P.

P. Soille, "Morphological Image Compositing," IEEE Trans. Pattern Analysis and Machine Intelligence 28, 673- 683 (2006).
[CrossRef]

Su, M. S.

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
[CrossRef]

Suen, S. T.

S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).

Wong, K. K.

S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).

Wu, J. L.

C. T. Hsu and J. L. Wu, "Multiresolution Mosaic," IEEE Trans. Consumer Electronics 42, 981-990 (1996).
[CrossRef]

IEEE Trans. Consumer Electronics (1)

C. T. Hsu and J. L. Wu, "Multiresolution Mosaic," IEEE Trans. Consumer Electronics 42, 981-990 (1996).
[CrossRef]

IEEE Trans. Image Processing (1)

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Analysis on multiresolution mosaic images," IEEE Trans. Image Processing 13, 952-959 (2004).
[CrossRef]

IEEE Trans. Pattern Analysis and Machine Intelligence (1)

P. Soille, "Morphological Image Compositing," IEEE Trans. Pattern Analysis and Machine Intelligence 28, 673- 683 (2006).
[CrossRef]

J. Visual Comm. and Image Representation (1)

F. Meyer and S. Beucher, "Morphological Segmentation," J. Visual Comm. and Image Representation 1, 21-46 (1990).
[CrossRef]

Proc. SPIE (1)

S. T. Suen, E. Y. Lam, and K. K. Wong, "Digital photograph stitching with optimized matching of gradient and curvature," Proc. SPIE 6069, 139-154 (2006).

Other (8)

S. T. Suen, E. Y. Lam, and K. K. Wong, "Photographic mosaic for camera phones based on minimization of curvature value variations," Tech. rep., Department of Electrical and Electronic Engineering, the University of Hong Kong (2006), http://www.eee.hku.hk/research/research_reports.htm.

G. Strang and T. Nguyen, Wavelets and Filter Banks (MA: Wellesley-Cambridge Press, 1996).

"The Panorama Factory V4.4 for Windows XP," http://www.panoramafactory.com/.

F. P. Junior and N. J. Leite, "A Morphological Algorithm for Photomosaicking," 8th European Signal Processing Conference 3, 1881-1884 (1996).

M. Uyttendaele, A. Eden, and R. Szeliski, "Eliminating Ghosting and Exposure Artifacts in Image Mosaics," IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2, 509-516 (2001).

A. Levin, A. Zomet, S. Peleg, and Y. Weiss, "Seamless image stitching in the Gradient Domain," 8th European Conference on Computer Vision 4, 377-389 (2004).

A. A. Efros andW. T. Freeman, "Image quilting for texture synthesis and transfer," SIGGRAPH ’01: 28th Annual Conference on Computer Graphics and Interactive Techniques, 341-346 (2001).

M. S. Su, W. L. Hwang, and K. Y. Cheng, "Variational calculus approach to multiresolution image mosaic," International Conference on Image Processing 2, 245 - 245 (2001).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1.
Fig. 1.

Input photographs with moving objects warped by commercial software. The rectangular boxes roughly indicate their overlap region.

Fig. 2.
Fig. 2.

(a) (b) Results of pixel selection according to the curve generated by Araújo and Leite, and Soille, respectively. The cutting curves are indicated by black lines. The rectangular boxes mark the regions where the curve passes through a moving object.

Fig. 3.
Fig. 3.

(a) Result of pixel selection according to the curve generated by our method. No moving object is cut. (b) Binary mask α(x, y) corresponds to the curve.

Fig. 4.
Fig. 4.

Stitching Result of (a) Exposure Compensation, (b) Wavelet Blending, (c) Gradient Stitching and (d) Our method, assigning 1 to the middle 85% vertical portion of β 2(x,y).

Fig. 5.
Fig. 5.

A zoom-in region of the result (a) Wavelet Blending, (b) Gradient Stitching and (c) Our method. They illustrate the transition across the cutting curve.

Fig. 6.
Fig. 6.

(a) and (b) Input photographs for the stitching example described in [13].

Tables (3)

Tables Icon

Table 1. Summary of the smoothness metric (Ms ) and fidelity metric (Mf ) for the stitching methods: (a) Exposure Compensation, (b) Wavelet Blending, (c) Gradient Stitching, (d) Our method minimizing second derivatives over the middle 65% vertical portion, and (e) minimizing over the middle 85% portion.

Tables Icon

Table 2. Summary of the smoothness metric (Ms ) and fidelity metric (Mf ) for the stitching methods: (a) Exposure Compensation, (b) Wavelet Blending, (c) Gradient Stitching, (d) Our method minimizing second derivatives over the overlap region and (e) Minimizing throughout the mosaic.

Tables Icon

Table 3. Summary of the average memory usage (M), total running time (Tt ) and the ratio of time consumed by optimization (To ) to the total running time when stitching with different number of decomposition levels.

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

Ψ { g ̂ x y } = i = 0 2 ( x y Ω 1 α x y β i x y D i ( g ̂ x y g 1 x y ) + x y Ω 2 ( 1 α x y ) β i x y D i ( g ̂ x y g 2 x y ) ) .
a x y = g 1 x y g 2 x y .
b x y = ρ A ( g 1 x y ) ρ A ( g 2 x y ) ,
c x y = b x y a x y .
g ̂ 0 x y = α x y g 1 x y + ( 1 α x y ) g 2 x y .

Metrics