Abstract

An unstructured triangulation approach, new to our knowledge, is proposed to apply triangular meshes for representing and rendering a scene on a cubic panorama (CP). It sophisticatedly converts a complicated three-dimensional triangulation into a simple three-step triangulation. First, a two-dimensional Delaunay triangulation is individually carried out on each face. Second, an improved polygonal triangulation is implemented in the intermediate regions of each of two faces. Third, a cobweblike triangulation is designed for the remaining intermediate regions after unfolding four faces to the top/bottom face. Since the last two steps well solve the boundary problem arising from cube edges, the triangulation with irregular-distribution feature points is implemented in a CP as a whole. The triangular meshes can be warped from multiple reference CPs onto an arbitrary viewpoint by face-to-face homography transformations. The experiments indicate that the proposed triangulation approach provides a good modeling for the scene with photorealistic rendered CPs.

© 2011 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1995), pp. 29–38.
  2. Wikipedia, “Google Street View” (2011), http://en.wikipedia.org/wiki/Google_Street_View.
  3. J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
    [CrossRef]
  4. C. Zhang, “A survey on image-based rendering: representation, sampling and compression,” Signal Process. Image Commun. 19, 1–28 (2004).
    [CrossRef]
  5. M. Lhuillier and Q. Long, “Match propagation for image-based modeling and rendering,” IEEE Trans. Pattern Anal. Machine Intell. 24, 1140–1146 (2002).
    [CrossRef]
  6. C. L. Zitnick and S. B. Kang, “Stereo for image-based rendering using image over-segmentation,” Int. J. Comput. Vis. 75, 49–65 (2007).
    [CrossRef]
  7. Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process. 6, 1270–1280 (1997).
    [CrossRef] [PubMed]
  8. A. M. K. Siu and R. W. H. Lau, “Image registration for image-based rendering,” IEEE Trans. Image Process. 14, 241–252 (2005).
    [CrossRef] [PubMed]
  9. A. M. K. Siu and R. W. H. Lau, “Relief occlusion-adaptive meshes for 3D imaging,” in Proceedings of the International Conference on Multimedia and Expo (IEEE, 2003), pp. 101–104.
  10. S. M. Seitz and C. R. Dyer, “Physically-valid view synthesis by image interpolation,” in Proceedings of IEEE Workshop on Representation of Visual Scenes (IEEE, 1995), pp. 18–25.
    [CrossRef]
  11. S. M. Seitz and C. R. Dyer, “View morphing,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1996), pp. 21–30.
  12. M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
    [CrossRef]
  13. F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
    [CrossRef]
  14. S. Ince, “Occlusion-aware view interpolation,” EURASIP J. Image Video Process. 2008, 803231 (2008).
    [CrossRef]
  15. S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
    [CrossRef]
  16. C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
    [CrossRef]
  17. M. Beermann and E. Dubois, “Acquisition processing chain for dynamic panoramic image sequences,” in IEEE International Conference on Image Processing (IEEE, 2007), pp. 217–220.

2011 (1)

Wikipedia, “Google Street View” (2011), http://en.wikipedia.org/wiki/Google_Street_View.

2010 (3)

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
[CrossRef]

2009 (2)

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
[CrossRef]

2008 (1)

S. Ince, “Occlusion-aware view interpolation,” EURASIP J. Image Video Process. 2008, 803231 (2008).
[CrossRef]

2007 (2)

M. Beermann and E. Dubois, “Acquisition processing chain for dynamic panoramic image sequences,” in IEEE International Conference on Image Processing (IEEE, 2007), pp. 217–220.

C. L. Zitnick and S. B. Kang, “Stereo for image-based rendering using image over-segmentation,” Int. J. Comput. Vis. 75, 49–65 (2007).
[CrossRef]

2005 (1)

A. M. K. Siu and R. W. H. Lau, “Image registration for image-based rendering,” IEEE Trans. Image Process. 14, 241–252 (2005).
[CrossRef] [PubMed]

2004 (1)

C. Zhang, “A survey on image-based rendering: representation, sampling and compression,” Signal Process. Image Commun. 19, 1–28 (2004).
[CrossRef]

2003 (1)

A. M. K. Siu and R. W. H. Lau, “Relief occlusion-adaptive meshes for 3D imaging,” in Proceedings of the International Conference on Multimedia and Expo (IEEE, 2003), pp. 101–104.

2002 (1)

M. Lhuillier and Q. Long, “Match propagation for image-based modeling and rendering,” IEEE Trans. Pattern Anal. Machine Intell. 24, 1140–1146 (2002).
[CrossRef]

1997 (1)

Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process. 6, 1270–1280 (1997).
[CrossRef] [PubMed]

1996 (1)

S. M. Seitz and C. R. Dyer, “View morphing,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1996), pp. 21–30.

1995 (2)

S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1995), pp. 29–38.

S. M. Seitz and C. R. Dyer, “Physically-valid view synthesis by image interpolation,” in Proceedings of IEEE Workshop on Representation of Visual Scenes (IEEE, 1995), pp. 18–25.
[CrossRef]

Ackermann, J.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Altunbasak, Y.

Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process. 6, 1270–1280 (1997).
[CrossRef] [PubMed]

Beermann, M.

M. Beermann and E. Dubois, “Acquisition processing chain for dynamic panoramic image sequences,” in IEEE International Conference on Image Processing (IEEE, 2007), pp. 217–220.

Biber, P.

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

Busch, F.

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

Chen, B.

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

Chen, S. E.

S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1995), pp. 29–38.

Cohen, M.

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

Dubois, E.

C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
[CrossRef]

F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
[CrossRef]

M. Beermann and E. Dubois, “Acquisition processing chain for dynamic panoramic image sequences,” in IEEE International Conference on Image Processing (IEEE, 2007), pp. 217–220.

Dyer, C. R.

S. M. Seitz and C. R. Dyer, “View morphing,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1996), pp. 21–30.

S. M. Seitz and C. R. Dyer, “Physically-valid view synthesis by image interpolation,” in Proceedings of IEEE Workshop on Representation of Visual Scenes (IEEE, 1995), pp. 18–25.
[CrossRef]

Fleck, S.

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

Fuhrmann, S.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Goesele, M.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Haubold, C.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Ince, S.

S. Ince, “Occlusion-aware view interpolation,” EURASIP J. Image Video Process. 2008, 803231 (2008).
[CrossRef]

Kang, S. B.

C. L. Zitnick and S. B. Kang, “Stereo for image-based rendering using image over-segmentation,” Int. J. Comput. Vis. 75, 49–65 (2007).
[CrossRef]

Klowsky, R.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Kopf, J.

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

Laganiere, R.

F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
[CrossRef]

Lau, R. W. H.

A. M. K. Siu and R. W. H. Lau, “Image registration for image-based rendering,” IEEE Trans. Image Process. 14, 241–252 (2005).
[CrossRef] [PubMed]

A. M. K. Siu and R. W. H. Lau, “Relief occlusion-adaptive meshes for 3D imaging,” in Proceedings of the International Conference on Multimedia and Expo (IEEE, 2003), pp. 101–104.

Lhuillier, M.

M. Lhuillier and Q. Long, “Match propagation for image-based modeling and rendering,” IEEE Trans. Pattern Anal. Machine Intell. 24, 1140–1146 (2002).
[CrossRef]

Long, Q.

M. Lhuillier and Q. Long, “Match propagation for image-based modeling and rendering,” IEEE Trans. Pattern Anal. Machine Intell. 24, 1140–1146 (2002).
[CrossRef]

Seitz, S. M.

S. M. Seitz and C. R. Dyer, “View morphing,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1996), pp. 21–30.

S. M. Seitz and C. R. Dyer, “Physically-valid view synthesis by image interpolation,” in Proceedings of IEEE Workshop on Representation of Visual Scenes (IEEE, 1995), pp. 18–25.
[CrossRef]

Shi, F.

F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
[CrossRef]

Siu, A. M. K.

A. M. K. Siu and R. W. H. Lau, “Image registration for image-based rendering,” IEEE Trans. Image Process. 14, 241–252 (2005).
[CrossRef] [PubMed]

A. M. K. Siu and R. W. H. Lau, “Relief occlusion-adaptive meshes for 3D imaging,” in Proceedings of the International Conference on Multimedia and Expo (IEEE, 2003), pp. 101–104.

Steedly, D.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

Straper, W.

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

Szeliski, R.

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

Tekalp, A. M.

Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process. 6, 1270–1280 (1997).
[CrossRef] [PubMed]

Zhang, C.

C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
[CrossRef]

C. Zhang, “A survey on image-based rendering: representation, sampling and compression,” Signal Process. Image Commun. 19, 1–28 (2004).
[CrossRef]

Zhao, Y.

C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
[CrossRef]

Zitnick, C. L.

C. L. Zitnick and S. B. Kang, “Stereo for image-based rendering using image over-segmentation,” Int. J. Comput. Vis. 75, 49–65 (2007).
[CrossRef]

ACM Trans. Graph. (2)

J. Kopf, B. Chen, R. Szeliski, and M. Cohen, “Street slide: browsing street level imagery,” ACM Trans. Graph. 29(4), Article 96 (2010).
[CrossRef]

M. Goesele, J. Ackermann, S. Fuhrmann, C. Haubold, R. Klowsky, D. Steedly, and R. Szeliski, “Ambient point clouds for view interpolation,” ACM Trans. Graph. 29, 95 (2010).
[CrossRef]

EURASIP J. Image Video Process. (1)

S. Ince, “Occlusion-aware view interpolation,” EURASIP J. Image Video Process. 2008, 803231 (2008).
[CrossRef]

IEEE Trans. Image Process. (2)

Y. Altunbasak and A. M. Tekalp, “Occlusion-adaptive, content-based mesh design and forward tracking,” IEEE Trans. Image Process. 6, 1270–1280 (1997).
[CrossRef] [PubMed]

A. M. K. Siu and R. W. H. Lau, “Image registration for image-based rendering,” IEEE Trans. Image Process. 14, 241–252 (2005).
[CrossRef] [PubMed]

IEEE Trans. Pattern Anal. Machine Intell. (1)

M. Lhuillier and Q. Long, “Match propagation for image-based modeling and rendering,” IEEE Trans. Pattern Anal. Machine Intell. 24, 1140–1146 (2002).
[CrossRef]

Image Vis. Comput. (1)

S. Fleck, F. Busch, P. Biber, and W. Straper, “Graph cut based panoramic 3D modeling and ground truth comparison with a mobile platform—the Wagele,” Image Vis. Comput. 27, 141–152 (2009).
[CrossRef]

Int. J. Comput. Vis. (1)

C. L. Zitnick and S. B. Kang, “Stereo for image-based rendering using image over-segmentation,” Int. J. Comput. Vis. 75, 49–65 (2007).
[CrossRef]

Signal Process. Image Commun. (1)

C. Zhang, “A survey on image-based rendering: representation, sampling and compression,” Signal Process. Image Commun. 19, 1–28 (2004).
[CrossRef]

Other (8)

S. E. Chen, “QuickTime VR: an image-based approach to virtual environment navigation,” in Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1995), pp. 29–38.

Wikipedia, “Google Street View” (2011), http://en.wikipedia.org/wiki/Google_Street_View.

A. M. K. Siu and R. W. H. Lau, “Relief occlusion-adaptive meshes for 3D imaging,” in Proceedings of the International Conference on Multimedia and Expo (IEEE, 2003), pp. 101–104.

S. M. Seitz and C. R. Dyer, “Physically-valid view synthesis by image interpolation,” in Proceedings of IEEE Workshop on Representation of Visual Scenes (IEEE, 1995), pp. 18–25.
[CrossRef]

S. M. Seitz and C. R. Dyer, “View morphing,” in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (ACM, 1996), pp. 21–30.

F. Shi, R. Laganiere, and E. Dubois, “On the use of ray-tracing for viewpoint interpolation in panoramic imagery,” in Proceedings of the 2009 Canadian Conference on Computer and Robot Vision (IEEE, 2009), pp. 200–207.
[CrossRef]

C. Zhang, E. Dubois, and Y. Zhao, “Intermediate cubic-panorama synthesis based on triangular re-projection,” in 17th IEEE International Conference on Image Processing (IEEE, 2010), pp. 3985–3988.
[CrossRef]

M. Beermann and E. Dubois, “Acquisition processing chain for dynamic panoramic image sequences,” in IEEE International Conference on Image Processing (IEEE, 2007), pp. 217–220.

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (23)

Fig. 1
Fig. 1

Pipeline of CP triangulation.

Fig. 2
Fig. 2

Step I of CP triangulation.

Fig. 3
Fig. 3

Step II of CP triangulation.

Fig. 4
Fig. 4

Step III of CP triangulation.

Fig. 5
Fig. 5

Intersection judgment of two segments on CP.

Fig. 6
Fig. 6

(a) Self-intersection; (b) rectified polygon without self- intersection.

Fig. 7
Fig. 7

A convex vertex c is not separable in both cases.

Fig. 8
Fig. 8

Cobweblike triangulation.

Fig. 9
Fig. 9

Removing invalid segments of a knot.

Fig. 10
Fig. 10

Rectifying the outer boundary.

Fig. 11
Fig. 11

Topological ordering confliction.

Fig. 12
Fig. 12

Three kinds of different projections of a triangle Δ a b d .

Fig. 13
Fig. 13

Face-to-face homography transformation.

Fig. 14
Fig. 14

Triangulation given 50 randomly distributed points on a cubic surface.

Fig. 15
Fig. 15

Triangulation given 500 randomly distributed points on a cubic surface.

Fig. 16
Fig. 16

Triangulation given 2000 randomly distributed points on a cubic surface.

Fig. 17
Fig. 17

Relationship between the run time and the number of input points.

Fig. 18
Fig. 18

Matching points in the first CP.

Fig. 19
Fig. 19

Matching points in the second CP.

Fig. 20
Fig. 20

Triangles in the first CP after performing CP triangulation.

Fig. 21
Fig. 21

Triangles in the second CP after performing CP triangulation.

Fig. 22
Fig. 22

Synthesized CP.

Fig. 23
Fig. 23

Ground truth of the synthesized CP.

Equations (1)

Equations on this page are rendered with MathJax. Learn more.

H face i 1 face j 2 = K R face j 1 R 12 1 ( I + T 12 π ˜ T ) R face i K 1 ,

Metrics