Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Diffusion-based three-dimensional reconstruction of complex surface using monocular vision

Open Access Open Access

Abstract

Three-dimensional (3D) reconstruction based on optical diffusion has certain significant advantages, such as its capacity for high-precision depth estimation with a small lens, distant-object depth estimation, a monocular vision basis, and no required camera or scene adjustment. However, few mathematical models to relate the depth information acquired using this technique to the basic principles of intensity distribution during optical diffusion have been proposed. In this paper, the heat diffusion equation of physics is applied in order to construct a mathematical model of the intensity distribution during optical diffusion. Hence, a high-precision 3D reconstruction method with optical diffusion based on the heat diffusion equation is proposed. First, the heat diffusion equation is analyzed and an optical diffusion model is introduced to explain the basic principles of the diffusion imaging process. Second, the novel 3D reconstruction method based on global heat diffusion is proposed, which incorporates the relationship between the depth information and the degree of diffusion. Finally, a simulation involving synthetic images and an experiment using five playing cards are conducted, with the results confirming the effectiveness and feasibility of the proposed method.

© 2015 Optical Society of America

1. Introduction

The three-dimensional (3D) reconstruction of a complex surface in computer vision is achieved by constructing a mapping function between the depth and the brightness information of 2D images. In recent years, various 3D reconstruction methods have been developed, including the depth from stereo (DFS), depth from focus (DFF), and depth from defocus (DFD) techniques, which have been investigated using real-world applications [1].

The DFS estimates depth using two images of the same scene, which are captured by cameras at different positions and with different orientations [2,3 ]. However, as this technique requires that feature points of these two images be extracted and matched, the computational cost of DFS is too large for it to be used in real-time applications. In contrast, DFF depth estimation uses a mapping relation between the focus and depth. A sequence of images with different depths is obtained, the degree of focus is determined using a measurement operator [4,5 ], and the desired depth when the measurement value is maximal or minimal is obtained. Compared to DFS, DFF is simple in principle, but its estimation accuracy is highly dependent on the number of acquired images and the sensitivity of the measurement operator [6,7 ]. Finally, DFD, which was developed by Pentland [8], measures the degree of blurring of two defocused images, and then estimates depth using a point spread function, such as a Gaussian function. DFD has proven to be an effective depth reconstruction method for the following reasons: 1) Only two defocused images of a scene are necessary; 2) Matching and masking are not required; 3) It is effective in both the frequency and spatial domains [9–14 ].

However, although DFD is comparatively mature as regards application in macro fields, some problems still occur when it is used in certain real-world applications. These problems are summarized as followings: 1) To improve the DFD reconstruction precision, the most direct approach is to increase the lens aperture. However, fabricating a large-aperture lens for real-world applications is time consuming and expensive. Moreover, when the aperture of a lens increases, its depth of field decreases accordingly, and the degree of blurring of the defocused image becomes difficult to measure. In addition, when a large aperture is used, because of the considerable differences between the focused image and its corresponding defocused image, the realism of the scene may be lost. 2) The DFD depth sensitivity is inversely proportional to the square of the object distance. Further, loss of depth sensitivity is unavoidable when a high-resolution camera is used to improve the depth-estimation precision. However, in many scenarios, it is necessary to place objects far from the camera in order to achieve a reasonable field of view. 3) In DFD techniques, it is necessary to adjust some camera parameters or the distance between the object and camera in order to capture two defocused images using monocular vision. However, in some applications, adjusting the camera parameters damages the camera, and moving the camera or the object during depth reconstruction is inconvenient.

In order to overcome these difficulties, a surface reconstruction method that can assure both depth reconstruction precision and depth sensitivity with a small aperture lens under monocular vision conditions is very necessary. Therefore, the use of optical diffusers in optical imaging has recently been considered, and a technique known as “depth reconstruction from diffusion” (DRFD) has been proposed, which is designed to reconstruct the surface of a scene. A diffuser is a light-scattering optical element that is widely used to soften and shape light in illumination and display applications [15,16 ]. This device can convert an incident ray into a cluster of scattered rays. Therefore, when it is placed in front of an object, the captured image is blurred and has a similar appearance to a defocused image; this can be seen in Fig. 1 , which is a diffused image of a wrinkled newspaper. A diffused image can be formulated as a convolution between a radiance image and a diffusion blur kernel with a locally constant diffusion angle, with the diffusion blur kernel being determined by both the diffusion function and the object-to-diffuser distance. If the diffusion function or diffusion angle is known, it is possible to calculate the object-to-diffuser distance. This is the core principle of the depth reconstruction from diffusion technique.

 figure: Fig. 1

Fig. 1 Diffusion theory and a diffused image.

Download Full Size | PDF

Compared to traditional DFD, DRFD has significant advantages, such as its capacity for high-precision depth estimation with a small lens, distant-object depth estimation, a monocular vision basis, no required camera or scene adjustment, and reduced sensitivity to lens aberrations [17–19 ]. However, since Zhou et. al. first proposed the original concept of DRFD, and mentioned that the basic principle of this technique is analogous to that of conventional DFD [19], this topic has been only minimally researched. Further, few mathematical models relating depth information to the basic principles of intensity distribution during optical diffusion have been proposed.

In this paper, a mathematical model of optical diffusion is constructed and a high-precision 3D reconstruction method with optical diffusion is proposed, which is based on the heat diffusion equation in physics.

Our present approach is novel in several respects and provides a fast, noninvasive, and precise 3D measurement. First, the heat diffusion equation of physics is analyzed and an optical diffusion model is introduced to explain the basic principles of the diffusion imaging process. Second, we consider an image obtained without an optical diffuser as constituting a diffused version of the diffused image acquired when an optical diffuser is placed in front of the scene. By appropriately choosing the reference image at each spatial location, such optical diffusion is always in the forward direction, and the degree of diffusion depends on the depth of the scene at that location. This diffusion is independent of the camera parameters, which means that adjustment of the aperture size or object distance is not required. Finally, the heat diffusion equation is applied in order to construct a mathematical model of the intensity distribution during optical diffusion. Hence, a high-precision DRFD based on the heat diffusion equation is proposed.

The contents of this paper are organized as follows. Firstly, in section II, the heat diffusion equation in physics is analyzed, and an optical diffusion model is introduced to explain the basic principle of the diffusion imaging process. Secondly, a novel 3D reconstruction method based on the diffusion model is proposed in section III. Subsequently, in section IV, the experiment results and error analysis based on evaluations of the new method are given. Finally, section V presents the conclusion of this paper.

2. Imaging model of optical diffusion

2.1 Heat diffusion in physics

In physics, heat diffusion of most fluids and some homogeneous solid materials like gels is same in every direction, and it is called isotropic heat diffusion, characterized by a single diffusion coefficient a.

Given the concentration u and the flux J, Fick's first law gives a relationship between the flux and the concentration gradient as,

J(x,y,t)=au(x,y,t)
where x and y are the horizontal and vertical coordinates of a diffusion point, respectively; the diffusion coefficient a controls the intensity of diffusion, and it is nonnegative; t is the diffusion time expending on the process of diffusion caloric in physics; “” denotes the gradient operator, =[xy]T.

Then, given the conservation of mass, the continuity equation relating the time derivative of concentration to the divergence of flux can be denoted as,

u(x,y,t)t=J(x,y,t)
where “” is the divergence operator, x+y.

Putting Eq. (1) and Eq. (2) together, we obtain the diffusion equation shown as,

u(x,y,t)t=a2u(x,y,t)

Therefore, the isotropic heat diffusion model can be denotes as,

{u(x,y,t)t=a2u(x,y,t)u(x,y,0)=u0(x,y)
where u 0(x, y) is the initial condition.

If the diffusion coefficient a varies spatially, the isotropic heat diffusion is transformed into the inhomogeneous heat diffusion, and its model is obtained yielding the following equation,

{u(x,y,t)t=(a(x,y)u(x,y,t))u(x,y,0)=u0(x,y)

The inhomogeneous heat diffusion arises from the phenomenon of the heat diffusion in physics, and it spatially controls the diffusion intensity of every point with the heat diffusion coefficient. Therefore, as the inhomogeneous diffusion action increases, the diffusion region with low density contrast will become smoother, while the one with high density contrast will be preserved unchanged. Beneficiated from this property, the diffusion equation has recently been used in image processing, such as image enhancement. In addition, optical diffusers as hardware to simulate optical diffusion are also designed to shape and soft an incident illumination in computer vision.

2.2. Imaging model for an optical diffuser

In the typical configuration, the incident illumination hits an optical diffuser on the patterned side of the substrate, and a small part of it is reflected at the surface while a majority of it penetrates into the substrate. Upon exiting the substrate the light refracts at the surface achieving the designed energy distribution or light shape. Figure 2 is the geometry of a diffusion process and the energy distribution of a laser source where θ is the diffusion angle. Therefore, optical diffusers can precisely shape, control, and distribute light into a cone angle (as shown in Fig. 3 ), which can be either symmetrical (circular) or asymmetrical (elliptical). This property is particularly important for a diffuser with a large angle to reduce the effects of total internal reflection and allows the diffuser to work as intended, and the scattering property of energy passing through an optical diffuser is called point spread function (PSF).

 figure: Fig. 2

Fig. 2 Geometry of a diffusion process and its energy distribution.

Download Full Size | PDF

 figure: Fig. 3

Fig. 3 Different diffusion forms (uniform, batwing and Gaussian).

Download Full Size | PDF

Therefore, if an optical diffuser is parallel placed between a source point and a pinhole camera, because of optical diffusion, the light from the source point is scattered, and the captured image of the source point is a round spot with a radius of b, as shown in Fig. 4 , where u is the distance of the object from the principal; v is the distance of the focused image from the len’s plane; U is the distance between the diffuser and the principal; Z is the distance between the object and the diffuser; AB is the diffusion size; α is the field angle; θ is the diffusion angle of the optical diffuser.

 figure: Fig. 4

Fig. 4 Geometry of diffusion in a pinhole camera.

Download Full Size | PDF

As shown in Fig. 4, the light from an arbitrary source point P is scattered by the diffuser. Due to the limit of the diffusion angle θ, only the light scattered from a specific region AB can reach the pinhole O. From the viewpoint of the pinhole, the line AB appears on the diffuser plane instead of the actual point P. With the principle of optical diffusion, the radius of the diffused spot resulted from optical diffusion is [19],

b=vUAB2=vtanθcos2αZZ+U
From Eq. (6), it can be seen that with a given U, the diffusion size AB is determined by both the object-to-diffuser distance Z and the diffusion angle θ.

Consider a scene with a smooth Lambertian surface. We take the image of a scene from the same point of view and assume that scene and illumination are static with respect to the camera. Under these conditions, we can represent the surface of the scene with a depth map Z and the radiance of the scene with a function r. If we use a real aperture camera and a diffuser with a diffusion angle of θ, the irrandiance I measured on the image plane can be approximated via the following equation,

I(x1,y1)=h(x1,y1,x,y,b)r(x,y)dxdy
where h is the PSF; x, x 1, y and y 1 are the horizontal and vertical coordinates of two different diffusion points.

An important case that we will consider is that of a scene made of an equifocal plane, that is, a plane parallel to the image plane. In this case, the depth map satisfies Z(x, y) = Z, the PSF h is shift-invariant, that is, h(x, y, x 1, y 1, b) = h(x-x 1, y-y 1, b). Hence, the image formation model in Eq. (7) becomes the following simple convolution,

I=h(,b)r

For most optical diffusers, the PSF can be denoted as a Gaussian function,

h(x,y,x1,y1)=12πσ2exp((xx1)2+(yy1)22σ2)
where σ denotes the spread parameter of the Gaussian kernel, and the standard deviation σγb for a certain constant γ>0 can be determined via a calibration procedure. More in general, one can approximate the PSF with other functions, as long as they satisfy the following normalization property,

h(x1,y1,x,y,b)dx1dy1=1

When the distance map Z is an equifocal plane, the PSF is approximated by a shift-invariant Gaussian, the image model in Eq. (7) can be formulated in terms of heat equation (see references [20–22 ]),

{u˙(x,y,t)=aΔu(x,y,t)a[0,)t(0,)u(x,y,0)=r(x,y)
where r(x, y) is the radiance image without diffusion, the solution u at a time t = τ plays the role of an image I(x, y) = u(x, y, τ), captured with a certain setting that is related to τ. The “dot” denotes differentiation in time, that is,u˙=˙ut, “Δ” denotes the Laplacian operator, and Δu=2ux2+2uy2.

The varianceσis related to the diffusion coefficient a via,

σ2=2ta

In order to verify Eq. (12), first we put Eq. (12) into Eq. (9), and obtain the PSF as,

h(x,y,x1,y1,t)=14πtaexp((xx1)2+(yy1)24ta)

Then, the diffused image at time t can be denoted as,

I(x1,y1,t)=(14πtaexp((xx1)2+(yy1)24ta))r(x,y)

Finally, we put Eq. (14) into Eq. (11), and it can be seen that Eq. (14) is the solution of Eq. (11). Therefore, the relationship in Eq. (12) is reasonable.

When the distance map Z is not an equifocal plane, the PSF is in general shift varying. The equivalence with the isotropic heat equation does not hold, and the diffusion process can be formulated in terms of the inhomogeneous diffusion equation as,

{u˙(x,y,t)=(a(x,y)u(x,y,t))t(0,)u(x,y,0)=r(x,y)

By assuming the surface Z is smooth, we can relate again the diffusion coefficient a to the space-varying variance σ via,

σ2(x,y)=2ta(x,y)

Therefore, the imaging process with optical diffusion can be formulated in terms of heat equation, and the solution of the diffusion equation can be obtained in terms of convolution of the image with a temporally evolving Gaussian kernel. Since σγb, it is immediate to see that a(x, y) encodes the depth map Z of the scene via,

a(x,y)=σ2(x,y)2t=γ2v2tan2(θ)2tcos4α(ZZ+U)2

3. 3D reconstruction of complex surface based on optical diffusion

As we have seen in Section 2, when the surface Z is not an equifocal plane, the corresponding PSF is shift varying, and we cannot use the homogeneous heat equation to model the diffused images. Therefore, we have introduced the inhomogeneous diffusion Eq. (15) by allowing the diffusion coefficient a to vary spatially as a function of locations on the image.

Suppose E 1(x, y) is the image before an optical diffuser is placed and E 2(x, y) is the diffused image after an optical diffuser is parallel placed before the scene, in this section, we will propose a global DRFD method with respect to E 1(x, y) and E 2(x, y). First, based on the diffusion process in section 2, the following functions can be given,

{u˙(x,y,t)=(a(x,y)u(x,y,t))t(0,)u(x,y,0)=E1(x,y)u(x,y,t2)=E2(x,y)

Recall that in Section 2, we required the diffusion coefficient a to be a smooth function that satisfies a number of properties in order to guarantee that energy is conserved in the imaging process. Since E 2(x, y) is the diffusion result of E 1(x, y), the diffusion from E 1(x, y) to E 2(x, y) is a forward diffusion, therefore, a(x, y) ≥ 0.

Notice that a(x, y) = 0,

γ2v2tan2(θ)2tcos4α(ZZ+U)2=0

When Z = 0 or θ = 0, there is no diffusion between E 1(x, y) and E 2(x, y). Therefore, we can rewrite here in a more compete form, including boundary conditions, as,

{u˙(x,y,t)=(a(x,y)u(x,y,t))t(0,)u(x,y,0)=E1(x,y)u(x,y,Δt)=E2(x,y)0=a(x,y)u(x,y,t)n(x,y)

Normally in an optical diffusion, θ≤90°, so tanθ≥0, and the depth value between the diffuser and the scene point can be denoted as,

Z=σUcos2α(γvtanθσcos2α)

This naturally suggests an iterative procedure to tackle the inverse problem of reconstructing shape from an image before diffusion and an image after diffusion. An intuitive description of the iteration is starting from an initial estimate of the depth map (for example, a flat plane such that a(x, y) = 0) and the corresponding partition, find the amount of diffusion for each image in the region where it is more blurred than the other image so that the two are close. The amount of diffusion required to match these two images encodes information on the depth of the scene. More formally, we could pose the problem as the minimization of the following functional,

Z˜=argminZ(x,y)(u(x,y,Δt)E2(x,y))2dxdy
where the two terms in the cost functional take into account the discrepancy between the simulated image u and the measured images E 1(x, y), E 2(x, y).

However, the optimization process above is ill posed, that is, the minimum may not exist, and even if it exists, it may not be stable with respect to data noise. A common way to regularize the problem is to add a Tikhonov Penalty,

Z˜=argminZ(x,y)(u(x,y,Δt)E2(x,y))2dxdy+ηZ(x,y)2+ηkZ(x,y)2
where the additional term imposes a smoothness constraint on the depth map. In practice, we use η>0,k>0 which are all very small, because this term has no practical influence on the cost energy denoted as,

F(Z)=(u(x,y,Δt)E2(x,y))2dxdy+ηZ2+ηkZ2

Therefore, the solution process is equal to the following,

Z˜=argminZF(Z)

Equation (25) is a dynamic optimization which can be solved by the gradient flow, the algorithm can be divided into the following steps (the detailed process can be seen in literature [20–22 ]), and the flow graph of our algorithm is shown in Fig. 5 .

 figure: Fig. 5

Fig. 5 Flow graph of our algorithm.

Download Full Size | PDF

  • (1) Give camera parameters f, γ, v, and set up the iteration parameters: the chosen threshold ε, the regularization parameter η, k, and the step size β;
  • (2) Capture the first image E 1 without optical diffusion; place an optical diffuser with the diffusion angle of θ in front of the camera and perpendicular to the optical axis, and capture the second image E 2. The distance from the camera to the diffuser is U;
  • (3) Compute Eq. (17) to obtain the diffusion coefficient;
  • (4) Compute Eq. (20);
  • (5) Compute Eq. (24) with the solution u(x,y,Δt) of Step (4). If the cost energy is below ε, the algorithm stops; Otherwise compute the following equation with the step size β,
    Zt=F'(Z)
  • (6) Compute Eq. (21), update the depth map, and return to Step (3).

4. Experiment

In order to validate the new proposed algorithm, we use a number of synthetic images and two real images of five playing cards to test it. First, in the simulation, the performance of the proposed algorithm is tested with a cosine plane and a box plane, respectively, and some basic parameters in the simulation are as follows: f = 12 mm, v = 12.2 mm, s 0 = 850 mm, F = 2, D = f/2, γ = 0.2, and θ = 10°. Then, the error maps for two synthetized planes are constructed and the mean-square-error of the proposed method is calculated to test the precision of this algorithm. In the real experiment, the camera that we use is Canon EOS 5D Mark III with a Canon EF 50 mm F/1.8 lens, and the optical diffuser with a diffusion angle of 10° in our experiment is from Edmund Optics.

4.1 Simulation with a diffusion angle of 10°

In this subsection, the simulation with a cosine plan is conducted. First, we synthesize the non-diffusion image when the focus-imaging-condition in the geometrical optics coincides and no optical diffuser is placed in front of the cosine plane. Then, the diffused image is synthesized considering of a diffusion angle of 10°. Finally, the global 3D surface is reconstructed with the algorithm in this paper, and the simulation results are shown in Fig. 6 to Fig. 8. Figure 6 is the synthesized images in which Fig. 6 (a) is the image before optical diffusion and Fig. 6(b) is that after diffusion; the depth maps of gray are shown in Fig. 7 in which Fig. 7(a) is the calculated depth map with our algorithm and Fig. 7(b) is the true depth map; Fig. 8 is the global 3D surface in which Fig. 8(a) is the estimated 3D surface with our algorithm and Fig. 8(b) is the true 3D surface.

 figure: Fig. 6

Fig. 6 The synthesized images of a consine plane.

Download Full Size | PDF

 figure: Fig. 7

Fig. 7 The depth maps of gray.

Download Full Size | PDF

 figure: Fig. 8

Fig. 8 The reconstructed 3D surface and the true 3D surface.

Download Full Size | PDF

From these figures, the following conclusion can be obtained,

  • 1) Because the depth of field of our simulated camera is much longer than the depth variation inside our sample, the image before optical diffusion can be considered as a focused image. After optical diffusion, the sample image is blurred, and the blurring degree of different points is different due to the distance between the points to the optical diffuser.
  • 2) The estimated depth map of gray has much similar bright features compared with the true depth map of gray, and the reconstructed 3D surface of our algorithm has the same waveform variation compared with the true surface.

In order to investigate the precision of the new algorism, we construct the error map ζ between the true depth Z and the estimated depth Ze with the cosine plane, and calculate the mean square error φ of the whole image. The calculation formulas are shown in Eq. (27) and Eq. (28),

ζ=Ze/Z1
φ=E[(Ze/Z1)2]

The error map is shown in Fig. 9 , where we can see that the maximal height of the error map is less than 0.012, and the average error is only 0.0057. Therefore, the average error is 0.57%. The mean-square-error of the whole image is 0.0058. At the edge of the reconstructed surface, the error value is higher than that in the middle of the image. The reason is that the optimization method we use to estimate the global 3D surface has some edge effect, and it is one of the future works to overcome it.

 figure: Fig. 9

Fig. 9 The error map between the estimated surface and the true surface.

Download Full Size | PDF

Secondly, the simulation with a rectangle plane is conducted, and in this simulation the sample surface is much sharper than the cosine plane. The simulation results are shown in Fig. 10 to Fig. 12, where Fig. 10 is the synthesized images in which Fig. 10(a) is the image before optical diffusion and Fig. 10(b) is that after diffusion; the depth maps of gray are shown in Fig. 11 in which Fig. 11(a) is the estimated depth map with our algorithm and Fig. 11(b) is the true depth map; Fig. 12 is the 3D surface of the rectangle plane, where Fig. 12(a) is the estimated 3D surface with our algorithm and Fig. 12(b) is the true 3D surface.

 figure: Fig. 10

Fig. 10 The synthesized images of a rectangle plane.

Download Full Size | PDF

 figure: Fig. 11

Fig. 11 The depth maps of gray.

Download Full Size | PDF

 figure: Fig. 12

Fig. 12 The reconstructed 3D surface and the true surface.

Download Full Size | PDF

From Fig. 11-Fig. 13 , we can see that although there are some noise points on the estimated depth map and the reconstruction 3D surface, the reconstructed 3D surface of our algorithm is close to the true surface, regardless of the shape of these surfaces. The error map is shown in Fig. 13, and the mean-square-error of the reconstruction result is 0.0046 in the second simulation.

 figure: Fig. 13

Fig. 13 The error map between the estimated surface and the true surface.

Download Full Size | PDF

4.2 Experiment with a diffusion angle of 10°

We use the images of five playing cards to validate our depth reconstruction method. The average height of our cards is 0.29mm, and the experimental results are shown in Fig. 14 to Fig. 16. Figure 14 is two experimental images, in which Fig. 14(a) is the image of the cards before optical diffusion and Fig. 14(b) is the diffused image; the global reconstructed surface of the arranged cards with our global 3D reconstruction method is shown in Fig. 15 and Fig. 16 , where Fig. 15 is the reconstructed depth map with a color bar and Fig. 16 is the 3D reconstructed surface. The unit of the depth axis is mm. In order to test the precision of our method in this experiment, we also construct the true 3D surface of our arranged cards because we already know the height of each card and the arrangement rule in this experiment, and the true surface is shown in Fig. 17 . The error surface between the true surface and the estimated surface is shown in Fig. 18 , and the calculation formula is as same as Eq. (27).

 figure: Fig. 14

Fig. 14 Focused image and diffused image of the arranged cards.

Download Full Size | PDF

 figure: Fig. 15

Fig. 15 Reconstructed surface of our method.

Download Full Size | PDF

 figure: Fig. 16

Fig. 16 3D reconstructed surface of our method.

Download Full Size | PDF

 figure: Fig. 17

Fig. 17 True 3D surface of the cards.

Download Full Size | PDF

 figure: Fig. 18

Fig. 18 Error surface of our method.

Download Full Size | PDF

From Fig. 16, Fig. 18 and our error calculation, we can obtain the following conclusion,

  • (1) Our global surface reconstruction method can precisely reconstruct depth variation of the arranged cards, as shown in Fig. 17, five depth levels are obtained. It coincides to the ground truth.
  • (2) At the edges of cards, the error is higher than at the other places. This is because that we use a global optimization method based on the gradient flow, and it takes longer time to obtain a comparatively sharp surface. In addition, the estimated error is proportional to the true depth.
  • (3) From our calculation, the mean reconstruction error of our proposed algorithm is 0.052 mm, and the mean-square-error is 0.0536. Because we use an optimization method to obtain a global 3D surface, no sharp depth-peaks appear between each depth level; therefore it is possible to obtain a smoother 3D surface, compared with the local 3D reconstruction methods.

5. Conclusion

In this paper, a novel 3D reconstruction method based on optical diffusion is proposed. Our primary contribution to the wider field is the introduction of the imaging model and the heat diffusion equation. Further, a blurred image with optical diffusion is constructed using relative blurring and this equation. A second contribution is the proposal of a global 3D surface reconstruction algorithm based on the relationship between the reconstructed surface and the degree of blurring of the associated diffused image. Finally, a third contribution made by this study is that we conduct simulations with synthetic images and experiments using various arrangements of five cards in order to validate our new method. The results show that the proposed algorithm is effective as a means of reconstructing a 3D surface using monocular vision without the need for camera parameter adjustment. In addition, as the proposed method requires only one fixed camera and an optical diffuser, the process is simple, and the error analysis results show that it can reconstruct depth with high precision. Therefore, this technique can be used t in monocular vision and hand-eye systems to obtain 3D information. The spatial resolution of this algorithm should be investigated in future work, because it is related to both the diffusion angle of the diffuser and the distance between the diffuser and the scene; these two characteristics have a mutual coupled relationship.

Acknowledgments

The authors thank the funding support from the National Natural Science Foundation of China (NSFC) (No. 61305025, 61473282) and the Fundamental Research Funds for the Central Universities (N13050411). The authors also thank for the support by the State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences (CAS).

References and links

1. C. Y. Yin, “Determining residual nonlinearity of a high-precision heterodyne interferometer,” Opt. Eng. 38(8), 1361–1365 (1999). [CrossRef]  

2. L. D. Wu, Computer Vision (Fudan University Press, 1993).

3. B. Girod and S. Scherock, “Depth from defocus of structured light,” in “Proceedings of Optics, Illumination, and Image Sensing for Machine Vision,” (1989), pp. 209–215.

4. V. M. J. Bove Jr., “Entropy-based depth from focus,” J. Opt. Soc. Am. A 10(4), 561–566 (1993). [CrossRef]  

5. S. K. Nayar, “Shape from focus system,” in “Proceedings of IEEE Computer Vision and Pattern Recognition,” (1992), pp. 302–308.

6. A. P. Pentland, S. Scheroch, T. Darrell, and B. Girod, “Simple range cameras based on focus error,” J. Opt. Soc. Am. A 11(11), 2925–2934 (1994). [CrossRef]  

7. S. K. Navar, M. Watanabe, and M. Noguchi, “Real-time focus range sensor,” IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1186–1198 (1996).

8. A. P. Pentland, “A new sense for depth of field,” IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 523–531 (1987). [CrossRef]   [PubMed]  

9. M. Gokstorp, “Computing depth from out-of-focus blur using a local frequency representation,” in “Proceedings of International Conference on Pattern Recognition, ” (1994), pp. 153–158. [CrossRef]  

10. M. Subbarao and G. Surya, “Depth from defocus: A spatial domain approach,” Int. J. Comput. Vis. 13(3), 271–294 (1994). [CrossRef]  

11. V. P. Namboodiri and S. Chaudhuri, “On defocus, diffusion and depth estimation,” Pattern Recognit. Lett. 28(3), 311–319 (2007). [CrossRef]  

12. P. Favaro, S. Soatto, M. Burger, and S. J. Osher, “Shape from defocus via diffusion,” IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 518–531 (2008). [CrossRef]   [PubMed]  

13. P. Favaro, A. Mennucci, and S. Soatto, “Observing shape from defocused images,” Int. J. Comput. Vis. 52(1), 25–43 (2003). [CrossRef]  

14. P. Favaro and A. Mennucci, “Learning shape from defocus,” in “Proceedings of Europeon Conference on Computer Vision,” (2002), pp.735–745.

15. K. Mori, “Apparatus for uniform illumination employing light diffuser,” July 17 1984, US Patent 4,460,940.

16. G. M. Mari-Roca, L. Vaughn, J. S. King, K. W. Jelley, A. G. Chen, and G. T. Valliath, “Light diffuser for a liquid crystal display,” February 14, 1995, US Patent 5,390,085.

17. A. Ashok and M. A. Neifeld, “Pseudorandom phase masks for superresolution imaging from subpixel shifting,” Appl. Opt. 46(12), 2256–2268 (2007). [CrossRef]   [PubMed]  

18. E. E. García-Guerrero, E. R. Méndez, H. M. Escamilla, T. A. Leskova, and A. A. Maradudin, “Design and fabrication of random phase diffusers for extending the depth of focus,” Opt. Express 15(3), 910–923 (2007). [CrossRef]   [PubMed]  

19. C. Y. Zhou, O. Cossairt, and S. Nayar, “Depth from diffusion,” in “Proceedings of IEEE Conference of Computer Vision and Pattern Recognition,” (2010), pp.1110–1117.

20. Y. J. Wei, Z. L. Dong, and C. D. Wu, “Depth measurement using single camera with fixed camera parameters,” IET Computer Vision 6(1), 29–39 (2012). [CrossRef]  

21. Y. Wei, C. Wu, and Z. Dong, “Nanoscale depth reconstruction from defocus: within an optical diffraction model,” Opt. Express 22(21), 25481–25493 (2014). [CrossRef]   [PubMed]  

22. Y. J. Wei, C. D. Wu, and Z. L. Dong, “Global shape reconstruction of the bended AFM cantilever,” IEEE Trans. NanoTechnol. 11(4), 713–719 (2012). [CrossRef]  

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (18)

Fig. 1
Fig. 1 Diffusion theory and a diffused image.
Fig. 2
Fig. 2 Geometry of a diffusion process and its energy distribution.
Fig. 3
Fig. 3 Different diffusion forms (uniform, batwing and Gaussian).
Fig. 4
Fig. 4 Geometry of diffusion in a pinhole camera.
Fig. 5
Fig. 5 Flow graph of our algorithm.
Fig. 6
Fig. 6 The synthesized images of a consine plane.
Fig. 7
Fig. 7 The depth maps of gray.
Fig. 8
Fig. 8 The reconstructed 3D surface and the true 3D surface.
Fig. 9
Fig. 9 The error map between the estimated surface and the true surface.
Fig. 10
Fig. 10 The synthesized images of a rectangle plane.
Fig. 11
Fig. 11 The depth maps of gray.
Fig. 12
Fig. 12 The reconstructed 3D surface and the true surface.
Fig. 13
Fig. 13 The error map between the estimated surface and the true surface.
Fig. 14
Fig. 14 Focused image and diffused image of the arranged cards.
Fig. 15
Fig. 15 Reconstructed surface of our method.
Fig. 16
Fig. 16 3D reconstructed surface of our method.
Fig. 17
Fig. 17 True 3D surface of the cards.
Fig. 18
Fig. 18 Error surface of our method.

Equations (28)

Equations on this page are rendered with MathJax. Learn more.

J ( x , y , t ) = a u ( x , y , t )
u ( x , y , t ) t = J ( x , y , t )
u ( x , y , t ) t = a 2 u ( x , y , t )
{ u ( x , y , t ) t = a 2 u ( x , y , t ) u ( x , y , 0 ) = u 0 ( x , y )
{ u ( x , y , t ) t = ( a ( x , y ) u ( x , y , t ) ) u ( x , y , 0 ) = u 0 ( x , y )
b = v U A B 2 = v tan θ cos 2 α Z Z + U
I ( x 1 , y 1 ) = h ( x 1 , y 1 , x , y , b ) r ( x , y ) d x d y
I = h ( , b ) r
h ( x , y , x 1 , y 1 ) = 1 2 π σ 2 exp ( ( x x 1 ) 2 + ( y y 1 ) 2 2 σ 2 )
h ( x 1 , y 1 , x , y , b ) d x 1 d y 1 = 1
{ u ˙ ( x , y , t ) = a Δ u ( x , y , t ) a [ 0 , ) t ( 0 , ) u ( x , y , 0 ) = r ( x , y )
σ 2 = 2 t a
h ( x , y , x 1 , y 1 , t ) = 1 4 π t a exp ( ( x x 1 ) 2 + ( y y 1 ) 2 4 t a )
I ( x 1 , y 1 , t ) = ( 1 4 π t a exp ( ( x x 1 ) 2 + ( y y 1 ) 2 4 t a ) ) r ( x , y )
{ u ˙ ( x , y , t ) = ( a ( x , y ) u ( x , y , t ) ) t ( 0 , ) u ( x , y , 0 ) = r ( x , y )
σ 2 ( x , y ) = 2 t a ( x , y )
a ( x , y ) = σ 2 ( x , y ) 2 t = γ 2 v 2 tan 2 ( θ ) 2 t cos 4 α ( Z Z + U ) 2
{ u ˙ ( x , y , t ) = ( a ( x , y ) u ( x , y , t ) ) t ( 0 , ) u ( x , y , 0 ) = E 1 ( x , y ) u ( x , y , t 2 ) = E 2 ( x , y )
γ 2 v 2 tan 2 ( θ ) 2 t cos 4 α ( Z Z + U ) 2 = 0
{ u ˙ ( x , y , t ) = ( a ( x , y ) u ( x , y , t ) ) t ( 0 , ) u ( x , y , 0 ) = E 1 ( x , y ) u ( x , y , Δ t ) = E 2 ( x , y ) 0 = a ( x , y ) u ( x , y , t ) n ( x , y )
Z = σ U cos 2 α ( γ v tan θ σ cos 2 α )
Z ˜ = arg min Z ( x , y ) ( u ( x , y , Δ t ) E 2 ( x , y ) ) 2 d x d y
Z ˜ = arg min Z ( x , y ) ( u ( x , y , Δ t ) E 2 ( x , y ) ) 2 d x d y + η Z ( x , y ) 2 + η k Z ( x , y ) 2
F ( Z ) = ( u ( x , y , Δ t ) E 2 ( x , y ) ) 2 d x d y + η Z 2 + η k Z 2
Z ˜ = arg min Z F ( Z )
Z t = F ' ( Z )
ζ = Z e / Z 1
φ = E [ ( Z e / Z 1 ) 2 ]
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.