Open Access
Add to BibSonomy
Abstract
A one-shot three-dimensional measurement method of a material surface with a color mapping imaging system of the light direction extracted BRDF (bidirectional reflectance distribution function) is proposed here. The imaging system can measure light directions reflected from a material surface using a stripe pattern multicolor filter having a translational symmetry in one direction. Assuming that surface inclination angles are sufficiently small and that the surface has a flat peripheral boundary, a reconstruction method of a three-dimensional surface from light directions is derived theoretically on the basis of the geometrical optics. The method is experimentally validated using a round ridge fabricated on an aluminum flat plate.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
In many manufacturing processes, a three-dimensional material surface is inspected for quality control. For example, in metal bonding processes such as welding, soldering, brazing, and 3D printing, a surface shape of a molten metal and its contact angle on a solid plate are indicators of adhesion and need to be measured for quality check [1,2]. Surface shape and contact angle of a liquid droplet on solid dry substrates in industrial processes such as coating and inkjet printing need also be inspected for quality control [3]. There are several methods to measure them using images captured by cameras [4–9]. When a surface inclination angle defined as an angle of a surface normal vector to a reference vector (e.g., an optical axis) is however too small, it is difficult to measure the surface shape and contact angle accurately. Moreover, it is often difficult to obtain an in-plane distribution of the surface normal vector by a conventional imaging.
The bidirectional reflectance distribution function (BRDF) describes several features of a material surface [10,11]. Actually, the surface normal vector affects the surface BRDF distribution. The surface BRDF are thus available for measuring the surface inclination angle. A one-shot imaging system that can obtain light directions extracted from the surface BRDF using a concentric multicolor filter is one of the candidates [12]. The one-shot imaging system can classify light directions by color and capture an image of surface inclination angles with respect to an optical axis of the imaging system. A three-dimensional surface, however, is not able to be reconstructed from the image captured by the imaging system having the concentric multicolor filter due to lack of information of an azimuth angle of a light direction. In general, two components that represent the light direction (i.e., zenith angle and azimuth angle) are needed to obtain the surface normal vector. The continuous distribution of the surface normal vector, however, is obtainable if a certain boundary condition of the surface is given [13,14].
A one-shot three-dimensional measurement method of a material surface is therefore proposed here using a color mapping imaging system of light direction extracted BRDF. The imaging system is here called one-shot BRDF imaging system or briefly one-shot BRDF. The one-shot BRDF imaging system can measure light directions reflected from a material surface using a stripe pattern multicolor filter. Assuming that surface inclination angles are sufficiently small and that the surface has a flat peripheral boundary, a reconstruction method of three-dimensional surface from the measured light directions is derived theoretically on the basis of the geometrical optics. The remainder of this paper is organized as follows. First, a basic structure of the one-shot BRDF imaging system is described. Second, a relationship between a color mapped light direction and a surface normal vector is described. Third, a reconstruction method using the one-shot BRDF is proposed under the assumption of the small surface inclination angle and the flat peripheral boundary. Fourth, a three-dimensional surface of an aluminum round ridge with a height of about 45 µm and a FWHM (full width at half maximum) of about 1200 µm is shown to be reconstructed using the proposed method. The reconstructed three-dimensional surface agrees with that measured by means of a white light interferometer system (ZYGO) [15], which validates the reconstruction method using the one-shot BRDF. Lastly, discussions and conclusions are described.
2. One-shot light direction imaging system with a stripe pattern multicolor filter
Figure 1 shows a schematic cross-sectional view of the one-shot BRDF. The one-shot BRDF consists mainly of an illumination optical system and an imaging optical system.
Fig. 1. Schematic cross-sectional view of one-shot BRDF imaging system (one-shot BRDF). The one-shot BRDF consists mainly of an illumination optical system and an imaging optical system. The imaging optical system has an imaging lens and a stripe pattern multicolor filter that is placed at the focal plane of the imaging lens. The optical axis of the imaging lens is set to z-axis in a global Cartesian coordinate system. The multicolor filter is set to parallel to the x – y plane and has a translational symmetry in y-direction. The coordinate origin O is in the multicolor filter.
The illumination optical system has an LED, a pinhole, and a collimator lens that can convert the diverging light rays emitted from the LED to collimated light rays. The collimated light rays are reflected by a beam splitter and travel toward a material surface. The imaging optical system has an imaging lens and a stripe pattern multicolor filter that consists of multiple regions with different color filters. The multicolor filter is placed at the focal plane of the imaging lens at a focal length f from a principal plane of the imaging lens. The optical axis of the imaging lens is set to z-axis in a global Cartesian coordinate system. The multicolor filter is set to parallel to the x – y plane and has a translational symmetry in y-direction. The coordinate origin O is in the multicolor filter. Reflected light rays from the material surface pass through the beam splitter and will be refracted by the imaging lens. The refracted light rays pass through the multicolor filter and will be imaged on an image sensor. In this way, a light ray reflected by an object point in the material surface is imaged on an image point in the image sensor with its color selected depending on its direction.
3. Relationship between color mapped light direction and surface normal vector
As shown in Fig. 1, a two-dimensional position vector r represents a point where a light ray with an angle θ to the optical axis passes in the multicolor filter at the focal plane. The r can be written with an azimuth angle ϕ to the x-direction and the focal length f on the basis of the geometrical optics under the paraxial approximation as
A two-dimensional angle vector ${\boldsymbol{\mathrm{\theta}}}$ having two components of θx and θy is here defined as
The position vector r can thus be written with the angle vector ${\boldsymbol{\mathrm{\theta}}}$ as
An incident light ray toward an object point on a material surface with its direction parallel to the optical axis will be reflected in a plane of incidence as shown in Fig. 2.
Fig. 2. Schematic view of plane of incidence. A normal vector of n on the local infinitesimal surface has an angle α to the optical axis. When the surface is sufficiently smooth, reflection can be considered to be a specular reflection with an angle θ of 2α to the optical axis.
A normal vector n at the object point in the local infinitesimal surface has an angle α to the optical axis. When the surface is sufficiently smooth, the reflection can be practically considered to be a specular reflection with an angle θ of 2α to the optical axis in the plane of the incidence. A unit vector e of the light direction can be written on the basis of the law of regular reflection as
where ez denotes a unit vector of z-direction.A height of the three-dimensional surface along z-axis from a reference plane is set to Ψ that is a function of x and y. The height Ψ can be thus written as
where z0 is a constant value and denotes a z-position of the reference plane. The surface normal vector n can be written with partial derivatives of the height Ψ asThe unit vector e of the light direction can thus be written using Eqs. (4) and (6) as
At the focal plane (i.e., z = 0), the two-dimensional position vector r where the light ray passes in the multicolor filter can be derived using Eq. (9) as
Note that light rays parallel to the z-axis always pass through the coordinate origin O in the multicolor filter. Inserting Eq. (3) into Eq. (10), the two-dimensional angle vector ${\boldsymbol{\mathrm{\theta}}}$ can be derived as
This equation represents the relationship between the color mapped light direction and the surface normal vector.
4. Reconstruction method of the three-dimensional surface
Assuming that a surface inclination angle at an object point is sufficiently small, the p and q can be considered small. In this case, two-dimensional angle vector ${\boldsymbol{\mathrm{\theta}}}$ represented by Eq. (11) can be approximated to the first order of the p and q as
The p can therefore be derived from Eq. (12) as
where the angle θx is observable and can be obtained by means of the multicolor filter color mapping [16,17].Equation (13) can be uniquely solved when a peripheral boundary condition of the surface is given. The height Ψ can be integrated along the x-direction as
Equation (15) means that the three-dimensional surface can be reconstructed from the one-directional color mapping of the light direction.
The y-component angle θy can also be derived once the height Ψ is obtained. The θy can be derived using Eqs. (12) and (15) as
Equation (16) means that the two-dimensional angle vector ${\boldsymbol{\mathrm{\theta}}}$ can be reconstructed from the one-component angle under the assumption of the flat peripheral boundary.
5. Experimental validation
5.1 Unprocessed image of the round ridge
A round ridge is fabricated on an aluminum base plate by a machining process. Figure 3 shows (a) an unprocessed top view color image of an aluminum round ridge captured by the one-shot BRDF and (b) an in-plane continuous distribution of x-component angle θx with color contour of the angle. The angle is in degrees. The round ridge has a height of about 45 µm with a FWHM of about 1200 µm. Surface roughness of the round ridge is considered to be sufficiently small. A surface inclination angle of the round ridge is set to about 3 degrees and can be considered sufficiently small. The peripheral regions close to the both edges of the base plate along y-axis is flat. The reconstruction method using Eq. (15) is thus applicable to the round ridge. The in-plane continuous distribution of the angle θx can be obtained from the unprocessed image using hue color interpolated mapping [16].
Fig. 3. (a) An unprocessed top view color image of an aluminum round ridge captured by the one-shot BRDF and (b) an in-plane continuous distributions of angle θx with color contour of the angle. The angle is in degrees. The multicolor filter of the one-shot BRDF is set to the one that consists of 22 graded hue color regions where the green region is center. The round ridge fabricated by a machining process has a height of about 45 µm with a FWHM of about 1200 µm. Surface roughness and surface inclination angles of the round ridge are considered to be sufficiently small.
In this work, the stripe pattern multicolor filter of the one-shot BRDF is set to the one that consists of 22 graded hue color regions where the green region is center. The focal length f of the imaging lens (Nikon, AF-S NIKKOR F/ 1.4) is set to 105 mm. The maximum capturable angle θ of a light ray with respect to the optical axis is about 7 degrees. A focal length of the illumination collimator lens is set to 80 mm. A diameter of the pinhole is set to 400 µm. A divergence angle of the collimated illumination light is thus estimated to be 0.28 degrees.
5.2 Reconstructed three-dimensional surface of the round ridge
A three-dimensional surface of the round ridge can be reconstructed using Eq. (15) where the base plate surface is considered as a reference plane (i.e., Ψ = 0). The observed x-component angle θx shown in Fig. 3 has a small bias error, which is corrected in order to satisfy the flat boundary condition. Figure 4 shows (a) a perspective view of the round ridge with color contour of the height, (b) a top view of the round ridge with a - a’ line, and (c) a cross-sectional view along the a - a’ line. A cross-section reconstructed by the one-shot BRDF is denoted by a solid line whereas that measured by the white light interferometer system [15] is denoted by a circle mark.
Fig. 4. Reconstructed three-dimensional surface of round ridge using the one-shot BRDF. The figure shows (a) a perspective view of the reconstructed round ridge with color contour of the height, (b) a top view of the reconstructed round ridge with a - a’ line, and (c) a cross-sectional view along the a - a’ line. A cross-section reconstructed with the one-shot BRDF is denoted by solid line whereas that measured by the white light interferometer system is denoted by circle mark.
The height and FWHM of the round ridge reconstructed by the one-shot BRDF are respectively 43 µm and 1220 µm whereas those measured by the white light interferometer system are 45 µm and 1222 µm respectively. The reconstructed values by the one-shot BRDF agree well with those measured by the white light interferometer system, which validates the reconstruction.
6. Discussions
Once a three-dimensional surface is reconstructed using Eq. (15), an in-plane surface normal vector distribution can be calculated using Eq. (6). A contact angle should thus also be calculable with the reconstructed three-dimensional surface.
The y-component angle θy can be reconstructed using Eq. (16) from the observed x-component angle θx. Figure 5 shows a reconstructed in-plane y-component angle θy. The angle is color-contoured and labeled in degrees. Although there is a small ripple-like error along y-direction, the y-component angle θy is shown to be reconstructed from the other component angle θx under the assumption of the flat peripheral boundary. The two-dimensional angle vector ${\boldsymbol{\mathrm{\theta}}}$ that represents light direction is thus obtainable.
Fig. 5. Reconstructed in-plane y-component angle θy. The angle is color-contoured and labeled in degrees.
Since the surface roughness affects the light direction, the height reconstruction may be affected by the surface roughness. The specular intensity used for the reconstruction, however, is often higher than the diffused intensity, which might suppress the deterioration.
The maximum difference between the round ridge height measured by the one-shot BRDF and that measured by the white light interferometer system is about 2 µm, which corresponds to about 5 percent error. The difference should be mainly caused by the illumination light divergence angle that corresponds to at most 10 percent of the surface inclination angle of the round ridge. The illumination light divergence angle should thus be small to improve reconstruction accuracy.
The color mapping of the light direction needs calibration if a material has a specific color. A material color effect can be removed once the calibration of color mapping is performed.
7. Conclusions
One-shot three-dimensional measurement method of a material surface with a color mapping imaging system of light direction extracted BRDF is proposed here. The imaging system can measure light directions reflected from the material surface using a stripe pattern multicolor filter having a translational symmetry in one direction. Assuming that surface inclination angles are sufficiently small and that the surface has a flat peripheral boundary, a three-dimensional surface reconstruction method from one-component of the light direction is derived theoretically on the basis of the geometrical optics. The basic equation of the reconstruction method is represented by Eq. (15). A surface shape of an aluminum round ridge with a height of about 45 µm can be measured using the one-shot BRDF and agrees with that measured by the white light interferometer system. Two-dimensional angle vector that represents a light direction is also shown to be reconstructed from the measured one-component angle θx under the assumption of the flat peripheral boundary. The proposed method derived theoretically is thus experimentally validated.
Disclosures
The author declares no conflicts of interest.
References
1. H. Ohno, Y. Shiomi, S. Tsuno, and M. Sasaki, “Design of a transmissive optical system of a laser metal deposition three-dimensional printer with metal powder,” Appl. Opt. 58(15), 4127–4138 (2019). [CrossRef]
2. V. Samavatian, M. Fotuhi-Firuzabad, M. Samavatian, P. Dehghanian, and F. Blaabjerg, “Correlation-driven machine learning for accelerated reliability assessment of solder joints in electronics,” Sci. Rep. 10(1), 14821 (2020). [CrossRef]
3. M. Q. Santiago, A. A. Castrejón-Pita, and J. R. Castrejon-Pita, “The Efect of Surface Roughness on the Contact Line and Splashing Dynamics of Impacting Droplets,” Sci. Rep. 9(1), 15030 (2019). [CrossRef]
4. G. Dutra, J. Canning, W. Padden, C. Martelli, and S. Dligatch, “Large area optical mapping of surface contact angle,” Opt. Express 25(18), 21127–21144 (2017). [CrossRef]
5. D. Luo, L. Qian, L. Dong, P. Shao, Z. Yue, J. Wang, B. Shi, S. Wu, and Y. Qin, “Simultaneous measurement of liquid surface tension and contact angle by light reflection,” Opt. Express 27(12), 16703–16712 (2019). [CrossRef]
6. Y. Bando, B. Chen, and T. Nishita, “Extracting depth and matte using a color-filtered aperture,” ACM Trans. Graph. 27(5), 1–9 (2008). [CrossRef]
7. J. S. Kim and T. Kanade, “Multiaperture telecentric lens for 3D reconstruction,” Opt. Lett. 36(7), 1050–1052 (2011). [CrossRef]
8. M. Bass, C. DeCusatis, J. Enoch, V. Lakshminarayanan, G. Li, C. Macdonald, V. Mahajan, and E. Van Stryland, Handbook of Optics (McGraw-Hill, 2010).
9. T. Kim, S. H. Kim, D. Do, H. Yoo, and D. Gweon, “Chromatic confocal microscopy with a novel wavelength detection method using transmittance,” Opt. Express 21(5), 6286–6294 (2013). [CrossRef]
10. B. K. P. Horn and R. W. Sjoberg, “Calculating the reflectance map,” Appl. Opt. 18(11), 1770–1779 (1979). [CrossRef]
11. F. E. Nicodemus, J. C. Richmond, and J. J. Hsia, “Geometrical considerations and nomenclature for reflectance,” Final Report National Bureau of Standards, October (1977).
12. H. Ohno, “One-shot color mapping imaging system of light direction extracted from a surface BRDF,” OSA Continuum 3(12), 3343–3350 (2020). [CrossRef]
13. H. Ohno, “Localized gradient-index field reconstruction using background-oriented schlieren,” Appl. Opt. 58(28), 7795–7804 (2019). [CrossRef]
14. H. Ohno and K. Toya, “Scalar potential reconstruction method of axisymmetric 3D refractive index fields with background-oriented schlieren,” Opt. Express 27(5), 5990–6002 (2019). [CrossRef]
15. ZYGO Corporation, “NewView 9000,” https://www.zygo.com/?/met/profilers/newview9000/
16. W. L. Howes, “Rainbow schlieren and its applications,” Appl. Opt. 23(14), 2449–2460 (1984). [CrossRef]
17. H. Ohno and H. Kano, “Depth reconstruction with coaxial multi-wavelength aperture telecentric optical system,” Opt. Express 26(20), 25880–25891 (2018). [CrossRef]
