Open Access
Add to BibSonomy
Abstract
Recently, intensive research on microlens array (MLA) was undertaken, which involved intertwining with the light-field imaging in order to obtain four-dimensional information. Although several fabrication approaches for MLA attempted to achieve high density and precision geometry, further progress is necessary. In this study, we present a cost-effective fabrication strategy for geometrically tunable polymer MLA with extremely low sag height (~3 μm) through improved MEMS wet etching process. Additionally, we assemble a hand-crafted light-field camera by integrating the elaborately customized MLA with a commercial digital camera. Finally, we demonstrate representative light-field imaging features including refocusing and all-in focusing image from a single exposure.
© 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Light-field imaging is also known as plenoptic imaging. It offers powerful capability involving the collection of the four-dimensional light information of a target object via a single exposure through sophisticated digital processing techniques that are tightly merged with exceptional optical designs such as camera arrays, moving cameras, and microlens arrays (MLAs) [1]. In contrast to existing imaging devices that only record two-dimensional position data, unconventional optical designs simultaneously obtain spatial and direction domain information. Digitally processed light-field data acquired from the unusual optics allows the intriguing imaging features to outperform the outcomes of the conventional imaging techniques such as multi-view imaging, digital refocusing, expansion of depth of focus, depth estimation, and holographic reconstruction. Recently, the aforementioned advantages significantly motivated the application of light-field imaging to various application fields such as human face/iris recognition [2], computer-generated hologram [3], morphology analyses [4], 3D telemedicine system [5], reconstruction of 3-dimensional temperature field [6], and multispectral light-field imaging [7]. Specifically, among unconventional optical designs, the MLAs are the most promising apparatus since the overall imaging devices are the most compact and correspond to a simple configuration that can be built in a commercial camera while the camera arrays or moving cameras are inevitably accompanied with bulky, large, and complicated configurations of light-field imaging devices [1,8]. In the light-field system, the MLA is located in the focal plane of the main lens (i.e., the lens of the commercial camera) and is at a distance from the image sensor that corresponds to the focal distance of the MLA (Fig. 1(a)). The imaging performance of MLA is a key point to obtain three-dimensional information since each microlens unit captures the object from different view angles at a given position [9,10]. Thus, the overall performance of the light-field camera is determined by geometrical parameters (i.e., radius of curvature (RoC), fill factor, diameter, and lattice of the array) of the MLA as follows.
Fig. 1 Description of a light field camera based on microlens array (MLA). (a) Schematic of a light-field camera based on MLA. (b) Difference in the focusing ability of MLA based on the radius of curvature (RoC). (c) Plot of an active fill factor with different F-numbers of the main lens. The F-number of the MLA is 4. This indicates that F-number matching between the main lens and MLA is imperative in terms of minimizing discarded pixels.
First, the RoC of microlens controls its focal length and coverage area in the active area of image sensor (Fig. 1(b)). Typically, MLAs exhibit short focal lengths (i.e., small RoC) and high spherical aberration due to their high curvatures. It is difficult to integrate these types of MLAs that form the focal plane at a short distance with an image sensor in a camera. Additionally, the coverage area of MLA that is adjusted by the focal length influences the angular information from the object. Hence, MLAs with controllable focal length are necessary. Second, the fill factor and diameter of MLA affect the spatial resolution of the light-field camera. With respect to the diameter, increases in the diameter of the microlens lead to more angular domain information although the resolution of the image decreases. Although the decrease in image resolution due to the diameter of MLA is inevitable, the closely packed MLA (i.e., maximum fill factor) is evidently effective for high image resolution since a high fill factor can minimize the number of wasted pixels in the image sensor. Additionally, even with a full filled MLA, when the F-number of the main lens exceeds that of MLA, the discarded pixels exist and result in low image resolution (Fig. 1(c)). In the opposite case, the coverage areas of MLA overlap lead to degradation of angular information acquisition. Well matched F-number between the main lens and MLA reveal the formation of maximized coverage areas of MLA on the image sensor without overlapping. Therefore, the light-field cameras require geometrically controllable MLAs to enhance the quality of processed images. To date, several extant studies reported various approaches to fabricate the MLA such as direct fabrication methods (thermal reflow [11], microplastic embossing [12], and microdroplet jetting [13]) and indirect methods that require a mold (MEMS-based and ultraprecision machining methods [14,15]). However, direct fabrication methods struggle to control the RoC of MLA and implement a highly-packed MLA. In indirect methods, it is difficult for the standard MEMS-based method to diversely vary the RoC of mold, and it is expensive to manufacture the metallic mold produced via the ultraprecision machining method.
In the study, we present a fabrication scheme that affords a pathway for obtaining a highly dense transparent polymeric (i.e., polydimethylsiloxane; PDMS) MLAs with excellent geometrical tunability. Consecutive semiconductor fabrication processes on Poly-Si deposited quartz wafer, such as photolithography, dry etching, and isotropic wet etching process, can generate a concave spherical shape on quartz mold in a large-area and with low-cost. Additionally, the curvature and sag height of quartz mold are well manipulated by modifying the time to remove hard mask (i.e., Poly-Si) on quartz mold, and thus it provides a wide tuning range of RoC in MLA with a low sag height. In the fabricated PDMS MLA, the sag height of convex shape is extremely low (approximately 3 μm) although the lenses exhibit large curvature (RoC 139 μm). In addition to the fabrication of MLA, we produce a hand-crafted light-field camera to validate the applicability of our engineered MLA for light-field imaging, by integrating the fabricated MLA with a commercial digital camera. We finally demonstrate a refocused image that corresponds to the adjustment of the focusing point in a one shot image through a post image process.
2. Geometrically controllable MLA fabrication method
Figure 2a shows the overall procedure of MLA fabrication. The MLA fabrication steps are as follows: (i) Poly-Si is used as a hard mask in hydrofluoric acid (HF) wet etching process and is deposited on both sides of the quartz substrate. The thickness of Poly-Si is approximately 700 nm. (ii) Photoresist (PR) patterning on a side of the Poly-Si is conducted via photolithography. (iii) The patterned sample is dry-etched by using SF6 gas via an inductive-coupled plasma reactive ion etcher (ICP-RIE). The ICP-RIE conditions are as follows: SF6 flow / chamber pressure / RF power / ICP power / etching time = 50 sccm / 4 mTorr / 50 W / 100 W / 2 min. (iv) The patterned sample is immersed in an HF bath, and the HF solution hemispherically etches the quartz substrate through the hole pattern (Fig. 2b). Because of tiny via hole (~2 μm), the etching amount of Quartz mold is isotropic in hemispherically from the center of each via hole. Therefore, the isotropic etching of HF creates almost the same radius, R, and sag height, H, which are controlled by the etching time. Subsequently, Poly-Si is removed by using potassium hydroxide (KOH) at a temperature of 150 °C for 30 min. (v) PDMS replica molding is performed by using the etched quartz substrate to form a concave lens array as a master mold. An anti-adhesive is sprayed on the mold prior to pouring PDMS on the mold. The fluorocarbon mold release agent is used as the anti-adhesive spray (DAIFREE GA-7550, DAIKIN, Japan). The spraying time is ~3 s, and the spraying distance between the mold is ~20 cm. In order to prevent thermal deformation of the PDMS during curing time, PDMS poured on quartz mold is cured at room temperature for approximately 4 d. Additionally, the slanted top surface of PDMS MLA hinders clear formation of the object image. Thus, curing of the PDMS MLA is performed on a leveled optical table. (vi) After complete solidification of the polymer, the MLA is gently detached with a tweezer. Although the aforementioned procedure is sufficient to fabricate a quartz mold with the hemispherical shape, it is not suitable to form a uniform MLA with an R value that exceeds the H value
Fig. 2 (a) Schematic illustration of the overall MLA fabrication procedure. (b) Schematic of formation of a hemispherical shape of quartz mold via the HF solution depicted in (iv) of Fig. 2a.
Figure 3a shows the situation when HF etching time is only adjusted to fabricate the MLA with a short period and large RoC (i.e., R>H). In order to lower H, the walls between each curvature in the mold should be collapsed by using HF over-etching. However, during the over-etching process, the contact area of Poly-Si with the mold is significantly reduced, and the Poly-Si layer is easily stripped off from the mold. The floating Poly-Si layer hinders formation of the uniform curvature array. Figure 3b shows a scanning electron microscope (SEM) image of non-uniformly formed quartz mold. The photography mask with the hole width of 2 μm and period of 26.4 μm was used to fabricate regular tiny hole arrays on Poly-Si layer. The diameters and centers of curvature are different for adjacent holes due to non-uniform etching for the period of time removing the walls. The aforementioned types of geometrical defects in MLAs degrade the imaging characteristics of a light-field camera. Figure 3c shows the wet etching procedure to overcome the limitation. First, a quartz mold with a hemispherical curvature is prepared with the method shown in Fig. 2(a). Subsequently, the removal of Poly-Si layer is performed in KOH at 150 °C for 30 min. Following the removal the hard mask, the quartz mold is re-immersed in a HF bath. In this step, the RoC increases and the sag height decreases and is proportional to the etching time. Based on the fabrication procedure, the quartz mold for PDMS MLA with high RoC and low sag height is uniformly implemented at a large-scale.
Fig. 3 (a) Schematic illustrations of non-uniform quartz etching. (b) Scanning electron microscope (SEM) image of the non-uniformly formed quartz mold. Red points denote the center of circles. Solid and dashed lines denote half and edge of circles. (c) Schematic illustration of the wet etching procedure for the uniform mold with low sag height and high RoC.
Figures 4a and 4b show the various quartz molds with different geometrical shapes. The lattice pattern is formed differently in the photolithography step, and the RoC is controlled in the HF and KOH etching steps by modifying the etching time. As previously explained, light-field cameras require complex optical designs. Specifically, the MLA should satisfy F-number matching with the main lens. Figure 4c shows the RoC MLAs manufactured by our MLA fabrication method as a function of HF etching time with two Poly-Si removal time (i.e., 40 and 50 min). The difference in the RoCs between two different Poly-Si removal times indicates that removing the Poly-Si affects the RoC size. The fabricated PDMS MLA exhibits haziness due to the microscale structure at the surface (Fig. 4(d); right) while the ‘GIST’ text is clearly observed beneath the bare PDMS (Fig. 4(d); left). Figure 4e shows a uniformly formed PDMS MLA in a large-area. The PDMS MLA exhibits a RoC of 139 μm. The inset image also shows the cross-sectional view of PDMS MLA with an extremely low sag height of approximately 3 μm. The used mask patterns are different in Fig. 4(a,b), and 4(e). The size and pitch of holes are 2 μm and 26.4 μm, respectively, for Fig. 4(a) and 4(b). In Fig. 4(c), the same size of hole was used as 2 μm, but a pitch was changed as 60 μm. All RoCs were measured via laser confocal microscopy (Olympus, OLS 4000).
Fig. 4 (a,b) SEM images of quartz molds (a) with two different lattice patterns and (b) with different etching times. (c) Graph of RoC versus HF etching time with respect to two different Poly-Si removal times. (d) Optical image of bare PDMS (left) and fabricated PDMS MLA (right). (e) SEM image of PDMS MLA with a high RoC of 139 μm and low sag height of 3 μm.
3. Light-field camera demonstration
The RoC of the microlens was designed considering the space of the image sensor and the pixel size. We use ZEMAX ray tracing simulation to design focal spots of microlens smaller than 4.26 μm pixel pitch of Samsung NX1000 image sensor. When RoC is above 139 μm, the focal spot becomes smaller than 4.26 μm. However, if RoC is larger than 139 μm, the focal length becomes also longer, thus the space of the image sensor for integration is reduced. Thus, we fabricated a MLA with a focal length and RoC of 240 and 139 um, respectively, for hand-crafted light field camera. In order to verify the applicability of our engineered MLA, a hand-crafted light-field camera is assembled by integrating the MLA with a commercial digital camera (Samsung NX1000). The image sensor of NX1000 is located on microscale thickness spacer to consider focal distance and alignment of PDMS MLA. Figure 5(a) shows the schematic illustration of PDMS MLA integrated with an image sensor in a Samsung NX1000 camera. With respect to the F-number matching between the main lens and engineered MLA, the cover glass of the image sensor is removed. Following the removal, the engineered MLA is located on a 240 μm spacer corresponding to the focal length of MLA. The inset in Fig. 5(a) shows the photograph of the integrated image sensor that is reassembled on an NX1000 camera. Figure 5(b) shows the light-field camera property wherein the multi-view point property is observed via the MLA. In order to demonstrate the hand-craft light field camera, the hand-crafted light-field camera is settled on an optical table, and targets are located at different distances (i.e., 30, 90, 120, 150, 180, 210, and 240 cm). Figure 5(c) shows the obtained raw data focused at 30 cm that exhibits a micro array pattern. Each microlens unit separates the incoming light into different viewpoints. Sub-aperture images that are essential in light-field imaging are easily obtained with a simple post-processing of the micro array pattern [16,17]. Initially, white micro light-field images are converted into a single greyscale image to calibrate. Subsequently, the centroid algorithm is utilized to determine the center of each microlens. Finally, the pixels are reordered to generate the sub-aperture images (Fig. 5(d)). Each viewpoint image in sub-aperture image has 182 x 300 pixels resolution. The sub-aperture image through the light-field camera based on MLA produces the same effect of the image obtained by the camera array [18,19]. The insets in Fig. 5(d) show different viewpoint images. The principle of the aforementioned type of multi-viewpoint imaging is shown in Fig. 5(e). In order to confirm the difference in the viewpoints, two images are converted into grayscale and subtracted from each other as shown in Fig. 5(f). The difference due to the view point appears as significant in nearby objects, such as the 30 cm target, since objects at long distances exhibit the effect of bringing virtual cameras closer to each other in stereo vision triangulation [20,21].
Fig. 5 Description of a hand-crafted light-field camera with conventional camera. (a) Schematic illustration of integration with a fabricated PDMS MLA and digital camera. (b) Photograph of the measurement setup to validate the applicability of the light-field camera. (c) Raw data obtained via the hand-crafted light-filed camera. (d) Processed sub-aperture image showing the multi view point property of the light-field camera. (e) Principle of the multi-view point property in the light-field camera. (f) View point difference between view point 1 and view point 2.
In order to measure the distance of focus plane due to the light field re-focusing process, the sharpness of fixed target was measured. Then, we compare the sharpness of each generated target sharpness of refocus image. There are several methods to refocusing process data in software by using a multi-view point that is used for triangulation. We use a basic refocusing process through a simple software process [22,23]. The main lens of camera is focused on the target at 60 cm. In order to digitally obtain the refocused images, we reordered sub-aperture images. which is interpolated 9 times larger for image process [22]. Thus, each refocused image has a resolution of 1638 x 2700 pixels. Figure 6(a) shows refocusing processed images that exhibit a normalized high sharpness at 30 and 90 cm from the image originally focused at 60 cm. The sharpness is measured for each target to evaluate the degree of refocusing of the processed results. The normalized sharpness is calculated via local gradient analysis [24]. The local gradient analysis aids in determining the focused plane by evaluating the degree of the image blur in window. Figure 6(b) shows the graph of normalized sharpness for the targets at 30, 60, and 90 cm. The black line of graph that is refocused at 30 cm shows low sharpness at the part with the exception of that at 30 cm. Details of the data on normalized sharpness are listed in Table 1.
Fig. 6 Results of light-field image processing. (a) Image of light-field refocusing processing. (b) Normalized sharpness graph of the refocused image at 30, 60, and 90 cm. (c) All-in-focused image by using the focal stacking method.
Table 1. Normalized sharpness of the refocused processed image
An all-in-focus image can be also generated by using the focal stacking method that replaces the part where the sharpness is high with the part where the sharpness is low (Fig. 6(c)) [25,26]. In the generated refocused images at various distances, relatively high sharpness area is masked at each refocused image via local gradient analysis through the window. Subsequently, stacking parts with relatively high sharpness in each refocused image can generate an all-in-focus image with high sharpness in the complete image area. Thus, the results confirm the functionality of the hand-crafted light-field camera including the refocusing process and all in focus process by using simple pixel reordering and focal stacking methods.
The light-field camera is tested under indoor and outdoor conditions as shown in Fig. 7 for a practical use demonstration. In order to compare each condition, the ISO sensitivity is maintained as fixed. The indoor condition is confirmed by capturing a billiard ball and tape measure. In the billiard raw data that is captured with a 35-mm main lens, the yellow ball is blurred since the original focus is formed at the far red ball. However, a closely refocused image shows an increase in the sharpness of the yellow ball. A far refocused image shows a second red ball blurred relative to the original image. An indoor test with a tape measure provides a refocus on the zoomed-in object. In order to provide a zoom effect, the main lens is set to 45 mm, and the focal point is placed on number 5 in the tape measure. In a closely or remotely refocused image, processed images show that each end of the tape measure are relatively blurred. An outdoor test is also performed to confirm the characteristics of the camera in places where the light intensity is high. The exposure time is reduced due to strong light intensity. A nearly refocused image shows that the left tree of the image exhibits high sharpness. However, the sharpness of the second tree is high at a distant refocused image. In summary, even in different ambient light and main lens conditions, the hand-crafted light-field camera produced by our PDMS MLA functions poorly with primitive light-field properties (Table 2).
Fig. 7 Test of the light field camera for practical use. Two indoor tests include billiard and tape measure object for general and zoomed-in objects. Outdoor test for a rich light intensity object Refocusing process was performed for various distances with 3 different conditions. (see Visualization 1, Visualization 2, Visualization 3).
Table 2. Detailed camera measurement parameters
4. Conclusions
In summary, we presented a cost-effective fabrication strategy for a geometrically engineered polymeric microlens array (MLA) based on a replica molding process by using a quartz mold. The quartz mold was easily fabricated via conventional semiconductor processes such as deposition, photolithography, and dry/wet etching steps. Initially, 700-nm thick Poly-Si layers were formed on both sides of a quartz substrate by chemical vapor deposition process. Subsequently, photolithography and dry etching process were performed to determine the lattice pattern and penetration via holes for the wet etching process. Subsequently, the dry-etched quartz substrate was immersed in a hydrofluoric acid (HF) bath and potassium hydroxide (KOH). In the wet etching step, the Poly-Si hard mask was removed via the KOH solution, and thus a uniform quartz mold with a large radius of curvature (RoC) and low sag height was implemented in a large-scale. The consecutive semiconductor processes provide a structural wide tuning range (i.e., RoC, sag height, density, and lattice pattern) of the quartz mold and lead to the fabrication of a highly dense and fine-tuned polymeric MLA. We used the engineered polymer MLA to assemble a customized hand-crafted light-field camera based on a commercial digital camera to demonstrate the applicability of our MLA as an unconventional optics for light-field imaging. As a demonstration, we obtained a micro array image via our light-field camera and digitally processed the raw image to convert it into a refocused image. Given the set of refocused images, we finally generated an all-in focus image that does not exhibit a blurred part in the image. Therefore, it is expected that the proposed fabrication scheme for widely controllable MLAs will be applied to diverse applications of micro optical devices including light-field imaging.
Funding
Agency for Defense Development of Korea and Defense Acquisition Program Administration Grant UD170079FD; GIST Research Institute (GRI) grant funded by the GIST in 2018; National Research Foundation of Korea (NRF) (NRF-2017H1A2A1042138/ NRF-2018H1A2A1060954).
References
1. I. Ihrke, J. Restrepo, and L. Mignard-Debise, “Principles of Light Field Imaging: Briefly revisiting 25 years of research,” IEEE Signal Process. Mag. 33(5), 59–69 (2016). [CrossRef]
2. S. Kim, Y. Ban, and S. Lee, “Face liveness detection using a light field camera,” Sensors (Basel) 14(12), 22471–22499 (2014). [CrossRef] [PubMed]
3. Y. Endo, K. Wakunami, T. Shimobaba, T. Kakue, D. Arai, Y. Ichihashi, K. Yamamoto, and T. Ito, “Computer-generated hologram calculation for real scenes using a commercial portable plenoptic camera,” Opt. Commun. 356, 468–471 (2015). [CrossRef]
4. F. Apelt, D. Breuer, Z. Nikoloski, M. Stitt, and F. Kragler, “Phytotyping(4D) : a light-field imaging system for non-invasive and accurate monitoring of spatio-temporal plant growth,” Plant J. 82(4), 693–706 (2015). [CrossRef] [PubMed]
5. G. Wang, W. Xiang, and M. Pickering, “A cross-platform solution for light field based 3D telemedicine,” Comput. Methods Programs Biomed. 125, 103–116 (2016). [CrossRef] [PubMed]
6. J. Sun, C. Xu, B. Zhang, M. M. Hossain, S. Wang, H. Qi, and H. Tan, “Three-dimensional temperature field measurement of flame using a single light field camera,” Opt. Express 24(2), 1118–1132 (2016). [CrossRef] [PubMed]
7. X. Huang, H. Qi, C. Niu, L. Ruan, H. Tan, J. Sun, and C. Xu, “Simultaneous reconstruction of 3D temperature distribution and radiative properties of participating media based on the multi-spectral light-field imaging technique,” Appl. Therm. Eng. 115, 1337–1347 (2017). [CrossRef]
8. S.-N. Li, Y. Yuan, B. Liu, F.-Q. Wang, and H.-P. Tan, “Influence of microlens array manufacturing errors on light-field imaging,” Opt. Commun. 410, 40–52 (2018). [CrossRef]
9. E. H. Adelson and J. Y. A. Wang, “Single lens stereo with a plenoptic camera,” IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 99–106 (1992). [CrossRef]
10. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Computer Science Technical Report CSTR 2, 1–11 (2005).
11. S. Di, H. Lin, and R. Du, “An artificial compound eyes imaging system based on mems technology,” in Robotics and Biomimetics (ROBIO), 2009 IEEE International Conference on, 13–18 (IEEE, 2009) [CrossRef]
12. S. Moore, J. Gomez, D. Lek, B. H. You, N. Kim, and I.-H. Song, “Experimental study of polymer microlens fabrication using partial-filling hot embossing technique,” Microelectron. Eng. 162, 57–62 (2016). [CrossRef]
13. Y. Luo, L. Wang, Y. Ding, H. Wei, X. Hao, D. Wang, Y. Dai, and J. Shi, “Direct fabrication of microlens arrays with high numerical aperture by ink-jetting on nanotextured surface,” Appl. Surf. Sci. 279, 36–40 (2013). [CrossRef]
14. J. Albero, L. Nieradko, C. Gorecki, H. Ottevaere, V. Gomez, H. Thienpont, J. Pietarinen, B. Päivänranta, and N. Passilly, “Fabrication of spherical microlenses by a combination of isotropic wet etching of silicon and molding techniques,” Opt. Express 17(8), 6283–6292 (2009). [CrossRef] [PubMed]
15. J. Yan, Z. Zhang, T. Kuriyagawa, and H. Gonda, “Fabricating micro-structured surface by using single-crystalline diamond endmill,” Int. J. Adv. Manuf. Technol. 51(9-12), 957–964 (2010). [CrossRef]
16. C. Hahne, A. Aggoun, V. Velisavljevic, S. Fiebig, and M. Pesch, “Baseline and triangulation geometry in a standard plenoptic camera,” Int. J. Comput. Vis. 126(1), 21–35 (2018). [CrossRef]
17. M. Levoy and P. Hanrahan, “Light field rendering,” in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 31–42, (ACM, 1996).
18. R. Ng, Digital light field photography (Stanford University, 2006).
19. K. Fife, A. El Gamal, and H.-S. P. Wong, “A 3D multi-aperture image sensor architecture,” in Custom Integrated Circuits Conference, 2006. CICC'06. IEEE, (IEEE, 2006), 281–284.
20. R. Hartley and A. Zisserman, Multiple view geometry in computer vision (Cambridge University, 2003).
21. G. Xu and Z. Zhang, Epipolar geometry in stereo, motion and object recognition: a unified approach. (Springer Science & Business Media, 2013).
22. C. Hahne, A. Aggoun, V. Velisavljevic, S. Fiebig, and M. Pesch, “Refocusing distance of a standard plenoptic camera,” Opt. Express 24(19), 21521–21540 (2016). [CrossRef] [PubMed]
23. T. Georgiev and C. Intwala, “Light field camera design for integral view photography,” Adobe Technical Report (2006).
24. D. Jacobs, “Image gradients,” Class Notes for CMSC 426 (2005).
25. D. Johnson, How to do everything: digital camera (McGraw-Hill, Inc., 2008).
26. S. Ray, Applied photographic optics (Routledge, 2002).
