In this paper, we propose a method for efficient gonio-imaging of optically variable devices (OVDs), which are applied as a counterfeit deterrence for valuable documents. A compound-eye image-capturing system composed by a microlens array, a signal separator, and an image sensor was used to capture directionally distributed light from OVDs after being collimated by a convex lens. Multiple images corresponding to different observation angles were obtained in the individual eyes of the system, simultaneously and independently. A demonstration involving a holographic grating provided 100 gonio images that exhibited sensitive color changes of the diffracted light according to the observation angle.
© 2011 OSA
In recent years, optically variable devices (OVDs) , such as holograms and diffraction gratings, have been widely used in security documents, such as banknotes, passports, credit cards, and driver's licenses. OVDs are devices whose color and appearance drastically change according to the illumination and observation directions. These gonio properties of OVDs are important in their role as anti-counterfeiting devices for security documents, preventing fraud and counterfeiting using conventional photocopiers, image scanners, and printers.
The gonio properties of OVDs are important in the authentication of security documents, and they are also crucial in the field of forensic document examination. The difference in gonio properties between a questioned sample and reference materials often gives us effective information about whether or not the sample is genuine. The optical phenomena involved in the gonio properties of OVDs are diffraction and interference in many cases. The light diffracted by gratings and the interference light in thin layers in OVDs show drastic changes according to slight changes in their microscopic structures or optical parameters.
Machine-based gonio property analysis will be necessary as we move toward secure authentication of OVDs. In many cases, the inspection of gonio properties of OVDs is based on visual inspection by human eyes. It is typically performed by holding a document and flipping and rotating it to observe the appearance under various illumination and observation conditions. However, it is quite difficult to detect and understand the complicated gonio properties of OVDs only with the human eye.
Efficient gonio imaging is important in machine inspection of OVDs' gonio properties. In general, gonio imaging is considered time-consuming and cumbersome since it requires a large number of image acquisition procedures under varying illumination and observation geometries. It also tends to require large, complicated equipment [2,3].
There are several studies concerning efficient measurement of gonio reflection properties of objects. Dana et al.  proposed an efficient acquisition method based on a bidirectional reflectance distribution function (BRDF)  by using a concave parabolic mirror. This method measures the BRDF of one point on the object at a time; however, it still requires two dimensional scanning to obtain gonio images. Han et al.  used a method involving simultaneous acquisition of bidirectional texture reflectance by using a kaleidoscope. This research is in the context of measurement of a bidirectional texture function (BTF) . Efficient measurement is achieved by skillfully expanding the observation angle via multiple reflections in the kaleidoscope. However, the obtained image is not rectangular. In addition, a large kaleidoscope is required in order to enlarge the range of observation directions.
Multichannel devices also provide efficient gonio imaging methods. Müller et al.  proposed a system with a dense array of digital still cameras uniformly mounted on a hemispherical structure. A similar gantry was used by Malzbender et al. to capture Polynomial Texture Maps . These systems enabled us to capture gonio images without scanning the image sensors and the light sources. However the size of these systems is still large.
A microlens array has the ability to acquire a light-field including directional information of the light wave, such as in 3D imaging by integral photography [10,11] and a plenoptic camera . Tanida et al. proposed a compound-eye image-capturing system called TOMBO (Thin Observation Module by Bound Optics) . The TOMBO system is an optoelectronic hybrid system for computational imaging with a thin optical system. It has been applied to a variety of applications, including color imaging , spectral imaging  and high-resolution image reconstruction . Recently, the framework for generalized sampling approaches for measuring multi-dimensional objects by using TOMBO was proposed .
In this paper, we propose an efficient method of gonio-imaging of OVDs by using compound-eye optics and a convex lens. The TOMBO system is used as the compound-eye optics. After the principle of the proposed method is described, an experiment involving gonio image acquisition of a holographic grating is demonstrated, and then we discuss issues related to the proposed method.
Figure 1 shows the principle of the proposed method. The optical element used in this method is composed of a convex lens and a microlens array. First, the light from a sample is collimated by the convex lens, which is located away from the sample by the focal length f of the lens. The collimated light proceeds to the microlens array composed of multiple convex lenses. The light is focused by each convex lens onto the image plane, and therefore, multiple images are obtained.
The key point of this method is that angle-resolved light from the sample is focused individually by each microlens. As seen from Fig. 1, the position of each convex lens in the microlens array corresponds to a different angle of the light from the sample. Therefore, the image focused by each convex lens corresponds to an image of the sample taken from a different observation direction.
Now, let us formulate the image formation in the proposed optical configuration shown in Fig. 2 . In Fig. 2, P1 is the object plane, L is the convex lens, M is the microlens array, and P2 is the image plane. For simplicity, assume that L and the center microlens in M are coaxial. The focusing condition is derived from the lens equation under the condition of the two-lens combination, as follows:
The observation direction at each microlens in M is introduced. Figure 3 is a schematic diagram of geometrical variables defining the measurement geometry. The vectors i and r are the directions of illumination and observation, respectively, and the vector n is the surface normal of the measurement sample. The angles and are the polar and azimuthal angles of the illumination direction, respectively. The observation angle is represented in the same manner as the incident direction by replacing suffix i with r. The polar angle of observation direction r at a microlens whose center is located at in plane M is represented by
The feasibility of proposed method was demonstrated by an experiment. Figure 4 shows a schematic diagram of the experimental setup. The distances corresponding to a, d, and b in Fig. 2 were 25.5 mm, 7 mm, and 1.3 mm, respectively.
The TOMBO system was used as the compound-eye optics. Figure 5 shows a schematic diagram of TOMBO. It consists of a microlens array, a signal separator, and an image sensor. Multiple images are captured by the individual imaging units. A small image captured by one unit is called a unit image. Table 1 summarizes the specifications of the TOMBO system used in this study. A CCD color image sensor with a Bayer filter array was used. The number of units was 10 × 10. A color image of 80 × 80 pixels was captured in each unit image. The illumination was white light from a quartz halogen light source (Dolan-Jenner Industries, DC-950H). The light was guided by a glass fiber bundle, and shaped by a holographic diffuser (Edmund, 54499) with diffuse angle of 25 deg. The polar angle and the azimuthal angle of the illumination direction i were approximately 67 deg and 0 deg, respectively. The objective lens was an achromatic lens (Edmund, 45399-F) with a focal length of 18 mm and a diameter of 9 mm. The magnification K of this imaging setup was 0.06, and measurement region on the sample was approximately 9 mm × 9 mm.
Figure 6 shows a schematic diagram of the compound-eye image consisting of unit images in the x”-y” plane. The row and column number of each unit image are defined as shown in this figure. Tables 2 and 3 list the polar angle and azimuthal angle of the observation direction r at the center of each unit image. These were calculated by using Eqs. (3) and 4. The range of polar angles projected on the x-z plane was 11 deg. The difference of the projected polar angle between adjacent units along a row of the compound image was approximately 1.2 deg.
Figure 7 is an image of an experimental sample for gonio-imaging. The sample was a holographic grating (Edmund, 43775) with a groove number of 1800 lines per millimeter. On the surface of the grating, five pieces of sticker was attached in order to demonstrate the efficiency of imaging. The size of the holographic grating was 12.7 mm square, and the grating grooves were in vertical direction of this figure.
Figure 8 shows the result of gonio-imaging of the experimental sample. One hundred images from different observation angles were captured in a single shot without geometrical scanning. The gonio-image consisted of 10 × 10 unit images of the TOMBO system. In each unit image, diffracted light from the grating was focused, and a sample image was captured. The row number and column number of the unit images are shown around the image.
The TOMBO system acquired a compound-eye image of high quality. There was no cross-talk between adjacent unit images due to the separating walls between unit images. The resolution was enough to show the details of the sticker on the grating. Significant distortions were not observed. The imaging region was almost the same, though slightly different, between neighboring unit images. The effect of vignetting became significant in unit images distant from the center of the compound-eye image. In such unit images, the edge of the convex lens came into the image region, limiting the imaging range.
In the gonio-image, gradual color changes of the sample grating according to the observation direction were successfully captured. The color of the sample grating was different between unit images. It was red in the left part, green in the middle part, and blue in the right part. That is, the color changed along the horizontal direction, which corresponds to the x”-axis in Fig. 2. Also, the color gradually changed in each unit image, from a longer wavelength color in the left part to a shorter wavelength color in the right part. On the other hand, the color was almost constant along the vertical direction.
Color changing of the gonio-image was consistent with gonio properties of the sample grating. In a holographic grating, the wavelength of the diffracted light becomes longer as the angle from the direction of specular reflection increases. In the left part of the gonio-image, the observation direction was farther from the specular direction than in the right part. According to the calculation using grating equation, the wavelength at the center of individual eye at column 1 in Fig. 8 was 565 nm. It was consistent with the color of captured image, which is yellowish green at this part. Also in the same way, the wavelength 500 nm at column 6 was consistent with the bluish green color, and the wavelength 455 nm at column 10 was consistent with the blue color in captured gonio images. The same behavior was observed also within each unit image.
The proposed method is considered to be effective for image-based inspection of OVDs. At first, the gonio image information was obtained efficiently in quite short time. In this experiment, the gonio images were captured simultaneously within a frame time less than one second. Real time monitoring of gonio images is also available by showing the image information from TOMBO system on a display device. It is at least 6400 times shorter than the method proposed by Dana et al. . Compared with the method capturing images by scanning the geometries one by one, it is at least 100 times shorter even if ignoring the time for geometrical scanning.
Secondly, the proposed system successfully captured sensitive color changes of the diffracted light from holographic grating according to observation angle. In this experiment, the difference of the wavelength of diffracted light between adjacent individual eyes in row direction of TOMBO system was 12 nm according to the calculation using grating equation. Now suppose the case of grating images, which are representatives of OVDs. The groove numbers of their diffractive grating range from 760 to 1700 lines per millimeter according to an author’s previous study . The difference of wavelength in first-order diffracted light between adjacent individual eyes ranges from 13 nm to 29 nm in the case that the grating direction is parallel to horizontal or vertical axis of TOMBO system. Therefore the proposed system will be able to detect gradual color change of the diffracted light from OVDs.
The proposed system also covered wide range of diffracted light. In this experiment, the range of observation direction along horizontal axis of TOMBO system was 11.4 deg, which covered 27% of diffracted light in visible wavelength region from 380 to 780 nm. In the case of grating images stated above, the expanses of first-order diffracted light in visible wavelength region range from 17 deg to 40 deg, therefore from 29% to 65% of them is able to be covered with the observation angle range of the proposed system.
The proposed system balanced between efficiency in the imaging and the resolution of observation angle. It is advantageous feature to perform rough classification of OVDs in the first step of inspection efficiently. Combining with the conventional method with higher resolution in observation angle or imaging, efficient and effective inspection scheme of OVDs will be achieved.
The proposed method will simplify the comparison of gonio reflection properties of OVDs. The comparison between a sample and reference material is achieved by using the same apparatus based on the proposed method, since the measurement geometries are considered to be constant between two image acquisition procedures. Especially, relative relationships of observation directions between individual eyes are rigid since the microlens array is fixed on an area image sensor. Therefore one can easily detect the differences between a sample and reference material by the difference of these two gonio-images. The decision whether or not the sample is genuine will be also achieved by setting a threshold.
The proposed method is now effective in the scheme of comparison between two or more samples, even if absolute value concerning the angle of measurement geometries is not determined. In future work, it should be improved into an absolute measurement method by confirming the angular accuracy in measurement geometries.
Furthermore, gonio-images of OVDs may give us information about the spatial distribution of structural parameters. This will allow structural parameters, such as the pitch of a grating and the thicknesses of thin films or multiple layers, to be used as quantitative indices for the examination of OVDs.
The proposed method is also advantageous for constructing compact, hand-held authentication devices for OVDs. The optical elements of the proposed experimental setup are small because of the integrated optical elements provided in the TOMBO system. Moreover, the absence of any mechanical scanning devices is also advantageous in reducing the size of the system.
However, there are some problems. First, the difference of the reflection angle within each unit image should be considered. As seen from the obtained gonio-image, the reflection angle was not constant within each unit image. Calibration of the reflection angle at each pixel in the unit image should be performed in future work.
Another problem is the specifications of the objective lens. In this experimental setup, the range of observation angles was not so large. In order to solve this problem, an objective lens with small F-number, i.e. the lens with larger diameter in the case of the same focal length, will be of help. The diameter of objective lens is also important factor. In order to avoid the problems of vignetting as seen in the obtained gonio-image, it must be at least larger than the size of microlens array. The larger diameter will be required as the distance between objective lens and microlens array increases.
The magnification K of this system is not adjustable in the proposed optical configurations. In future work, it should be optimized for the inspection of OVDs by investigating the size and resolution of OVDs. The magnification K has relationship with F-number of imaging system. The F-number of this experimental setup was 3.8, which may be slightly larger than conventional camera system. However this drawback is able to be compensated for by sufficient amount of illumination.
The number of compound eyes is also an important factor in this method. It affects the observation angle resolution and the number of pixels in each unit image. These parameters are in the a trade-off relationship. Suppose the system has N × N individual eyes, the number of pixels in each unit image is in proportion to . The half angle of the light field cone captured by an elementary channel is proportional to . The illumination angle and the range of observation angles are also crucial. In future work, it will be important to optimize the configuration of the system based on the properties of the target OVDs.
In this paper, we proposed a method for efficient gonio-imaging of OVDs by combining compound-eye optics and a convex lens. A compound-eye image-capturing device called TOMBO and an achromatic lens were used for the experiment. Experimental results for a holographic grating showed that 100 gonio images from different observation angles were successfully captured. The gonio images showed sensitive color changes of the diffracted light from the holographic grating according to the observation angle. The gonio images were captured in a single shot without geometrical scanning. The proposed method is considered to be effective for efficient gonio-imaging of OVDs. In future work, a gonio-imaging system based on this method will be constructed. Optimization of the optical configuration, such as the measurement geometry and the number of compound eyes, will be required.
The authors thank Dr. R. Horisaki of the Graduate School of Information Science and Technology of Osaka University for his dedicated assistance in the experiment.
References and links
1. R. L. van Renesse, Optical Document Security, Third edition (Artech House, 2005).
2. M. Koudelka, S. Magda, P. Belhumeur, and D. Kriegman, “Acquisition, compression, and synthesis of bidirectional texture functions,” in Texture 2003: Proceedings of 3rd International Workshop on Texture Analysis and Synthesis, (2003), pp. 59–64.
3. M. Sattler, R. Sarlette, and R. Klein, “Efficient and realistic visualization of cloth,” in Eurographics Symposium on Rendering 2003, P. Christensen and D. Cohen-Or ed., (Eurographics/ACM, 2003), pp. 167–177.
4. K. J. Dana and J. Wang, “Device for convenient measurement of spatially varying bidirectional reflectance,” J. Opt. Soc. Am. A 21(1), 1–12 (2004). [CrossRef]
6. J. Y. Han and K. Perlin, “Measuring bidirectional texture reflectance with a kaleidoscope,” ACM Trans. Graph. 22(3), 741–748 (2003). [CrossRef]
7. K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink, “Reflectance and texture of real world surfaces,” ACM Trans. Graph. 18(1), 1–34 (1999). [CrossRef]
8. G. Müller, G. Bendels, and R. Klein, “Rapid synchronous acquisition of geometry and BTF for cultural heritage artefacts,” in the 6th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST), (Eurographics Association, 2005), pp. 13–20.
9. T. Malzbender, D. Gelb, and H. Wolters, “Polynomial texture maps,” in ACM SIGGRAPH 2001: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, (ACM Press, 2001), pp. 519–528.
10. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. 7, 821–882 (1908).
11. H. E. Ives, “Parallax panoramagrams made with a large diameter lens,” J. Opt. Soc. Am. 20(6), 332–342 (1930). [CrossRef]
12. R. Ng, M. Levoy, M. Brdif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-heldplenoptic camera,” Stanford Tech. Rep. CTSR 2005–02 (Stanford University, 2005).
13. J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, “Thin observation module by bound optics (TOMBO): concept and experimental verification,” Appl. Opt. 40(11), 1806–1813 (2001). [CrossRef]
14. J. Tanida, R. Shogenji, Y. Kitamura, K. Yamada, M. Miyamoto, and S. Miyatake, “Color imaging with an integrated compound imaging system,” Opt. Express 11(18), 2109–2117 (2003). [CrossRef] [PubMed]
16. Y. Kitamura, R. Shogenji, K. Yamada, S. Miyatake, M. Miyamoto, T. Morimoto, Y. Masaki, N. Kondou, D. Miyazaki, J. Tanida, and Y. Ichioka, “Reconstruction of a high-resolution image on a compound-eye image-capturing system,” Appl. Opt. 43(8), 1719–1727 (2004). [CrossRef] [PubMed]
17. R. Horisaki, K. Choi, J. Hahn, J. Tanida, and D. J. Brady, “Generalized sampling using a compound-eye imaging system for multi-dimensional object acquisition,” Opt. Express 18(18), 19367–19378 (2010). [CrossRef] [PubMed]
18. Y. Akao, A. Yamamoto, and Y. Higashikawa, “Study on the groove number of 2D hologram applied to security documents,” in Abstract of 14th Annual Meeting of Japanese Association of Forensic Science and Technology, M. Yoshino, ed. (Komiyama Printing Co., Ltd., Tokyo, Japan, 2008), p.191 [in Japanese].