Open Access
Add to BibSonomyAbstract
Spontaneous pattern formation in an optoelectronic system with an optical diffractive feedback loop exhibits a contrast enhancement effect, a spatial filtering effect, and filling-up of vacant space while maintaining surrounding structures. These effects allow image processing with defect tolerance. Aberrations and slight misalignments that inevitably exist in optical systems distort the spatial structures of the formed patterns. Distortion also increases due to a small aspect ratio difference between a display device and an image sensor. We experimentally demonstrate that the spatial distortion of the optoelectronic feedback system is reduced by electronic distortion correction and the initial structure of a seed optical pattern is preserved for a long time. We also demonstrate image processing of a fingerprint pattern based on seeded spontaneous optical pattern formation with electronic distortion correction.
©2005 Optical Society of America
1. Introduction
Spontaneous optical pattern formation in nonlinear systems with either an optical or optoelectronic diffractive feedback loop has recently become an important area of research in nonlinear optics [1–11]. An optical feedback system (OFS) is typically composed of an optically addressable spatial light modulator (OASLM) in a two-dimensional diffractive feedback loop. OASLMs exhibit a large space-bandwidth product, low light intensity operation, and highly tunable nonlinearity, which make them attractive for studying novel phenomena and applications [12–18]. An investigation of the temporal evolution of spontaneously formed optical patterns from an initial seed pattern is also interesting in the context of optical parallel image processing, but only a few studies have been reported. Recently, the formation of such patterns from seed optical patterns, which are continuously supplied by an external source, has been investigated in some nonlinear optical systems, and a spatial frequency filtering effect, spatial locking, and spatial synchronization were observed [19, 20].
Optoelectronic feedback systems (OEFSs), composed of an image sensor and display device, for example, also spontaneously form interesting patterns, typically, fringes (rolls) and hexagons [21–23], like OFSs. The advantages of OEFSs include the ease of supplying an initial pattern, the wide controllability of the nonlinearity, fewer constraints on the optical system, operation under low light levels due to the high sensitivity of the image sensor, the suitability of recently developed electronic hardware resources, and the availability of image processing software resources. OEFSs are typically composed of a liquid crystal display (LCD) and a charge coupled device (CCD) image sensor, which are readily available and fairly low cost.
We have demonstrated that the temporal evolution of the optical patterns in OEFS exhibits a contrast enhancement effect, a spatial filtering effect, and filling-up of vacant space while maintaining surrounding structures [23]. The existence of these effects was confirmed under the condition that a seed pattern was provided for just the first frame; that is, its operation was not continuously forced by a pattern. We have also demonstrated that these effects can perform optical image processing with defect tolerance, in particular, optical image processing for fingerprint patterns [23]. Fingerprint patterns are being widely applied for identification and security purposes. The verification of fingerprint patterns generally requires time-consuming preprocessing, including contrast adjustment and noise elimination, before extracting feature points (minutia), such as ridge endings, ridge divergences, bifurcations, dots, islands, and enclosures. However, the outcome of the feature extraction is strongly influenced by the fingerprint conditions, such as the existence of dry skin, sweat, and damage or injuries. Nevertheless, the OEFS in the above-mentioned study could transform the seed fingerprint pattern to the pattern without defects while maintaining surrounding structures.
In a practical OEFS, aberrations of the optical system and slight misalignments are inevitable. They distort the isotropic spontaneous pattern formation. Furthermore, the distortion of the patterns formed increases as a result of a small difference in aspect ratio between the LCD and the CCD image sensor. Consequently, the resulting distortions prevent preservation of features in the pattern evolution process, and reduce the performance of the OEFS for the optical image processing.
In this paper, we experimentally demonstrate that such distortions in optical pattern evolutions from an initial seed optical pattern, including fingerprint patterns, can be reduced with electronic distortion correction. In Section 2, the experimental setup of the OEFS is described. In Section 3, the principle of the electronic distortion correction for stabilization of patterns formed in the OEFS, that is, for long-term preservation of initial structures of the seed pattern, is described. In Section 4, it is experimentally demonstrated that the initial structures of the seed pattern is preserved with the electronic distortion correction, and the optical pattern evolutions with the long-term preservation property of the initial structures performs an optical image processing with defect tolerance for a fingerprint pattern. The defect tolerance is originated from filling-up effect of vacant space while maintaining surrounding structures. This is the most important characteristic in the use of the optical pattern evolutions in the OEFS for optical image processing. In Section 5, we conclude our study.
2. Optoelectronic feedback system
Figure 1 shows a schematic representation of the OEFS used in our experiments. Using a simple exponential relaxation response of an internal state u(x, y, t) of the LCD at a position (x, y) and time t to a control signal p(x, y, t), the space-time evolution of the internal state is
where τ is the time constant of the LCD and is the Laplacian in the x and y directions that describes transverse diffusion with a diffusion length l [21]. The LCD performs intensity modulation with a modulation characteristic F on the internal state u, and the output light wave A and its intensity I is
The intensity modulation characteristic F is approximated as F(u)=I 0[1-cos(u)]/2, where I 0 is the incident light intensity. The control signal p(x, y, t) for a diffractive feedback light intensity I d (x, y, t) on the CCD image sensor is described as
where G includes the input/output characteristic of the CCD image sensor, the electric conversion performed by a computer, which in this experiment is an inversion characteristic, and the conversion from an electric signal given to the LCD to the internal state u. The characteristic between the light intensities I 1 and I 2 as described by I 2=G[F(I 1)] is a sigmoid function. The characteristic is presented in Ref. 22, when the CCD image sensor and the LCD are directly connected without the electric conversion by a computer. The feedback light is formulated by considering free-space propagation over a length Z. By use of the stationary, scalar, paraxial wave equation, the amplitude of the diffracted feedback wave A d(x, y, Z, t) satisfies
where k=2π/λ is the wave number and A d(x, y, 0, t)=A(x, y, t). The model equations from (1) to (4) are equivalent to those for an OFS [10], because the time difference, given by a scanning mechanism of the LCD and the CCD image sensor, in the area interacting laterally with diffractive feedback is smaller than the response time of the LCD, and it is regard as the state of the LCD changes about the same time in the local area.
Fig. 1. Schematic representation of a model of an OEFS. CCD: charge-coupled device image sensor, LCD: liquid crystal display.
As shown in Fig. 2, the OEFS is mainly composed of an LCD and a CCD image sensor, which respectively perform conversion from a serial electrical signal to a parallel optical signal and vice versa. The components from the CCD image sensor to the LCD play a similar role to an OASLM, except for their spatially discrete pixels and scanning mechanism. Spontaneous pattern formation occurs when the intensity conversion characteristic from the CCD image sensor to the LCD is an inversion characteristic [22]. In our previous study, the inversion characteristic was achieved by the parallel Nicol arrangement of a polarizer and analyzer; this is different from conventional LCDs, where the crossed Nicol arrangement is used. The input/output characteristic that was determined experimentally is described in Ref. 22. In the present OEFS, the crossed Nicol arrangement, which gives the best performance of the LCD, is used, and the inversion characteristic is calculated in a personal computer. The computational load is very small compared with the calculation required for electronic distortion correction described later. The LCD is controlled by a frame grabber with a resolution of 640×480 pixels. The frame rate of the OEFS is less than or approximately equal to 30 Hz. The frame rate cannot be determined exactly, because the CCD image sensor and the LCD are operating asynchronously in the present OEFS.
The light from a He-Ne laser with wavelength λ=633 nm is spatially modulated by the LCD, and the modulated light is fed to the CCD image sensor through an optical system. The optical system performs a ~1/4 reduction of the image size on the LCD to match the image size on the CCD image sensor. The optical system can perform the linear image transformations of rotation, diffraction, and spatial frequency filtering for controlling the transverse interactions, in addition to expansion and reduction of the patterns. In the system, a low-pass spatial frequency filter (SFF) eliminates diffracted light originating from the pixel structure of the LCD. The CCD image sensor and a lens, which are enclosed within the dashed rectangle in Fig. 2, are axially moved as a single unit to adjust the free propagation length Z. Thus, Z is the distance from the image plane of the LCD to the plane P, which is imaged onto the plane of the CCD image sensor.
In practice, the OEFS has spatial distortions, including aberrations of the optical system, slight misalignments, and a small difference in aspect ratio between the LCD and CCD image sensor; therefore, the feedback intensity distribution I d(x, y, Z, t) is distorted. A personal computer and fram-grabber are used not only to give an initial seed pattern, to capture the image sequences, and to analyze the temporal evolution of the patterns, but also to calculate the correction of these distortions.
3. Electronic distortion correction
In order to reduce the distortion, electronic distortion correction is introduced in the OEFS. In this electronic distortion correction, the transformation from the LCD to the CCD image sensor is approximated with geometrical optics, and the image detected with the CCD image sensor is returned to the LCD with an inverse geometrical transformation. Let the coordinates on the plane of the LCD and the CCD image sensor be (x, y) and (u, v), respectively. The geometrical transformation from the LCD plane to the CCD image sensor plane involves a lateral shift between the devices, lateral magnifications, including the aspect ratio mismatch between the devices, device rotation, and nonlinear distortions. The output point (x 1, y 1) is transformed by lateral deviation and lateral magnification from the input point (x, y) as:
The point (x 1, y 1) is transformed to the point (x 2, y 2) by a rotation of the CCD image sensor by an angle Θ, and is denoted as:
When (x 2, y 2) is converted to polar coordinates (r 2, θ 2), the nonlinear distortion is approximated with a 3rd order polynomial as:
Where C is a coefficient, and r 2=(+)1/2. Then, (r 2, θ 2) is converted back to the rectangular coordinates (u, v) on the CCD image sensor by:
where θ 2=arctan(y 2/x 2). Therefore, the transformation from (x, y) to (u, v) that represents the geometry of the OEFS is described by Eqs. (5–8).
The six parameters A 0, A 1, B 0, B 1, C, and Θ are set based on the following calculation. At first, a 27×19 array of N small bright points is displayed on the LCD as sampling points at the same time, and the image is detected by the CCD image sensor through the optical system. Next, the coordinates of n-th detected point, [u 0(n), v 0(n)] (n=1, 2, …, N), are obtained by the threshold operation of the image and the centroid detection of each bright area. From the relation between a point at (x, y) on the LCD and the corresponding point at (u, v) on the CCD image sensor, the coefficients in Eqs. (5–8) are determined by the downhill simplex method [24] for the estimation function [u-u 0 (n)]2+[v-v 0(n)]2. Finally, the coordinates (u, v) on the CCD image sensor corresponding to all pixels on the LCD are calculated. The coefficients calculated with the method described above were A 0=0.977, A 1=1.00, B 0=-0.267, B 1=-0.537, C=0.000176, and Θ=0.00125. In this case, since the optical system was set such that the deviation at its center was small, the distortion became bigger at the periphery of the operation region. The system has not only linear distortions, such as a lateral displacement, magnification mismatch, rotation, and aspect ratio difference, but also the nonlinear distortion caused by aberrations of the optical system.
In general, the coordinates (u, v) calculated from a point (x, y) by Eqs. (5–8) do not match the pixel position of the CCD image sensor. Therefore, I(u, v) is calculated using linear interpolation of the four pixel values on the CCD image sensor, I(i,j), I(i,j+1), I (i+1,j), I (i+1,j+1), where i and j are integer, that are neared to (u, v) as follows:
where p=u/W-i and q=v/W-j, and W is the defined size of a pixel; in our experiments, W=1.
4. Experimental results
Fringes formed in the OEFS have a main spatial wave number K z=2π(λ|Z|)-0.5 on the plane P, for the free-propagation length Z [8, 16, 17]. Therefore, when circular fringes with the wave number K z are initially supplied, the circular fringes should be maintained for a long time. In a practical OEFS, however, the initially-given circular fringes with K z became distorted gradually, as shown in Fig. 3, because the OEFS has aberrations, slight misalignments, and an aspect ratio difference, as mentioned above. Figure 3(a) shows the temporal evolution of patterns whose center coincides with the optical axis of the system (on-axis region). The size of the images is 100×100 pixels on the LCD. Figure 3(b) shows patterns whose center is shifted from the optical axis (off-axis region) by 100 pixels in the x-axis and 100 pixels in the y-axis on the LCD. These patterns were observed at 1/30 s, 5/30 s, 10/30 s, 15/30 s, and 20/30 s, respectively. The distortion of the formed patterns becomes larger at the periphery. The patterns in Figs. 3(a) and 3(b) were obtained without electronic distortion correction.
Next, an experiment using the same circular fringes as the initial input was performed in the OEFS with electronic distortion correction. As shown in Fig. 4, the distortion was reduced drastically; the difference between the patterns formed in the on-axis and off-axis regions was very small. Furthermore, the initial structure of the formed patterns was well-maintained in both regions. Figure 5 shows the changes of the sum of the squared difference (SSD) between each temporally evolved pattern I 1(i, j, t) and the pattern at the 1st frame. The SSD for measuring the difference between a temporally evolving pattern I 1(i, j, t) with a mean µ 1(t) and a variance σ 1(t)2 and a temporally evolving pattern I 2(i, j, t) with a mean µ 2(t) and a variance σ 2(t)2 is defined as
where N is the number of pixels. In this experiment, I 2(i, j, t)=I 1(i, j, 1), µ 2(t)=µ 1(1), and σ 2(t)=σ 1(1). For example, the SSD between two random patterns is about 0.015. In the on-axis region, because the distortion of the system is small, the effect of the distortion correction is not noticeable. On the other hand, in the off-axis region, the reduction of the SSD is remarkable and the distortion correction worked very well. By introducing the electronic distortion correction, the OEFS became to preserve the initial structures for long time in the pattern evolution process.
Fig. 3. Temporal evolution of (a) patterns whose operating center coincides with the optical axis of the system (on-axis region), and (b) patterns whose operating center is shifted from the optical axis (off-axis region) by 100 pixels in the x-axis and 100 pixels in the y-axis on the LCD. These patterns are observed at 1/30 s, 5/30 s, 10/30 s, 15/30 s, and 20/30 s from the left, respectively.
Fig. 4. Temporal evolutions of patterns (a) in the on-axis region, and (b) in the off-axis region in the OEFS with electronic distortion correction.
Fig. 5. Temporal changes of the SSDs between each temporally evolving pattern and the pattern at the 1st frame (a) in the on-axis region and (b) in the off-axis region in the OEFS with electronic distortion correction. The dotted and the solid lines indicate the SSDs of the pattern evolutions in the OEFS (a) without and (b) with the electronic distortion correction, respectively.
We next apply the OEFS with electronic distortion correction to image processing of a fingerprint pattern. Figures 6(a) and 6(b) show an original fingerprint pattern A with a natural defect and the same fingerprint pattern A′ with an artificial defect, respectively. The fingerprint pattern was obtained by pushing it against a glass plate, illuminating it with a red light emitting diode, and detecting the reflected light intensity distribution with a CCD image sensor. When the fingerprint patterns without and with the artificial defect, A and A′, were initially supplied to the OEFS with electronic distortion correction, the respective temporal evolutions at 1/30 s, 7/30 s, and 22/30 s are shown in Figs. 7(a) to 7(c) and Figs. 7(d) to 7(f), respectively. These patterns were in the off-axis region. Both temporal evolutions proceed while increasing the contrast and removing both the natural and the artificial defects, when the system is configured so that the principal wave number of the fingerprint pattern matches K z, because the optical pattern in the vacant space (defect) is formed while maintaining and connecting surrounding structures with K z, when the shorter side of the defect has the length shorter than or approximately equal to the fringe spacing (2π/K z), as the artificial defect given to the fingerprint pattern shown in Fig. 6(b). The bifurcation, which is a typical minutia, which initially existed in the fingerprint pattern, was well preserved, and its position was almost fixed. Furthermore, the pattern transformed from the fingerprint pattern A′ with an artificial defect, shown in Fig. 7(c), almost agreed with the pattern transformed from the original fingerprint pattern according to the temporal evolution of patterns, as shown in Fig. 7(f). Therefore, the OEFS performs image conversion while exhibiting defect tolerance. The defect tolerance is originated from filling-up effect of vacant space while maintaining surrounding structures in spontaneous optical pattern formation. This characteristic is most important and novel in the use of the optical pattern evolutions in OEFS for optical image processing.
Fig. 6. (a) An original fingerprint pattern A and (b) the same pattern A′ with an artificial defect.
Fig. 7. Temporal evolutions at 1/30 s, 10/30 s, and 22/30 s when the fingerprint patterns without and with the artificial defect are initially supplied to the OEFS with the electronic distortion correction.
Figure 8 shows the temporal changes of the SSDs between the temporal evolutions of the original fingerprint pattern A and those of the fingerprint pattern A′ with the defect. The bold dashed curve and the bold solid curve indicate the SSDs when the OEFS was used without and with the electronic distortion correction, respectively. The SSDs in the OEFS gradually decayed with oscillation. Introducing the electronic distortion correction resulted in a smaller SSD. The effect of the correction was particularly noticeable in the off-axis region. The oscillation is caused by a drop in the frame rate of the OEFS [22] caused by the increased computational complexity required for the electronic distortion correction. The temporal pattern evolutions starting from the fingerprint pattern with the defect gradually approached those from the original fingerprint pattern; that is, the OEFS operated properly regardless of the defect. The SSDs, however, started to increase from the 15th frame (0.5s). The SSDs between two trials starting from same initial pattern in the OEFS without and with the electronic distortion correction gradually increased, as indicated by the thin dashed curve and the thin solid curve in Fig. 8. From the experiment, it is thought that slight distortions have not been removed completely.
Fig. 8. The bold dashed curve and the bold solid curve indicate the temporal changes of the SSDs between the temporal evolutions of the original fingerprint pattern and those of the fingerprint pattern with the artificial defect, when the OEFS was used without and with the electronic distortion correction, respectively. The thin dashed curve and the thin solid curve indicate the temporal changes of the SSDs between two trials starting from same initial pattern in the OEFS without and with the electronic distortion correction, respectively.
5. Conclusions
We have experimentally demonstrated spontaneous optical pattern formation from an initial seed pattern in an optoelectronic feedback system composed of an LCD and a CCD image sensor. The optoelectronic feedback system, with electronic distortion correction, produces patterns with reduced distortion, caused by inevitable aberrations and slight misalignments that exist an optoelectronic system, and a small difference in aspect ratio between the display (LCD) and the camera (CCD). Therefore, the initial structure of the seed pattern can be maintained for a long time with electronic distortion correction. A contrast enhancement effect, a spatial filtering effect, and filling-up of vacant space while maintaining surrounding structures allow image processing with defect tolerance. In particular, the filling-up of vacant space in spontaneous optical pattern formation is a unique effect that is unattainable using linear optical processing such as a spatial filtering. Preprocessing, including contrast adjustment and noise elimination, which is generally carried out before feature extraction and verification, are important in the image-processing field. The spontaneous optical pattern formation system we propose performs such processing automatically. Therefore, we expect that spontaneous optical pattern formation created with a nonlinear system including an optical diffractive feedback loop will offer various advantages in the image-processing field.
Acknowledgments
This research was partially supported by the Ministry of Education, Culture, Sports, Science and Technology of Japan, Grant-in-Aid for Scientific Research on Priority Area: molecular synchronization for design of new materials systems and Grant-in-Aid for Scientific Research (B), and the Satellite Venture Business Laboratory of The University of Tokushima.
References and Links
1. M. Kreuzer, W. Balzer, and T. Tschudi, “Formation of spatial structures in bistable optical elements containing nematic liquid crystals,” Appl. Opt. 29, 579–582 (1990). [CrossRef] [PubMed]
2. F. T. Arecchi, G. Giacomelli, P. L. Ramazza, and S. Residori, “Experimental evidence of chaotic itinerancy and spatiotemporal chaos in optics,” Phys. Rev. Lett. 65, 2531–2534 (1990). [CrossRef] [PubMed]
3. G. D’Alessandro and W. J. Firth, “Spontaneous hexagon formation in a nonlinear optical medium with feedback mirror,” Phys. Rev. Lett. 66, 2597–2600 (1991). [CrossRef] [PubMed]
4. S. A. Akhmanov, M. A. Vorontsov, V. Yu. Ivanov, A. V. Larichev, and N. I. Zheleznykh, “Controlling transverse-wave interactions in nonlinear optics: generation and interaction of spatiotemporal structures,” J. Opt. Soc. Am. B 9, 78–90 (1992). [CrossRef]
5. R. Macdonald and H. J. Eichler, “Spontaneous optical pattern formation in a nematic liquid crystal with feedback mirror,” Opt. Commun. 89, 289–295 (1992). [CrossRef]
6. B. Thüring, R. Neubecker, and T. Tschudi, “Transverse pattern formation in liquid crystal light valve feedback system,” Opt. Commun. 102, 111–115 (1993). [CrossRef]
7. E. Ciaramella, M. Tamburrini, and E. Santamoto, “Talbot assisted hexagonal beam patterning in a thin liquid crystal film with a single feedback mirror at negative distance,” Appl. Phys. Lett. 63, 1604–1606 (1993). [CrossRef]
8. T. Honda, “Hexagonal pattern formation due to counterpropagation in KNbO3,” Opt. Lett. 18, 598–600 (1993). [CrossRef] [PubMed]
9. P. P. Banerjee, H. L. Yu, D. A. Gregory, N. Kukhtarev, and H. J. Caufield, “Self-organization of scattering in photorefractive KNbO3 into reconfigurable hexagonal spot array,” Opt. Lett. 20, 10–12 (1995). [CrossRef] [PubMed]
10. R. Neubecker, G.-L. Oppo, B. Thuering, and T. Tschudi, “Pattern formation in a liquid-crystal light valve with feedback including polarization, saturation, and internal threshold effect,” Phys. Rev. A 52, 791–808 (1995). [CrossRef] [PubMed]
11. M. A. Vorontsov and W. B. Miller (Eds.), “Self-organization in Optical systems and applications in information technology,” Chapter 2 (Berlin, Springer-Verlag, 1995).
12. E. V. Degtiarev and M. A. Vorontsov, “Spatial filtering in nonlinear two-dimensional feedback systems: phase-distortion suppression,” J. Opt. Soc. Am. B 12, 1238–1248 (1995). [CrossRef]
13. Y. Hayasaki, H. Yamamoto, and N. Nishida, “Optical dependence of spatial frequency of formed patterns on focusing condition in a nonlinear optical ring resonator,” Opt. Commun. 151, 263–267 (1998). [CrossRef]
14. V. Mamaev and M. Saffman, “Selection of unstable patterns and control of optical turbulence by Fourier plane filtering,” Phys. Rev. Lett. 80, 3499–3502 (1998). [CrossRef]
15. S. J. Jensen, M. Schwab, and C. Denz, “Manipulation, stabilization, and control of pattern formation using Fourier space filtering,” Phys, Rev. Lett. 81, 1614–1617 (1998). [CrossRef]
16. G. K. Harkness, G.-L. Oppo, R. Martin, A. J. Scroggie, and W. J. Firth, “Elimination of spatiotemporal disorder by Fourier space techniques,” Phys. Rev. A 58, 2577–2585 (1998). [CrossRef]
17. Y. Hayasaki, H. Yamamoto, and N. Nishida, “Selection of optical patterns using direct modulation method of spatial frequency in a nonlinear optical feedback system,” Opt. Commun. 187, 49–55 (2001). [CrossRef]
18. R. Neubecker, E. Benkler, R. Martin, and G. -L. Oppo, “Manipulation and removal of defects in spontaneous optical patterns,” Phys. Rev. Lett. 91, 113903 (2003). [CrossRef] [PubMed]
19. R. Neubecker and A. Zimmermann, “Spatial forcing of spontaneous optical patterns,” Phys. Rev. E 65, 035205(R) (2002). [CrossRef]
20. R. Neubecker, A. Zimmermann, and O. Jakoby, “Utilizing nonlinear optical pattern formation for a simple image-processing task,” Appl. Phys. B 76, 383–392 (2003). [CrossRef]
21. M. A. Vorontsov, G. W. Carhart, and R. Dou, “Spontaneous optical pattern formation in a large array of optoelectronic feedback circuits,” J. Opt. Soc. Am. B 17, 266–274 (2000). [CrossRef]
22. Y. Hayasaki, Y. Tamura, H. Yamamoto, and N. Nishida, “Spatial property of formed patterns depending on focus condition in a two-dimensional optoelectronic feedback system,” Jpn. J. Appl. Phys. 40, 165–169 (2001). [CrossRef]
23. Y. Hayasaki, E. Hikosaka, H. Yamamoto, and N. Nishida, “Image processing based on seeded spontaneous optical pattern formation using optoelectronic feedback,” Appl. Opt. 44, 236–240 (2005). [CrossRef] [PubMed]
24. J. A. Nelder and R. Mead, “Simplex method for function minimization,” Computer J. 7, 308–313 (1965).
