Abstract

We describe a nonlinear joint transform correlator-based two-layer neural network that uses a supervised learning algorithm for real-time face recognition. The system is trained with a sequence of facial images and is able to classify an input face image in real time. Computer simulations and optical experimental results are presented. The processor can be manufactured into a compact low-cost optoelectronic system. The use of the nonlinear joint transform correlator provides good noise robustness and good image discrimination.

© 1995 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. H.-Y. Li, Y. Qiao, D. Psaltis, “Optical network for real-time face recognition,” Appl. Opt. 32, 5026–5035 (1993).
    [Crossref] [PubMed]
  2. J. H. Hong, S. Campbell, P. Yeh, “Optical pattern classifier with perceptron learning,” Appl. Opt. 29, 3019–3025 (1990).
    [Crossref] [PubMed]
  3. D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [Crossref]
  4. A. D. McAulay, J. Wang, X. Xu, “Optical perceptron learning for binary classification with spatial light rebroadcasters,” Appl. Opt. 32, 1346–1353 (1993).
    [Crossref] [PubMed]
  5. Applied Opticsspecial issue on optical implementations of neural networks, Appl. Opt. 32(8) (1993).
    [PubMed]
  6. C. S. Weaver, J. W. Goodman, “A technique for optically convolving two functions,” Appl. Opt. 5, 1248–1249 (1966).
    [Crossref] [PubMed]
  7. B. Javidi, “Nonlinear joint power spectrum based optical correlation,” Appl. Opt. 28, 2358–2367 (1989).
    [Crossref] [PubMed]
  8. B. Javidi, Q. Tang, D. A. Gregory, T. D. Hudson, “Experiments on nonlinear joint transform correlator using an optically addressed spatial light modulator in the Fourier plane,” Appl. Opt. 30, 1772–1776 (1991).
    [Crossref] [PubMed]
  9. K. H. Fielding, J. L. Horner, “1-f binary joint correlator,” Opt. Eng. 29, 1081–1087 (1990).
    [Crossref]
  10. Q. Tang, B. Javidi, “Sensitivity of the nonlinear joint transform correlator: experimental investigations,” Appl. Opt. 31, 4016–4024 (1992).
    [Crossref] [PubMed]
  11. B. Javidi, J. L. Horner, “Single spatial light modulator joint transform correlator,” Appl. Opt. 28, 1027–1032 (1989).
    [Crossref] [PubMed]
  12. B. Javidi, “Nonlinear joint transform correlators,” in Real-Time Optical Information Processing, B. Javidi, J. L. Horner, eds. (Academic Press, San Diego, 1994), pp. 115–183.
  13. W. B. Hahn, D. L. Flannery, “Design elements of binary joint transform correlation and selected optimization techniques,” Opt. Eng. 31, 896–905 (1992).
    [Crossref]
  14. Ph. Réfrégier, V. Laude, B. Javidi, “Nonlinear joint transform correlation: an optimal solution for adaptive image discrimination and input noise robustness,” Opt. Lett. 19, 405–407 (1994).
    [PubMed]
  15. W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

1994 (1)

1993 (3)

1992 (2)

Q. Tang, B. Javidi, “Sensitivity of the nonlinear joint transform correlator: experimental investigations,” Appl. Opt. 31, 4016–4024 (1992).
[Crossref] [PubMed]

W. B. Hahn, D. L. Flannery, “Design elements of binary joint transform correlation and selected optimization techniques,” Opt. Eng. 31, 896–905 (1992).
[Crossref]

1991 (1)

1990 (2)

1989 (2)

1988 (1)

1978 (1)

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

1966 (1)

Bleha, W. P.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Brady, D.

Brown, H. B.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Campbell, S.

Casasent, D.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Fielding, K. H.

K. H. Fielding, J. L. Horner, “1-f binary joint correlator,” Opt. Eng. 29, 1081–1087 (1990).
[Crossref]

Flannery, D. L.

W. B. Hahn, D. L. Flannery, “Design elements of binary joint transform correlation and selected optimization techniques,” Opt. Eng. 31, 896–905 (1992).
[Crossref]

Goodman, J. W.

Gregory, D. A.

Grinberg, J.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Hahn, W. B.

W. B. Hahn, D. L. Flannery, “Design elements of binary joint transform correlation and selected optimization techniques,” Opt. Eng. 31, 896–905 (1992).
[Crossref]

Hong, J. H.

Horner, J. L.

Hudson, T. D.

Javidi, B.

Laude, V.

Li, H.-Y.

Lipton, L. T.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Markevitch, B. V.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

McAulay, A. D.

Psaltis, D.

Qiao, Y.

Réfrégier, Ph.

Reif, P. G.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Tang, Q.

Wagner, K.

Wang, J.

Weaver, C. S.

Wiener-Avnear, E.

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

Xu, X.

Yeh, P.

Appl. Opt. (10)

H.-Y. Li, Y. Qiao, D. Psaltis, “Optical network for real-time face recognition,” Appl. Opt. 32, 5026–5035 (1993).
[Crossref] [PubMed]

J. H. Hong, S. Campbell, P. Yeh, “Optical pattern classifier with perceptron learning,” Appl. Opt. 29, 3019–3025 (1990).
[Crossref] [PubMed]

D. Psaltis, D. Brady, K. Wagner, “Adaptive optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
[Crossref]

A. D. McAulay, J. Wang, X. Xu, “Optical perceptron learning for binary classification with spatial light rebroadcasters,” Appl. Opt. 32, 1346–1353 (1993).
[Crossref] [PubMed]

Applied Opticsspecial issue on optical implementations of neural networks, Appl. Opt. 32(8) (1993).
[PubMed]

C. S. Weaver, J. W. Goodman, “A technique for optically convolving two functions,” Appl. Opt. 5, 1248–1249 (1966).
[Crossref] [PubMed]

B. Javidi, “Nonlinear joint power spectrum based optical correlation,” Appl. Opt. 28, 2358–2367 (1989).
[Crossref] [PubMed]

B. Javidi, Q. Tang, D. A. Gregory, T. D. Hudson, “Experiments on nonlinear joint transform correlator using an optically addressed spatial light modulator in the Fourier plane,” Appl. Opt. 30, 1772–1776 (1991).
[Crossref] [PubMed]

Q. Tang, B. Javidi, “Sensitivity of the nonlinear joint transform correlator: experimental investigations,” Appl. Opt. 31, 4016–4024 (1992).
[Crossref] [PubMed]

B. Javidi, J. L. Horner, “Single spatial light modulator joint transform correlator,” Appl. Opt. 28, 1027–1032 (1989).
[Crossref] [PubMed]

Opt. Eng. (3)

W. B. Hahn, D. L. Flannery, “Design elements of binary joint transform correlation and selected optimization techniques,” Opt. Eng. 31, 896–905 (1992).
[Crossref]

W. P. Bleha, L. T. Lipton, E. Wiener-Avnear, J. Grinberg, P. G. Reif, D. Casasent, H. B. Brown, B. V. Markevitch, “Application of the liquid crystal light valve to real-time optical data processing,” Opt. Eng. 17, 371–384 (1978).

K. H. Fielding, J. L. Horner, “1-f binary joint correlator,” Opt. Eng. 29, 1081–1087 (1990).
[Crossref]

Opt. Lett. (1)

Other (1)

B. Javidi, “Nonlinear joint transform correlators,” in Real-Time Optical Information Processing, B. Javidi, J. L. Horner, eds. (Academic Press, San Diego, 1994), pp. 115–183.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1
Fig. 1

Diagram of the two-layer neural network used in face recognition.

Fig. 2
Fig. 2

(a) Examples of various head perspectives used in the training process, (b) Selected training samples (right) and composite images (left column) used as the weight functions. Each composite image is constructed by the six images shown at right. These composite images are displayed at the input of the nonlinear JTC.

Fig. 3
Fig. 3

Correlations between eight composite images shown in Fig. 2(b) by the use of (a) a kth-law nonlinear JTC with k = 0.3, (b) a linear correlator.

Fig. 4
Fig. 4

Effects of Fourier-plane nonlinear transformation on the input image and the composite images.

Fig. 5
Fig. 5

Computer simulations for face recognition: (a) input plane of the system displaying input image of class 1 and eight composite images for class 1. The composite images are spatially multiplexed when they are displayed next to one another. (b) Output plane of the first layer showing the response to the input image of (a).

Fig. 6
Fig. 6

(a) Input plane of the system displaying input image of class 2 and composite images for class 1, (b) output plane of the first layer showing the response to the input image of (a).

Fig. 7
Fig. 7

(a) Examples of various distorted facial images of class 1 used as input images in testing the neural network. (b) Neural network response to facial images of class 1 and class 2 with different head perspectives and various distortions. Weights for class 1 are used in the tests. (c) Plot of error probability versus the output threshold level.

Fig. 8
Fig. 8

Use of time multiplexing of the input image to reduce the probability of error. Weights for class 1 are used in the tests: (a) system response to facial images of class 1 and class 2 with different head perspectives and various distortions, (b) plot of error probability versus the output threshold level.

Fig. 9
Fig. 9

(a) Examples of input images from five image classes. The neural network is programmed to recognize the leftmost image, which is from class 1. (b) System response to facial images of class 1 and other classes. (c) Plot of error probability versus the output threshold level.

Fig. 10
Fig. 10

Performance of a pattern-recognition system in which a correlator is used instead of a neural network system. Correlation tests are performed with a composite image obtained by averaging 48 input training images of class 1: (a) composite image, (b) correlator response to facial images of class 1 and class 2 with different head perspectives and various distortions when the composite image in (a) is used, (c) plot of correlator error probability versus the correlator output threshold level.

Fig. 11
Fig. 11

Optical implementation of a neutral network for face recognition.

Fig. 12
Fig. 12

Optical experimental results: (a) input plane of the system displaying the input image of person 1 and one of the composite images for class 1, (b) output plane of the first layer for the composite image and the input image of (a), (c) input plane of the system displaying the input image of person 2 and one of the composite images for class 1, (d) output plane of the first layer for the composite image and the input image of (c).

Fig. 13
Fig. 13

Optical neural network response to facial images of class 1 and class 2 with different head perspectives and distortions. Composite images for class 1 are used in the tests. (a) System responses to facial images of class 1, (b) system responses to facial images of class 2, (c) plot of error probability versus the second-layer output threshold level, (d) optical neural network response to facial images of class 1 and class 2 when time multiplexing of the input images is used, (e) error probability versus the output threshold level when time multiplexing of the input images is used.

Fig. 14
Fig. 14

Single SLM nonlinear JTC-based neural network for face recognition. The LCTV is used to display the input image and the composite images as well as the joint power spectrum. FTL, Fourier-transform lens.

Fig. 15
Fig. 15

Optical experimental results of the neural network when the single SLM nonlinear JTC architecture shown in Fig. 14 is used. In both cases, composite images for class 1 are used. (a) Neural network response to facial images of class 1, (b) neural network response to facial images of class 2, (c) plot of error probability versus the second-layer output threshold level, (d) optical neural network response to facial images of class 1 and class 2 when time multiplexing of the input images is used, (e) probability of error versus the second-layer output threshold level when time multiplexing of the input images is used.

Fig. 16
Fig. 16

Diagram of a compact device having the laser source, the detector array, and the Fourier-transform processing packaged together to implement the neural networks.

Equations (9)

Equations on this page are rendered with MathJax. Learn more.

y = f ( i = 0 N 1 w i x i θ ) ,
{ if y 0 then x C 1 if y < 0 then x C 2 ,
w ( t + 1 ) = w ( t ) + μ ( t ) x ( t ) ,
μ ( t ) = { 0 if y ( t ) 0 and x ( t ) C 1 or y ( t ) < 0 and x ( t ) C 2 1 if y ( t ) 0 and x ( t ) C 2 1 if y ( t ) < 0 and x ( t ) C 1 .
E ( α , β ) = X 2 ( α , β ) + W 2 ( α , β ) + 2 X ( α , β ) W ( α , β ) × cos [ 2 x 0 α + Φ X ( α , β ) Φ W ( α , β ) ] ,
g [ E ( α , β ) ] = ν = 0 H ν [ W ( α , β ) , X ( α , β ) ] × cos [ 2 ν x 0 α + ν Φ X ( α , β ) ν Φ W ( α , β ) ] ,
H ν [ W ( α , β ) , X ( α , β ) ] = ν 2 π ( i ) ν G ( ω ) exp { i ω [ W 2 ( α , β ) + X 2 ( α , β ) ] } × J ν [ 2 ω W ( α , β ) X ( α , β ) ] d ω .
ν = { 1 , ν = 0 2 , ν = 1 .
g 1 k ( E ) = C k [ W ( α , β ) X ( α , β ) ] k × cos [ 2 x 0 α + Φ X ( α , β ) Φ W ( α , β ) ] ,

Metrics