Abstract

We describe two optical systems based on the radial basis function approach to pattern classification. An optical-disk-based system for handwritten character recognition is demonstrated. The optical system computes the Euclidean distance between an unknown input and 650 stored patterns at a demonstrated rate of 26,000 pattern comparisons/s. The ultimate performance of this system is limited by optical-disk resolution to 1011 binary operations/s. An adaptive system is also presented that facilitates on-line learning and provides additional robustness.

© 1993 Optical Society of America

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. D. Psaltis, M. A. Neifeld, A. A. Yamamura, “Image correlators using optical memory disks,” Opt. Lett. 14, 429–431 (1989).
    [CrossRef] [PubMed]
  2. D. Psaltis, M. A. Neifeld, A. A. Yamamura, S. Kobayashi, “Optical memory disks in optical information processing,” Appl. Opt. 29, 2038–2057 (1990).
    [CrossRef] [PubMed]
  3. J. Moody, C. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computat. 1, 281–294 (1989).
    [CrossRef]
  4. D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.
  5. T. M. Cover, P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory IT-13, 21–27 (1967).
    [CrossRef]
  6. T. Poggio, F. Girosi, “A theory of networks for approximation and learning,” AI Memo No. 1140 (MIT Artificial Intelligence Laboratory, Cambridge, Mass., 1989).
  7. T. Poggio, F. Girosi, “Networks for approximation and learning,” Proc. IEEE 78, 1481–1495 (1990).
    [CrossRef]
  8. J. MacQueen, “Some methods for classification and analysis of multivariate observations,” in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, L. M. LeCam, J. Neyman, eds. (U. California Press, Berkeley, Calif., 1967), Vol. 1, pp. 281–297.
  9. R. Duda, P. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973), Chap. 5, pp. 141–147.
  10. M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).
  11. D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
    [CrossRef]
  12. T. J. Drabik, M. A. Handschy, “Silicon VLSI ferroelectric liquid-crystal technology for micropower optoelectronic computing devices,” Appl. Opt. 29, 5220–5223 (1990).
    [CrossRef] [PubMed]
  13. D. Psaltis, D. J. Brady, K. Wagner, “Adpative optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
    [CrossRef]
  14. M. A. Neifeld, S. Rakshit, D. Psaltis, “Handwritten zip code recognition using an optical radial basis function classifier,” in Applications of Artificial Neural Networks II, S. K. Rogers, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1469, 250–255 (1991).

1990 (4)

T. Poggio, F. Girosi, “Networks for approximation and learning,” Proc. IEEE 78, 1481–1495 (1990).
[CrossRef]

D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
[CrossRef]

T. J. Drabik, M. A. Handschy, “Silicon VLSI ferroelectric liquid-crystal technology for micropower optoelectronic computing devices,” Appl. Opt. 29, 5220–5223 (1990).
[CrossRef] [PubMed]

D. Psaltis, M. A. Neifeld, A. A. Yamamura, S. Kobayashi, “Optical memory disks in optical information processing,” Appl. Opt. 29, 2038–2057 (1990).
[CrossRef] [PubMed]

1989 (2)

J. Moody, C. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computat. 1, 281–294 (1989).
[CrossRef]

D. Psaltis, M. A. Neifeld, A. A. Yamamura, “Image correlators using optical memory disks,” Opt. Lett. 14, 429–431 (1989).
[CrossRef] [PubMed]

1988 (1)

1967 (1)

T. M. Cover, P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory IT-13, 21–27 (1967).
[CrossRef]

Brady, D. J.

Cover, T. M.

T. M. Cover, P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory IT-13, 21–27 (1967).
[CrossRef]

Darken, C.

J. Moody, C. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computat. 1, 281–294 (1989).
[CrossRef]

Drabik, T. J.

Duda, R.

R. Duda, P. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973), Chap. 5, pp. 141–147.

Girosi, F.

T. Poggio, F. Girosi, “Networks for approximation and learning,” Proc. IEEE 78, 1481–1495 (1990).
[CrossRef]

T. Poggio, F. Girosi, “A theory of networks for approximation and learning,” AI Memo No. 1140 (MIT Artificial Intelligence Laboratory, Cambridge, Mass., 1989).

Handschy, M. A.

Hart, P.

R. Duda, P. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973), Chap. 5, pp. 141–147.

Hart, P. E.

T. M. Cover, P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory IT-13, 21–27 (1967).
[CrossRef]

Hinton, G. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Jared, D.

D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
[CrossRef]

Johnson, K. M.

D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
[CrossRef]

Kobayashi, S.

MacQueen, J.

J. MacQueen, “Some methods for classification and analysis of multivariate observations,” in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, L. M. LeCam, J. Neyman, eds. (U. California Press, Berkeley, Calif., 1967), Vol. 1, pp. 281–297.

Moddel, G.

D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
[CrossRef]

Moody, J.

J. Moody, C. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computat. 1, 281–294 (1989).
[CrossRef]

Neifeld, M. A.

D. Psaltis, M. A. Neifeld, A. A. Yamamura, S. Kobayashi, “Optical memory disks in optical information processing,” Appl. Opt. 29, 2038–2057 (1990).
[CrossRef] [PubMed]

D. Psaltis, M. A. Neifeld, A. A. Yamamura, “Image correlators using optical memory disks,” Opt. Lett. 14, 429–431 (1989).
[CrossRef] [PubMed]

M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).

M. A. Neifeld, S. Rakshit, D. Psaltis, “Handwritten zip code recognition using an optical radial basis function classifier,” in Applications of Artificial Neural Networks II, S. K. Rogers, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1469, 250–255 (1991).

Poggio, T.

T. Poggio, F. Girosi, “Networks for approximation and learning,” Proc. IEEE 78, 1481–1495 (1990).
[CrossRef]

T. Poggio, F. Girosi, “A theory of networks for approximation and learning,” AI Memo No. 1140 (MIT Artificial Intelligence Laboratory, Cambridge, Mass., 1989).

Psaltis, D.

D. Psaltis, M. A. Neifeld, A. A. Yamamura, S. Kobayashi, “Optical memory disks in optical information processing,” Appl. Opt. 29, 2038–2057 (1990).
[CrossRef] [PubMed]

D. Psaltis, M. A. Neifeld, A. A. Yamamura, “Image correlators using optical memory disks,” Opt. Lett. 14, 429–431 (1989).
[CrossRef] [PubMed]

D. Psaltis, D. J. Brady, K. Wagner, “Adpative optical networks using photorefractive crystals,” Appl. Opt. 27, 1752–1759 (1988).
[CrossRef]

M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).

M. A. Neifeld, S. Rakshit, D. Psaltis, “Handwritten zip code recognition using an optical radial basis function classifier,” in Applications of Artificial Neural Networks II, S. K. Rogers, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1469, 250–255 (1991).

Rakshit, S.

M. A. Neifeld, S. Rakshit, D. Psaltis, “Handwritten zip code recognition using an optical radial basis function classifier,” in Applications of Artificial Neural Networks II, S. K. Rogers, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1469, 250–255 (1991).

M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).

Rumelhart, D. E.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Wagner, K.

Williams, R. J.

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

Yamamura, A. A.

D. Psaltis, M. A. Neifeld, A. A. Yamamura, S. Kobayashi, “Optical memory disks in optical information processing,” Appl. Opt. 29, 2038–2057 (1990).
[CrossRef] [PubMed]

D. Psaltis, M. A. Neifeld, A. A. Yamamura, “Image correlators using optical memory disks,” Opt. Lett. 14, 429–431 (1989).
[CrossRef] [PubMed]

M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).

Appl. Opt. (3)

IEEE Trans. Inf. Theory (1)

T. M. Cover, P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory IT-13, 21–27 (1967).
[CrossRef]

Neural Computat. (1)

J. Moody, C. Darken, “Fast learning in networks of locally tuned processing units,” Neural Computat. 1, 281–294 (1989).
[CrossRef]

Opt. Commun. (1)

D. Jared, K. M. Johnson, G. Moddel, “Joint transform correlator using an amorphous-silicon ferroelectric liquid-crystal spatial light modulator,” Opt. Commun. 76, 97–102 (1990).
[CrossRef]

Opt. Lett. (1)

Proc. IEEE (1)

T. Poggio, F. Girosi, “Networks for approximation and learning,” Proc. IEEE 78, 1481–1495 (1990).
[CrossRef]

Other (6)

J. MacQueen, “Some methods for classification and analysis of multivariate observations,” in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, L. M. LeCam, J. Neyman, eds. (U. California Press, Berkeley, Calif., 1967), Vol. 1, pp. 281–297.

R. Duda, P. Hart, Pattern Classification and Scene Analysis (Wiley, New York, 1973), Chap. 5, pp. 141–147.

M. A. Neifeld, S. Rakshit, A. A. Yamamura, D. Psaltis, “Optical disk implementation of radial basis classifiers,” in Optical Information Processing Systems and Architectures, II, B. Javidi, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1347, 4–15 (1990).

D. E. Rumelhart, G. E. Hinton, R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland, eds. (MIT, Cambridge, Mass., 1986), Vol. 1, pp. 318–362.

T. Poggio, F. Girosi, “A theory of networks for approximation and learning,” AI Memo No. 1140 (MIT Artificial Intelligence Laboratory, Cambridge, Mass., 1989).

M. A. Neifeld, S. Rakshit, D. Psaltis, “Handwritten zip code recognition using an optical radial basis function classifier,” in Applications of Artificial Neural Networks II, S. K. Rogers, ed., Proc. Soc. Photo-Opt. Instrum. Eng.1469, 250–255 (1991).

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1

(a) Definition and schematic of RBF units (top) and linear units (bottom). (b) General RBF network.

Fig. 2
Fig. 2

(a) RBF network for estimating a scalar function of two variables. (b) Example input space configuration resulting from the network shown in (a).

Fig. 3
Fig. 3

Examples of handwritten numerals from the training set (top) and the testing set (bottom) used in the optical RBF experiment.

Fig. 4
Fig. 4

RBF network for handwritten digit recognition.

Fig. 5
Fig. 5

RBF widths computed using the one-nearest-neighbor rule with σ ˜ = 1 . 2.

Fig. 6
Fig. 6

Second-layer weights computed with the perceptron algorithm after initialization with the binary address algorithm. The weights for neurons 1–10 appear consecutively from left to right, top to bottom.

Fig. 7
Fig. 7

Optical system used to compute the distance between an input and an array of stored templates.

Fig. 8
Fig. 8

Example of raw detector output indicating the optically computed inner products.

Fig. 9
Fig. 9

Predicted recognition rate versus illumination profile width.

Fig. 10
Fig. 10

(a) Experimental and (b) actual distance versus template number for a single input image (handwritten 3, number three). Template numbers 195–260 represent the class of handwritten 3’s.

Fig. 11
Fig. 11

Parallel optical distance computer.

Fig. 12
Fig. 12

Optoelectronic postprocessing chip.

Fig. 13
Fig. 13

On-line learning postprocessing module.

Tables (2)

Tables Icon

Table 1 Classification Results Obtained with a One-Nearest Neighbor Rule for Training the Radial-Basis-Fuction Widths

Tables Icon

Table 2 Performance Comparison between Optical RBF Classifier and Simulationa

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

f ˆ ( w , x ) = i = 1 M ˜ a i exp ( | x t i | 2 / σ i 2 ) ,
y i = exp ( | x t i | 2 / σ i 2 ) ,
σ i = σ ˜ min j i | t j t i | ,
w i j = { 1 if t j Ω i 1 otherwise .
| x t i | 2 = | x | 2 + | t i | 2 2 x · t i ,
| x | 2 = ( x · 1 ) [ 1 = ( 1 , 1 , , 1 ) ] = x · ( t i + t i ¯ ) ,
| x t i | 2 = | t i | 2 + x · t i ¯ x · t i .
Δ D rms = [ 1 M i = 1 M ( d i Opt d i sim ) 2 ] 1 / 2 1 M i = 1 M d i sim ,
d i = | x t i | 2 = j = 1 N ( x j t j i ) 2 = j = 1 N d j i .
d j i ¯ = x j t j i + x j ¯ t j i ¯ .
E = i = 1 M [ f ˆ ( w , x i ) f ( x i ) ] 2 = i = 1 M E i 2 ,
Δ ( 1 / σ p ) 2 = α σ E l | x l t p | 2 a p exp ( | x l t p | 2 / σ p 2 ) ,
Δ ( a p ) = α a E l exp ( | x l t p | 2 / σ p 2 ) ,

Metrics