Abstract

This paper presents the Static Computational Optical Undersampled Tracker (SCOUT), an architecture for compressive motion tracking systems. The architecture uses compressive sensing techniques to track moving targets at significantly higher resolution than the detector array, allowing for low cost, low weight design and a significant reduction in data storage and bandwidth requirements. Using two amplitude masks and a standard focal plane array, the system captures many projections simultaneously, avoiding the need for time-sequential measurements of a single scene. Scenes with few moving targets on static backgrounds have frame differences that can be reconstructed using sparse signal reconstruction techniques in order to track moving targets. Simulations demonstrate theoretical performance and help to inform the choice of design parameters. We use the coherence parameter of the system matrix as an efficient predictor of reconstruction error to avoid performing computationally intensive reconstructions over the entire design space. An experimental SCOUT system demonstrates excellent reconstruction performance with 16X compression tracking movers on scenes with zero and nonzero backgrounds.

© 2012 OSA

Full Article  |  PDF Article

References

  • View by:
  • |
  • |
  • |

  1. M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.
  2. M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
    [CrossRef] [PubMed]
  3. R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
    [CrossRef]
  4. M. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt.46, 5293, (2007).
    [CrossRef] [PubMed]
  5. M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.
  6. Y. Rivenson, A. Stern, and B. Javidi, “Single exposure super-resolution compressive imaging by double phase encoding,” Opt. Express18, 15094–15103 (2010).
    [CrossRef] [PubMed]
  7. Y. Kashter, O. Levi, and A. Stern, “Optical compressive change and motion detection,” Appl. Opt.51, 2491–2496 (2012).
    [CrossRef] [PubMed]
  8. W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
    [CrossRef]
  9. H. Rauhut, “Circulant and Toeplitz matrices in compressed sensing,” http://arxiv.org/abs/0902.4394 .
  10. J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci.2, 1098–1128 (2009).
    [CrossRef]
  11. F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50.
    [CrossRef]
  12. B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).
  13. S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
    [CrossRef]
  14. J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory52, 1030–1051 (2006).
    [CrossRef]

2012 (1)

2011 (1)

R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
[CrossRef]

2010 (1)

2009 (1)

J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci.2, 1098–1128 (2009).
[CrossRef]

2008 (1)

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

2007 (3)

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
[CrossRef] [PubMed]

M. Neifeld and J. Ke, “Optical architectures for compressive imaging,” Appl. Opt.46, 5293, (2007).
[CrossRef] [PubMed]

2006 (1)

J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory52, 1030–1051 (2006).
[CrossRef]

Bajwa, W.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Baraniuk, R.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Baron, D.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Boyd, S.

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

Donoho, D.

M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
[CrossRef] [PubMed]

Duarte, M.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Gehm, M.

M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.

Gorinevsky, D.

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

Haupt, J.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Javidi, B.

Kashter, Y.

Ke, J.

Kelly, K.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Kim, S.

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

Koh, K.

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

Laska, J.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Levi, O.

Liu, B.

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

Lustig, M.

M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
[CrossRef] [PubMed]

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

Marcia, R.

R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
[CrossRef]

Neifeld, M.

Nichols, J.

R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
[CrossRef]

Nowak, R.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Pauly, J.

M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
[CrossRef] [PubMed]

Raz, G.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Rivenson, Y.

Romberg, J.

J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci.2, 1098–1128 (2009).
[CrossRef]

Sarvotham, S.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Sebert, F.

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50.
[CrossRef]

Stenner, M.

M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.

Stern, A.

Takhar, D.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Townsend, D.

M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.

Tropp, J.

J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory52, 1030–1051 (2006).
[CrossRef]

Wakin, M.

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

Willett, R.

R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
[CrossRef]

Wright, S.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

Ying, L.

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50.
[CrossRef]

Zou, Y.

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50.
[CrossRef]

Appl. Opt. (2)

IEEE J. Sel. Top. Sig. Proc. (1)

S. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, “An interior-point method for large-scale l1-regularized least squares,” IEEE J. Sel. Top. Sig. Proc.1, 606–617 (2007).
[CrossRef]

IEEE Trans. Inf. Theory (1)

J. Tropp, “Just relax: Convex programming methods for identifying sparse signals in noise,” IEEE Trans. Inf. Theory52, 1030–1051 (2006).
[CrossRef]

Magn. Reson. Med. (1)

M. Lustig, D. Donoho, and J. Pauly, “Sparse MRI: The application of compressed sensing for rapid MR imaging,” Magn. Reson. Med.58, 1182–1195 (2007).
[CrossRef] [PubMed]

Opt. Eng. (1)

R. Willett, R. Marcia, and J. Nichols, “Compressed sensing for practical optical imaging systems: a tutorial,” Opt. Eng.50, 072601 (2011).
[CrossRef]

Opt. Express (1)

Proceedings of the 16th Annual Meeting of ISMRM (1)

B. Liu, F. Sebert, Y. Zou, and L. Ying, “SparseSENSE: randomly-sampled parallel imaging using compressed sensing,” in Proceedings of the 16th Annual Meeting of ISMRM3154 (2008).

SIAM J. Imaging Sci. (1)

J. Romberg, “Compressive sensing by random convolution,” SIAM J. Imaging Sci.2, 1098–1128 (2009).
[CrossRef]

Other (5)

F. Sebert, Y. Zou, and L. Ying, “Toeplitz block matrices in compressed sensing and their applications in imaging,” in Proceedings of IEEE International Conference on Information Technology and Applications in Biomedicine, (IEEE, 2008), pp. 47–50.
[CrossRef]

M. Wakin, J. Laska, M. Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, and R. Baraniuk, “An architecture for compressive imaging,” in Proceedings of IEEE Intl. Conference on Image Processing, (IEEE, 2006), pp. 1273–1276.

W. Bajwa, J. Haupt, G. Raz, S. Wright, and R. Nowak, “Toeplitz-structured compressed sensing matrices,” in Proceedings of IEEE Workshop on Statistical Signal Processing, (IEEE, 2007), pp. 294–298.
[CrossRef]

H. Rauhut, “Circulant and Toeplitz matrices in compressed sensing,” http://arxiv.org/abs/0902.4394 .

M. Stenner, D. Townsend, and M. Gehm, “Static architecture for compressive motion detection in persistent, pervasive surveillance applications,” in Imaging Systems, OSA Technical Digest Series (Optical Society of America, 2010), paper IMB2.

Supplementary Material (2)

» Media 1: AVI (441 KB)     
» Media 2: AVI (581 KB)     

Cited By

OSA participates in CrossRef's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (12)

Fig. 1
Fig. 1

A canonical CS imaging architecture. An object is imaged onto micromirror pattern, which represents a projection vector, then condensed onto a photodetector for a single measurement. Each set of projections, created by different mirror configurations, is captured sequentially in time.

Fig. 2
Fig. 2

An example of a typical parallel optical CS architecture. Capturing M simultaneous projections requires using M spatial light modulators or masks and M detector elements.

Fig. 3
Fig. 3

A matrix formulation describing the SCOUT system.

Fig. 4
Fig. 4

Experimentally measured system response matrix for the SCOUT system described in Sec. 5. The approximate block-Toeplitz structure is clearly evident, as is the deviation from the Bernoulli or Gaussian ensembles typically considered in CS treatments.

Fig. 5
Fig. 5

A diagram of the SCOUT architecture. A defocused lens projects light through a pair of binary occlusion masks onto a low-resolution sensor to capture compressive, multiplexed measurements with a shift-variant PSF.

Fig. 6
Fig. 6

The coherence μ (left vertical axis - solid blue) and reconstruction error P (right vertical axis - dashed green) plotted as a function of defocus distance dim.

Fig. 7
Fig. 7

The coherence μ (left vertical axis - solid blue) and the reconstruction error P (right vertical axis - dashed green) is plotted as a function of the pitch of mask 2.

Fig. 8
Fig. 8

The reconstruction error, P, for three different choices of dim, p1, and p2 are plotted versus number of movers. The performance obtained from a set of parameters that led to a minimum μ in our parameter space is shown in dashed red. The solid blue line shows performance of parameters chosen individually using actual reconstruction performance. A set of parameters that leads to a maximum μ is shown in dotted green. The error bars on each line plot represent the standard deviation of the mean.

Fig. 9
Fig. 9

Photographs of a SCOUT implementation. (a) An optical tube contains mask m2 and the lens; mask m1 (not shown) is attached near the sensor. (b) The camera captures images of scenes displayed on a plasma television approximately 2 meters away.

Fig. 10
Fig. 10

( Media 1) Frame 1 of a reconstruction video of a 32 × 32 scene with two movers of equal amplitude on a black background. (a) ground-truth scene 1 (b) ground-truth scene 2 (c) ground-truth frame difference and (d) measured 8 × 8 frame difference, scaled so that it is discernible (e) reconstructed 32 × 32 difference frame

Fig. 11
Fig. 11

( Media 1) Frame 9 of a reconstruction video of a 32 × 32 scene with two movers of equal amplitude on a black background. This frame shows the results when the past and present mover locations are adjacent in the difference frame. (a) ground-truth scene 1 (b) ground-truth scene 2 (c) ground-truth frame difference and (d) measured 8 × 8 frame difference, scaled so that it is discernible (e) reconstructed 32 × 32 difference frame

Fig. 12
Fig. 12

( Media 2) Frame 1 of a video demonstration of compressive tracking of a 32 × 32 difference scene with a nonzero background. Scene background ©2012 Google. (a) ground-truth scene 1 (b) ground-truth scene 2 (c) ground-truth frame difference and (d) measured 8 × 8 frame difference, scaled so that it is discernible (e) reconstructed 32 × 32 difference frame

Equations (5)

Equations on this page are rendered with MathJax. Learn more.

g = Hf ,
P = | a e | 2 n where a = [ 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 1 / 9 ] and e = Δ f Δ f ^
μ = max | ϕ i , ϕ j | ; i j
H normed = H j = 1 M i = 1 N h i , j
H recon = t exp t cal H cal ,

Metrics