## Abstract

We propose and experimentally realize the concept of frequency-division-multiplexed single-pixel imaging. Our technique relies entirely on metamaterial spatial light modulators, the advent of which has permitted advanced modulation techniques difficult to achieve with alternative approaches. So far, implementations of single-pixel imaging have used a single encoding frequency, making them sensitive to narrowband noise. Here, we implement frequency-division methods to parallelize the single-pixel imaging process at 3.2 THz. Our technique enables a trade-off between signal-to-noise ratio and acquisition speed—without altering detector integration time—thus realizing a key development due to the limitations imposed by slow thermal detectors in terahertz and far IR. In addition, our technique yields high image fidelity and marries communications concepts to single-pixel imaging, opening a new path forward for future imaging systems.

© 2016 Optical Society of America

## 1. INTRODUCTION

Imaging with a single pixel was first shown in 1976, when a hyperspectral imager was fashioned using a mechanically scanned coded aperture wheel [1]. A spatial resolution of 1023 pixels with 63 wavelength values was realized in the thermal IR across a band of 8–14 μm. There were even earlier predictions of the possibility of single-pixel imaging, but it was understood to be a computational technique that required the development of efficient computers [2]. Within the last several years, the development of an innovative new method—termed compressive sensing (or compressive imaging)—has utilized digital mirror devices to sample signals, attracting significant attention as a method for imaging in the optical range [3–5]. Since the first demonstration of compressive imaging with a single-pixel system, there have been significant efforts to extend this technique to lower frequencies. However, due to the lack of commercially available spatial light modulators (SLMs) operating at longer wavelengths, many researchers have once again reverted to utilization of mechanically scanned apertures [6–8].

More recently, demonstration of single-pixel imaging in the long-wavelength regime was enabled through the advent of efficient, all-electronic metamaterial SLMs [9]. Detector arrays in this region of the electromagnetic spectrum are often costly and require high-powered sources, making single-pixel imaging schemes an obvious alternative. Active metamaterial-based SLMs permit multiplexing of the image, thus yielding the full signal-to-noise ratio (SNR) increase of Fellgett’s advantage, i.e., an increase proportional to the number of samples [10]. Additionally, metamaterial SLMs are capable of implementing the $[+1,-1]$ orthogonal mask sets shown to be ideal for single-pixel imaging systems [11]. Metamaterials themselves possess the advantage of spectral scalability and can be dynamically controlled with external stimuli, making them ideal candidates for the demonstration of more advanced light modulation techniques of potential use for imaging and communications [9,11–16].

The metamaterial SLM utilized in our imaging experiments uses a periodic time-varying external voltage applied to a semiconductor layer (GaAs), thus modulating the absorption of each pixel independently. Figure 1(c) illustrates a single-pixel imaging system in which light from an object is imaged onto the SLM. In previous systems, incident light was modulated at a single frequency ${f}_{\text{mod}}$, thereby encoding a single mask pattern in the phase of the beam intensity [11]. Here, however, we utilize a metamaterial SLM to simultaneously modulate reflected light at some number $p$ of different modulation frequencies, thereby realizing many masks at the same time and a consequent $p$-fold decrease in acquisition time.

## 2. EXPERIMENTAL SETUP

#### A. Model

We describe the imaging process by considering a pixelated version of a two-dimensional scene lexicographically arranged into a column vector $\mathit{X}$. The set of masks displayed serially on the SLM can be described as a matrix $\mathrm{\Phi}$, known as the measurement matrix, in which each column of $\mathrm{\Phi}$ corresponds to a single pixel in the scene and on the SLM, and each row represents a single mask pattern. The collection of measurements, one for each mask pattern, can then be described by matrix multiplication between the two [17]:

We require that the inverse problem be well-posed in the Hadamard sense, i.e., a unique solution exists and varies continuously with the initial conditions, in our case the object vector [18]. In terms of $\mathrm{\Phi}$, this means the rows must be linearly independent and also equal in number to the image pixels. We can then reconstruct the image via a simple matrix inversion:

Typical encoding matrices include the identity matrix (equivalent to raster scan), $S$ matrices, and random matrices with entries of [0,1] or $[1,-1]$. In particular, we have chosen the Hadamard matrix on account of its A-optimal and D-optimal properties, i.e., it minimizes the average mean square error of our measurements and maximizes the measurement matrix determinant [19]. Although usually applied in the context of spectroscopy, these optimal properties are desirable in a more general class of problem known as the weighing problem [19]. Additionally, they have been shown relevant even with the introduction of compressive techniques [11].#### B. Encoding

The mask values are encoded in the phase of intensity modulation, and thus we require phase-sensitive detection, in this case provided by a lock-in amplifier (LIA). Here we achieve true $[1,-1]$ pixel values rather than the classic [0,1] values associated with most SLMs. This distinction is key to our use of the Hadamard matrix, as opposed to an $S$-matrix, for example. In our present context these $[1,-1]$ values correspond to intensity modulations with distinct phases of [0, $\pi $]. This mapping of two phases to two numerical (bit) values is known in communications as binary phase shift keying (BPSK). Each pixel of the SLM is modulated with either 0 or $\pi $ phase on $p$ frequencies simultaneously, according to the present mask patterns. The modulated light from the SLM is then multiplexed into a single-pixel detector, which produces a signal containing the phase and modulation frequency information. The signal is then demodulated by a number ($p$) of LIAs that compare the total signal to reference waveforms. Each produces a DC amplitude ${V}_{0}$ and phase $\theta $, corresponding to one of the $p$ modulation frequencies. Thus each LIA yields one element of the measurement vector $Y$. For $Y$ of length $N$, this process is repeated $N/p$ times, in order to acquire all elements of the measurement vector.

#### C. Frequency Selection

The choice of modulation frequencies is critical and has a long history in the communications world. Here we choose frequencies such that all carriers are mutually orthogonal, as defined by

where $\tau $ is the integration time of the detector, called the symbol duration, and $g(t)$ gives the waveform of the carrier, in our case a square wave. The inner product in Eq. (3) represents the interference between two subcarriers of a channel, and thus the virtue of orthogonal frequencies is minimization of cross talk between carriers. Reducing this interference allows carriers to overlap in the frequency domain without sacrificing signal integrity, thus greatly increasing spectral efficiency. Orthogonality can also be represented in terms of beat frequencies, which complete an integral number of cycles in a single symbol period, therefore ideally averaging to zero. In the context of communications engineering, this concept is referred to as orthogonal frequency-division multiplexing (OFDM), and the carriers are typically transmitted as sinusoids [20], making it trivial to choose frequencies that fulfill the orthogonality condition of Eq. (3). In our case, however, the carriers are not sinusoids and do not undergo the digital-to-analog conversion associated with classic OFDM, instead remaining square waves throughout our system. Figure 1(a) illustrates the chosen carriers of ${f}_{1}=204\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{Hz}$, ${f}_{2}=208\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{Hz}$, ${f}_{3}=213\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{Hz}$, and ${f}_{4}=216\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{Hz}$ in the frequency domain, while Fig. 1(b) shows a representation in the time domain; it can be verified that these square waves are indeed orthogonal for our symbol duration of $\tau =100\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{ms}$.## 3. EXPERIMENTAL RESULTS

#### A. State Characterization

In order to quantify the capability of our metamaterial to perform frequency diverse modulation, we first characterize the states of our device by modulating all SLM pixels with BPSK, for different numbers of carrier frequencies ranging from 1 to 4, i.e., $\{{f}_{1}\},\{{f}_{1}+{f}_{2}\},\{{f}_{1}+{f}_{2}+{f}_{3}\}$, and $\{{f}_{1}+{f}_{2}+{f}_{3}+{f}_{4}\}$. The results are plotted in Fig. 2, where each individual plot, known as a constellation diagram, shows the phase states that are accessible for each frequency. In such diagrams, quantitative information is contained in the phase value of the data, while the radius is simply proportional to signal strength. The states shown are disjoint due to removal of the transition intervals between states, but these removed points represent only a small fraction of the total. In order to encode a mask using BPSK, the states must be distinct from each other; otherwise a loss in the SNR is incurred. As can be observed in Fig. 2(a), the states are indeed well separated, with standard and relative standard deviations (RSDs) of $\{{s}_{\theta}(\mathrm{rad}),\mathrm{RSD}(\%)\}=\{.01,.93\}$. The large dashed rings show the average radii for both states. Figure 2(b) illustrates the persistence of state separation even when multiple frequencies—four in this case—are modulated simultaneously. In Fig. 2(c) we show the time domain lock-in signal for different four-frequency combination states. The average SNR across all subcarriers remained stable for different aggregate states, with an RSD of 23%. Thus we can encode a Hadamard mask on each frequency without disrupting the channel via cross-carrier interference.

#### B. Imaging

Having characterized the pure BPSK states of our system, we now turn toward demonstration of frequency diverse single-pixel imaging. Figure 3 shows imaging results for an aperture shaped into the letter “D” (inset), reconstructed for $p=1\{{f}_{1}\},p=2\{{f}_{1},{f}_{4}\}$, and $p=4\{{f}_{1},{f}_{2},{f}_{3},{f}_{4}\}$ carrier frequencies. The solid red curve in Fig. 3 shows the normalized acquisition time as a function of the number of carriers, with the aforementioned fourfold increase in efficiency for the final case. We emphasize that this curve holds for any image size, i.e., the frequency diversity technique we have demonstrated is completely scalable.

To achieve the aggregate pure states shown in Fig. 2, we have necessarily made adjustments to the SLM voltage. This is related to the well-known peak power problem in telecommunications engineering, which arises from the fact that $N$ coherent sinusoids sum to produce a peak power $N$ times larger than the average power. Although there are many techniques to avoid doing so [21–24], input power can in principle be increased in such communications systems in order to achieve these peak powers, but the same is not true in our case. For our system there is some maximum modulation depth, a bound on the amplitude of the intensity modulation. Summing two square wave carriers must still yield a waveform of larger amplitude than either carrier alone, meaning that we must lower the voltage input to the SLM when only a single carrier is modulated.

To see the effect of frequency multiplexing on image fidelity, we perform an ${\ell}_{2}$-based comparison of the reconstructed images. As the images in Fig. 3 have been post-processed, we use the directly reconstructed $X$ vectors instead. Figure 4 shows the pixel-by-pixel ${\ell}_{2}$ distance between the multifrequency object vectors and the single-frequency vector, normalized by maximum pixel value. As is perhaps evident in Fig. 3, the four-frequency image is both the most unique and the least sparse.

In addition, we can more directly investigate the cost of implementing multiple subcarriers. Figure 5(a) shows time domain data of the lock-in signal for steady BPSK states. As more frequencies are added to the signal, the signal power decreases at a rate greater than that anticipated by a linear detection model. For the SLM and LIA, respectively, linear power addition and detection imply a $\frac{1}{n}$ proportionality between power and subcarrier number. This is not observed, suggesting a nonlinear effect, most likely at the SLM. Figure 5(b) shows the average voltage for each subcarrier number, with fit curves for both $\frac{1}{n}$ (linear power) and $\frac{1}{{2}^{n}}$ (nonlinear power). The $\frac{1}{n}$ curve is fit only to the first point, illustrating the expected power falloff for a linear model, while the $\frac{1}{{2}^{n}}$ curve represents the best fit for all four points. Evidently, the latter curve fits the observed loss of power much more accurately, and indeed SLMs of the type used here are characterized by lower modulation depth for lower average input power [15], i.e., a nonlinear relationship between input power and modulation power. This relationship can be seen as a result of the minimum reflectance increasing for lower bias voltages, which yields a lower modulation depth overall.

## 4. DISCUSSION

We have demonstrated a technique for frequency-division-multiplexed imaging that provides an increase in frame rate with—as mentioned before—a reduction in the SNR. The total power of our signal spectrum is conserved, which implies that increasing the number of subcarriers brings each closer to the system noise floor. In this way, our technique is a means of sacrificing some SNR in order to increase acquisition speed. We emphasize, however, that this trade-off is fundamental to any communications technique, as dictated by Shannon’s channel capacity [25]. For example, a straightforward means of increasing acquisition speed would be to utilize only a single modulation frequency and simply decrease the integration time of the detector. If we assume that the primary source of noise is additive white Gaussian (AWG), the SNR is proportional to the square root of the integration time [26]. Thus in order to achieve a factor $p$ of increase in acquisition speed, the detector integration time is reduced by the same factor $p$, resulting in a decrease in the SNR by a factor $\sqrt{p}$.

Although it appears that the two approaches produce similar results, it is important to note that detector integration time cannot be reduced without bound. All detectors have inherent limits—typically characterized by the 3 dB down point in the frequency-dependent responsivity (${f}_{3\text{\hspace{0.17em}}\mathrm{dB}}$). At frequencies greater than ${f}_{3\text{\hspace{0.17em}}\mathrm{dB}}$—or, equivalently, shorter integration times—the noise increases significantly and it is no longer advantageous to trade off SNR for integration time. Our FDM scheme, on the other hand, sacrifices some SNR for acquisition speed without lowering the integration time of each measurement, and thus does not suffer from the same fundamental limitation.

Our frequency-division multiplexing (FDM) approach to imaging may realize additional advantages if the system noise is not AWG. For example, multiplicative noise is common in scintillation or when source or detector fluctuation is present [27]. In astronomy [28] or microelectromechanical systems [29], for example, the SNR is not proportional to the square root of the integration time, and thus there may exist some characteristic time for optimal SNR, away from which results are degraded [30]. As FDM affords a decrease in acquisition time without deviation from such an optimal integration time, it should prove an effective technique in such cases.

The FDM imaging architecture presented here utilized BPSK, which itself is among the simplest of phase-based modulation techniques and requires only one axis of the constellation diagram. By including the second axis we comply with convention, but also mean to suggest that other modulation schemes are possible in our present context. Indeed, a second quadrature component in the phase space is both plausible and desirable, promising an additional factor of reduction in acquisition time [31]. Such a scheme would be completely compatible with frequency division, and capable of being realized on any number of orthogonal frequency channels. While here we have made use of a reference signal in order to detect phase, such a reference is not strictly necessary (as in differential phase shift keying), and hence our method remains relevant to applications where no reference is possible. Finally, we remark that given a fast enough modulator, the number of frequency channels in a similar system is limited only by bandwidth and noise. Notably, when the number of subcarriers equals or exceeds the number of pixels in the scene, the imaging process becomes “one-shot,” i.e., all masks are displayed simultaneously during one mask cycle and all measurements are taken in a single detector integration.

## 5. CONCLUSION

In conclusion, we have demonstrated the parallelization of single-pixel imaging, which relies on metamaterial SLMs. OFDM thus implemented can increase image acquisition speed at the cost of SNR, without altering detector integration time. Here we have employed the simplest of phase modulation schemes (BPSK), but more complex schemes are possible and promise further gains in acquisition speed. We expect the techniques realized here to be useful in number of IR, far-IR, and millimeter-wave applications such as all-weather navigation [32], security screening [33], and biosensing [34,35]. Single-pixel imaging has truly revolutionized imaging at long wavelengths, and FDM has the potential to overcome the weaknesses shared by many single-frequency single-pixel systems.

## 6. METHODS

Our imaging system [36] employs a 5500 K blackbody arc lamp as a terahertz source, the beam of which is initially collimated by a 90° off-axis parabolic mirror. After striking the object, the beam is focused onto the metamaterial SLM, where it is simultaneously encoded with multiple rows of the Hadamard matrix. Our SLM is controlled with a Spartan 6 field-programmable gate array (FPGA), which applies square-wave voltages at either 0 or $\pi $ phases at multiple modulation frequencies. The SLM itself is embedded in a 15 V transistor printed circuit board, which enables FPGA control. Due to a failure in this board, three pixels of the SLM do not modulate [row, column = (5,1), (4,7), and (4,8)]. Our SLM is characterized thoroughly elsewhere [11], but here by way of summary we report that it exhibits a peak absorption at approximately 3.2 THz with zero reverse-bias voltage, and 3.26 THz when fully reverse biased at 15 V. There is a resulting differential absorption peak at approximately 3.1 THz. Hadamard rows are encoded by the time-varying oscillation in pixel absorption. The light from these pixels is then focused (spatially multiplexed) into a bolometer, after which a set of LIAs (Stanford Research Systems SR830) processes the modulated light, producing a signal proportional to the integrated differential absorption.

## Funding

National Science Foundation (NSF) (ECCS-1002340); Office of Naval Research (N00014-11-1-0864).

## Acknowledgment

The authors extend their thanks to Jonathan Suen for many enlightening discussions.

## REFERENCES

**1. **M. J. E. Golay, “Static multislit spectrometry and its application to the panoramic display of infrared spectra,” J. Opt. Soc. Am. **41**, 468–472 (1951). [CrossRef]

**2. **M. J. E. Golay, “Multi-slit spectrometry,” J. Opt. Soc. Am. **39**, 437–444 (1949).

**3. **M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. **25**(2), 83–91 (2008). [CrossRef]

**4. **E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory **52**, 489–509 (2006). [CrossRef]

**5. **T. Strohmer, “Measure what should be measured: progress and challenges in compressive sensing,” IEEE Signal Process. Lett. **19**, 887–893 (2012). [CrossRef]

**6. **W. L. Chan, K. Charan, D. Takhar, K. F. Kelly, R. G. Baraniuk, and D. M. Mittleman, “A single-pixel terahertz imaging system based on compressed sensing,” Appl. Phys. Lett. **93**, 121105 (2008). [CrossRef]

**7. **K. Kawase, Y. Ogawa, Y. Watanabe, and H. Inoue, “Non-destructive terahertz imaging of illicit drugs using spectral fingerprints,” Opt. Express **11**, 2549–2554 (2003). [CrossRef]

**8. **N. Karpowicz, H. Zhong, C. Zhang, K. I. Lin, J. S. Hwang, J. Xu, and X. C. Zhang, “Compact continuous-wave subterahertz system for inspection applications,” Appl. Phys. Lett. **86**, 054105 (2005). [CrossRef]

**9. **D. Shrekenhamer, C. M. Watts, and W. J. Padilla, “Terahertz single pixel imaging with an optically controlled dynamic spatial light modulator,” Opt. Express **21**, 12507–12518 (2013). [CrossRef]

**10. **P. B. Fellgett, “On the ultimate sensitivity and practical performance of radiation detectors,” J. Opt. Soc. Am. **39**, 970–976 (1949). [CrossRef]

**11. **C. M. Watts, D. Shrekenhamer, J. Montoya, G. Lipworth, J. Hunt, T. Sleasman, S. Krishna, D. R. Smith, and W. J. Padilla, “Terahertz compressive imaging with metamaterial spatial light modulators,” Nat. Photonics **8**, 605–609 (2014). [CrossRef]

**12. **W. L. Chan, H. T. Chen, A. J. Taylor, I. Brener, M. J. Cich, and D. M. Mittleman, “A spatial light modulator for terahertz beams,” Appl. Phys. Lett. **94**, 3–5 (2009).

**13. **T. Hand and S. Cummer, “Characterization of tunable metamaterial elements using MEMS switches,” IEEE Antennas Wireless Propag. Lett. **6**, 401–404 (2007). [CrossRef]

**14. **H. T. Chen, W. J. Padilla, J. M. O. Zide, A. C. Gossard, A. J. Taylor, and R. D. Averitt, “Active terahertz metamaterial devices,” Nature **444**, 597–600 (2006).

**15. **D. Shrekenhamer, J. Montoya, S. Krishna, and W. J. Padilla, “Four-color metamaterial absorber THz spatial light modulator,” Adv. Opt. Mater. **1**, 905–909 (2013). [CrossRef]

**16. **H. T. Chen, W. J. Padilla, J. M. O. Zide, S. R. Bank, A. C. Gossard, A. J. Taylor, and R. D. Averitt, “Ultrafast optical switching of terahertz metamaterials fabricated on ErAs/GaAs nanoisland superlattices,” Opt. Lett. **32**, 1620–1622 (2007). [CrossRef]

**17. **M. Harwit and N. J. A. Sloane, *Hadamard Transform Optics* (Academic, 1979).

**18. **A. N. Tikhonov and V. Y. Arsenin, “Solutions of ill-posed problems,” Math. Comput. **32**, 1320–1322 (1978). [CrossRef]

**19. **N. J. A. Sloane, “Hadamard and other discrete transforms in spectroscopy,” in *Fourier, Hadamard, and Hilbert Transforms in Chemistry* (Plenum, 1982), pp. 45–67.

**20. **J. Proakis and M. Salehi, *Digital Communications* (McGraw-Hill, 2007), Vol. 6.

**21. **R. Prasad, *OFDM for Wireless Communications Systems* (Artech House, 2004).

**22. **J. A. Davis and J. Jedwab, “Peak-to-mean power control in OFDM, Golay complementary sequences and Reed-Muller codes,” IEEE Trans. Inform. Theory **45**, 2397–2417 (1998).

**23. **T. May and H. Rohling, “Reducing the peak-to-average power ratio in OFDM radio transmission systems,” in *48th IEEE Vehicular Technology Conference* (IEEE, 1998), pp. 2474–2478.

**24. **M. Pauli and H. P. Kuchenbecker, “Minimization of the intermodulation distortion of a nonlinearly amplified OFDM signal,” Wireless Personal Commun. **4**, 93–101 (1997). [CrossRef]

**25. **C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J. **27**, 379–423 (1948). [CrossRef]

**26. **S. P. P. Denny, J. Y. Y. Suen, and P. M. M. Lubin, “Fundamental limits of detection in the far infrared,” New Astron. **25**, 114–129 (2013). [CrossRef]

**27. **P. R. Griffiths and J. A. De Haseth, *Fourier Transform Infrared Spectrometry* (Wiley, 2007).

**28. **D. L. Fried, “Optical resolution through a randomly inhomogeneous medium for very long and very short exposures,” J. Opt. Soc. Am. **56**, 1372–1379 (1966). [CrossRef]

**29. **R. Ramalingam, G. Anitha, and J. Shanmugam, “Microelectromechnical systems inertial measurement unit error modelling and error analysis for low-cost strapdown inertial navigation system,” Defence Sci. J. **59**, 650–658 (2009).

**30. **J. Almodovar-Faria and J. McNair, “Optimal integration time for energy-detection PPM UWB systems,” in *IEEE Global Communications Conference (GLOBECOM)* (IEEE, 2012), pp. 4054–4059.

**31. **S. Haykin, *Communication Systems* (Wiley, 2001).

**32. **P. F. Goldsmith, C.-T. Hsieh, G. R. Huguenin, J. Kapitzky, and E. L. Moore, “Focal plane imaging systems for millimeter wavelengths,” IEEE Trans. Microwave Theory Tech. **41**, 1664–1675 (1993).

**33. **K. B. Cooper, R. J. Dengler, N. Llombart, B. Thomas, G. Chattopadhyay, and P. H. Siegel, “THz imaging radar for standoff personnel screening,” IEEE Trans. Terahertz Sci. Technol. **1**, 169–182 (2011). [CrossRef]

**34. **R. M. Woodward, V. P. Wallace, R. J. Pye, B. E. Cole, D. D. Arnone, E. H. Linfield, and M. Pepper, “Terahertz pulse imaging of ex vivo basal cell carcinoma,” J. Invest. Dermatol. **120**, 72–78 (2003). [CrossRef]

**35. **T. M. Korter and D. F. Plusquellic, “Continuous-wave terahertz spectroscopy of biotin: vibrational anharmonicity in the far-infrared,” Chem. Phys. Lett. **385**, 45–51 (2004). [CrossRef]

**36. **C. C. Nadell, C. M. Watts, J. A. Montoya, S. Krishna, and W. J. Padilla, “Single pixel quadrature imaging with metamaterials,” Adv. Opt. Mater., doi: 10.1002/adom.201500435 (to be published).