Metasurfaces have shown promising potentials in shaping optical wavefronts while remaining compact compared to bulky geometric optics devices. The design of meta-atoms, the fundamental building blocks of metasurfaces, typically relies on trial and error to achieve target electromagnetic responses. This process includes the characterization of an enormous amount of meta-atom designs with varying physical and geometric parameters, which demands huge computational resources. In this paper, a deep learning-based metasurface/meta-atom modeling approach is introduced to significantly reduce the characterization time while maintaining accuracy. Based on a convolutional neural network (CNN) structure, the proposed deep learning network is able to model meta-atoms with nearly freeform 2D patterns and different lattice sizes, material refractive indices and thicknesses. Moreover, the presented approach features the capability of predicting a meta-atom’s wide spectrum response in the timescale of milliseconds, attractive for applications necessitating fast on-demand design and optimization of a meta-atom/metasurface.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Metasurfaces, the 2D version of metamaterials, provide a novel platform for the realization of ultra-thin and large-scale optical components and systems. By manipulating the geometry of individual meta-atoms, desired responses (e.g. phase and amplitude) can be realized at the unit-cell level over the flat surface for full control of light propagation. The key challenge in the field of metasurface/meta-atom design is the non-intuitive design process, which makes it difficult to find optimal design parameters to meet specific requirements. Current design approaches include trial-and-error methods and inverse design methods based on optimization algorithms or deep neural networks (DNNs). For the traditional trial-and-error method, a commonly adopted design process includes complete exploration of the entire design space and careful selection of results that fit the design requirements. In this case, the design time is determined by the simulation time of each trial design and the number of design degrees of freedom (DOFs). Therefore, exhaustive exploration of all design parameters is unrealistic without a fast evaluation tool when massive design DOFs must be taken into consideration (e.g. in freeform meta-atom designs). For inverse design approaches, which are based on either optimization algorithms [1–4] or neural networks [5–13], their converging speed and design accuracy largely rely on the simulation speed and accuracy of the local or commercial solver that is cascaded to the optimizer. One exception is the recent deep learning metasurface design approach based on generative adversarial networks (GANs) [14–20], which incorporates a self-evolving Critic, rather than a well-trained DNN simulator to evaluate the performance of the generated designs. However, since a GAN requires a noise vector as part of the input, the corresponding output designs have unstable performance and thus still need to be verified by a simulator [14,17]. Therefore, modeling and characterization tools play a pivotal role in almost all current metasurface/meta-atom design approaches. In this context, reliable and time-efficient modeling tools are being heavily investigated to facilitate the design of next generation meta-devices often characterized by large numbers of design DOFs and multi-functionality.
One approach being considered is the development of analytical effective medium models, such as the Lewin model  and the GEM model . Although these models are simple and efficient, their application is limited to metamaterials with spherical inclusions under the long-wavelength approximations. Another widely adopted approach relies on iterative numerical full-wave simulations based on different methods, including FEM (finite-element method), FDTD (finite-difference time-domain) and FIT (finite integration technique). This approach provides accurate results but requires considerable computing resources. A data-driven modeling tool based on deep neural networks (DNNs) [9,11,13,23] emerged recently and has been proven to be both accurate and time-efficient. Previous work has employed fully-connected layers (FCLs) to realize accurate spectral response predictions for plasmonic [5,9,10] or all-dielectric [6,8,12,13] nanophotonic structures with bulk layers , cavities  and meta-atoms in the shape of cylinders [8,9,12,13,24], elliptic cylinders , spheres  and bars [5,10]. While most works [5,6,8–11,25] has been focused on amplitude response predictions, recent work  has demonstrated that neural networks are also capable of predicting meta-atoms’ phase responses, which is crucial to the design of phase-gradient meta-optical devices. After being fully-trained with sufficient data, the DNN models are highly accurate and are able to generate electromagnetic (EM) responses on the time scale of milliseconds, which enables fast, on-demand meta-atom/metasurface designs. However, there are remaining issues with these existing networks constructed with FCLs. Firstly, these works mainly deal with simple meta-atom structures that are described by only 3-5 parameters, which heavily constrains the accessible meta-atoms design DOF. Secondly, these DNN models are operating in a very restricted design space. Design parameters including lattice sizes, meta-atom thicknesses and material properties are fixed in these networks. Once one of these design parameters falls outside the design space, a new dataset needs to be collected and the model must be re-trained, a process which can be time-consuming. The current DNN approach is therefore inadequate as a general design scheme.
In this paper, we present a new DNN approach for the modeling and characterization of three-dimensional (3D) meta-atoms, which addresses the issues and limitations discussed above. To expand the design space and demonstrate the network’s generality, our approach considers high DOFs structures and accounts for the meta-atom’s two-dimensional (2D) geometrical pattern, material refractive index, thickness and lattice size. After being trained with sufficient data, the proposed network is able to generate accurate phase and amplitude predictions of meta-atoms with complex shapes across a wide spectrum. Furthermore, we demonstrated our network’s generalizability by testing it with meta-atoms which contain features that do not exist in the training data. To show the efficacy of the proposed method, it has been applied for practical metasurface/meta-device design and optimization. The performance of the resulting metasurface/meta-device prototypes corroborates that the network achieves two important features for DNN-based meta-atom modeling: 1) fast and accurate performance evaluation of geometrically complex meta-atoms and metasurfaces; and 2) a modeling tool that covers an extensive design space of 3D meta-atoms. It is envisioned that the proposed deep learning network can be readily applied to various meta-atom modeling and optimization tasks, as well as be extended to other fields such as the characterization and design of dielectric resonator antennas, optical circuits and chiral metamaterials.
2. Network architecture and results
To address the two goals discussed above, a predicting neural network (PNN) was constructed based on a CNN architecture (Fig. 1). The PNN aims to uncover the hidden relationship between meta-atom models and their spectral responses and thus accurately predict responses for given meta-atom designs. The meta-atom model under evaluation consists of a quasi-freeform dielectric structure (preferably with a higher refractive index) sitting on a dielectric substrate (preferably with a lower refractive index) to form a square-shaped unit cell. To prove the proposed method is not limited to meta-atoms with certain shapes or materials, we parameterized each meta-atom structures’ 2D pattern, lattice size, thickness and refractive index and combined them as the input of the PNN. To reconcile the huge dimensional mismatch between the 2D cross-section (in this case, an image composed of 64 × 64 pixels) and other properties (i.e. material index, thickness, lattice size in the form of a 1D vector), the 2D structure images (circled in blue in Fig. 1) and 1D property vector (circled in yellow in Fig. 1) were processed through separate input networks. Both were later combined and passed through the rest of the network to yield output spectra. To speed up the training process and achieve better accuracy for the 1D property-processing network, we applied a Neural Tensor Network (NTN) layer [10,13,26] to process the relational information provided with the 1D input properties. The output of the NTN layer was then expanded to match the spatial dimensions  of the output given by the 2D image processing network. The two outputs were then concatenated together and processed with convolution and pooling layers. After flattening the output of the CNNs and passing through two fully connected layers, predictions for the real and imaginary parts of the transmission coefficient were generated and ready for evaluation (Supplement 1, Section I). Without loss of generality, the spectra of interest were set to be from 30 to 60 THz (5 μm to 10 μm in wavelength).
Over 100,000 groups of quasi-freeform meta-atom patterns were randomly generated using the “needle drop” approach (Supplement 1, Section II), while the other parameters were created randomly within the following ranges (all lengths in microns): thickness ∈ [0.5, 1], refractive index ∈ [3.5, 5], lattice size ∈ [2.5, 3], since these ranges include ample samples of phase and amplitude coverage. The electromagnetic responses of these meta-atom models were then calculated in a FEM-based simulation tool and assigned with labels. The meta-atoms were randomly split into training and test data sets, with 70% used during the training process and the remaining 30% used to evaluate the trained network. The spectral response predictions generated with the PNN were compared with the labels to extract the error, which were minimized during the training process (learning curves and hyperparameters are included in Supplement 1, Section III). When the training was completed, the average mean squared error (MSE) for the real and imaginary parts of the predicted transmission coefficients in the test data were 0.00035 and 0.00023, respectively (equivalent to an average prediction standard deviation of 0.005 (amplitude) and 0.78 degrees (phase) at each single frequency point). An ablation analysis (Supplement 1, Section IV) is carried out to justify the necessity of the different data processing approaches adopted in this network, including the use of NTN layers, batch normalization layers, spatial tiling and split real/imaginary component prediction method. The ablation analysis shows that including these layers improved the converging speed and final accuracy of prediction results (Supplement 1, Section III).
We demonstrate the validity of the well-trained PNN with two meta-atom samples (Fig. 2(a)) that were randomly selected from the test dataset. Their transmissive amplitudes (blue dots) and phase responses (red dots) were evaluated and compared with the results derived from FEM-based simulations (blue and red curves). As indicated by the minimal test error, the PNN prediction results agreed well with the full-wave electromagnetic simulations. Moreover, to verify the PNN’s applicability to new, previously unseen data, we tested the PNN with meta-atom designs possessing features that did not exist in either the training or test datasets. This included meta-atoms with the shapes of circles (Fig. 2(b)) and rings (Fig. 2(c)), which cannot be obtained using the needle drop method adopted here. The radii defining the circles and rings in Figs. 2(b) and 2(c) were randomly generated. The patterns showing as insets in Figs. 2(b) and 2(c) are pixelated images that can best describe the generated circles and rings with a resolution of 64 by 64 pixels. Other parameters, including refractive index, meta-atom thickness and lattice size, were randomly selected within the preset parameter range. Similarly, as in Fig. 2(a), the amplitude and phase responses of both designs in Figs. 2(b) and 2(c) were evaluated using the PNN (dots), and then compared with the full-wave electromagnetic simulation results (curves). The PNN maintained its accuracy even with those meta-atoms that have previously unseen features, indicating its broad generalization ability. More importantly, once trained with enough data, the proposed PNN can produce the predictions in milliseconds, which makes it an appealing substitute for conventional simulation tools for applications such as high-throughput device evaluation and optimization.
The well-trained PNN can be adopted in various application scenarios. Below, we discuss the possibility of utilizing this fast and accurate modeling tool to solve problems, including meta-atom inverse designs and meta-device optimizations.
3.1 Comparing performances of meta-atoms with different shapes
Dielectric metasurface/meta-atom design platforms built with various constituent materials [28–30] feature a large number of design DOFs, including the 2D patterns, refractive indices, thicknesses and lattice sizes of meta-atoms. With the help of this fully trained PNN, we are able to evaluate the performance of numerous meta-atoms in a short time. By comparing the performance between quasi-freeform shaped meta-atoms and those with regular colonial shapes, such as circles, rectangles and “H”s, we demonstrate how the added DOFs can be exploited to boost the overall efficiency and phase coverage of all-dielectric meta-atoms. In Fig. 3, four meta-atom pattern datasets of circles, rectangles, “H”-shapes, and quasi-freeform shapes were constructed, each containing 3,000 randomly generated patterns. Each pattern was pixelated into a 64 by 64 pixel image (Fig. 3(a)), which is in accordance with the input dimensions of the PNN. We compare the performance of these meta-atoms, as predicted by the PNN, at two different working frequencies, where materials, thicknesses and lattice sizes are considered as design variables. In these two cases (Figs. 3(b) and 3(c)), the meta-atoms with quasi-freeform cross sections are clearly superior to the meta-atoms from the other three categories in terms of both efficiency and phase coverage. It is apparent from the superior amplitude/phase coverage obtained by the complex meta-atom shapes that the extra design DOFs would largely improve the performance of metasurfaces, especially multifunctional metasurfaces where a single meta-atom must provide good performance for multiple conditions such as changes in frequencies, polarizations, materials, etc. This further highlights the need to model freeform metasurfaces and the efficacy of the proposed DNN approach. Below we discuss several practical applications enabled by this approach that are prohibitively time-consuming using conventional methods (e.g. full-wave simulations).
3.2 Fast phase and amplitude coverage evaluation
Since most metasurfaces/meta-devices are composed of elements with the same lattice size, thickness and material, the choice of this parameter combination determines the limits of phase and amplitude coverage. With inappropriate choices of these parameters, it is difficult, if not impossible to realize large phase coverages, even with massive design DOFs. Meanwhile, most high-efficiency meta-devices, including lenses and beam deflectors, require meta-atoms that can achieve full 2π phase coverage while maintaining high transmission efficiency. As a result, designers usually explore vast parametric spaces to find maximum phase and amplitude coverage. This design space exploration process is time consuming and inefficient (and often impossible due to the prohibitively large parameter space). Such a design process becomes even more challenging and prohibitive when freeform-shaped patterns are under consideration.
Alternatively, the proposed PNN is able to evaluate the phase and amplitude responses of a given meta-atom with quasi-freeform pattern in a short, one-time calculation process. For example, meta-atoms with larger volumes and refractive indices can support more electromagnetic resonances, and are thus more likely to achieve high efficiency with full 2π phase coverage . However, under certain circumstances, increasing thickness or lattice sizes can lead to mismatch between a meta-atom’s intrinsic electric and magnetic dipoles and reduce the overall phase coverage. To address this issue, we randomly generated over 20,000 quasi-freeform meta-atom patterns and combined them with different refractive indices, thicknesses and lattices sizes, and evaluated their performance at 57 THz (5.26 μm in wavelength) using the proposed PNN. As shown in Fig. 4(a), we fixed the index to be 4.5 and lattice size to be 2.6 μm, and evaluated the influence of varying thickness on the phase coverage of the same group of meta-atom patterns. It is observed that the phase coverage increased at first but started to decrease when the thickness continued to increase. When the thickness is equal to 1 μm, it’s hard to pick a group of high efficiency meta-atoms in the area highlighted in the red dashed line. Similarly, in Fig. 4(b), when fixing the index to be 5 and thickness to be 0.7 μm, the meta-atom phase coverage drops as the lattice size increases from 2.6 μm to 3 μm. In both cases, the proper meta-atom design parameter combinations with maximum phase coverage can be identified in seconds using the presented PNN, which highlights its efficacy in searching for new meta-atom designs.
3.3 Fast meta-device optimization
Another application of the PNN is adopting it as an optimization tool for DNN-based meta-device designs. One major advantage of DNN-based metasurface/meta-atom design approaches is that they are able to generate multiple designs at almost no additional cost. Here however, all the generated designs need to be screened via full-wave simulations to identify the design with optimal performance, which can be much more time-consuming than the design-generation process. The proposed PNN for quasi-freeform-shaped metasurfaces can significantly alleviate the issue due to its fast prediction capability.
As a demonstration, we employed a well-trained GAN model  to design a transmissive meta-lens working at 57 THz. The lens has a focal length of 80 μm, a numerical aperture (NA) of 0.63, and is composed of 50 by 50 dielectric meta-atoms (refractive index 4.7 and thickness 0.75 μm) on a dielectric substrate (refractive index 1.4) with a lattice size of 2.6 μm (aka a total lens size of 130 μm by 130 μm). After the phase mask of this lens was calculated, we trained a generative meta-atom design network  to generate a meta-atom design for each unit cell within the metalens. The EM responses of the generated meta-atoms were then evaluated using the proposed PNN. This cascaded design-evaluate process was executed for several iterations so that the best meta-atom designs generated during these iterations were selected to assemble the final device. To verify the prediction accuracy of the PNN and the efficacy of this design-evaluate process, we employ this cascaded network (GAN + PNN) to run 1, 2, 5 and 10 iterations to generate 4 different meta-lenses (shown in Supplement 1, Section VI), and then tested their performance using full-wave simulations. The metalenses are situated in the x-y plane, with the optical axis along the z-axis. The simulated 2D electric fields of four different metalenses in the x-z plane, along with the 1D electric field along the x axis in the focal plane, are plotted in Fig. 5. The peak electric field amplitude at the center of the focal spot was improved with increasing optimization iterations, validating that meta-atom designs with better performance (corresponding to higher transmission and more precise phase shift) have been identified with the help of the proposed PNN during the optimization iterations. Importantly, with this data-driven approach, time taken for the performance evaluation process is dramatically reduced and becomes comparable with the design generation time, thereby solving the problem of computational overhead for verification that limits existing DNN-based approaches. A detailed time efficiency contrast analysis is included in Section VII in Supplement 1.
In this paper, a deep learning-based dielectric meta-atom modeling approach is proposed and demonstrated. Accurate spectrum responses of dielectric meta-atoms were derived using a novel CNN-based network structure. Compared with previous work, our approach can handle a significantly larger set of input parameters, including various shapes, thickness, lattice size and refractive index of the meta-atoms. The proposed deep learning network processes enable strong generalizability and make accurate predictions for meta-atoms with features that do not exist in the training dataset. We have further demonstrated that the presented network can be adopted as an efficient modeling tool in various application scenarios, including rapid meta-atom design and meta-device optimization. The proposed DNN-based methodology presented herein can be readily applied to other physics domains where a fast, accurate modeling tool is highly desired to provide the link between a broad and sophisticated parametric space and corresponding physical responses.
Defense Advanced Research Projects Agency (HR00111720029).
The authors declare no conflicts of interest.
See Supplement 1 for supporting content.
The all-dielectric meta-atom dataset used during training is open source and available at https://github.com/SensongAn/Meta-atoms-data-sharing. This work also participates in the MetaNet  data sharing project and will be available at http://metanet.stanford.edu/ soon.
1. D. Z. Zhu, E. B. Whiting, S. D. Campbell, D. B. Burckel, and D. H. Werner, “Optimal high efficiency 3D plasmonic metasurface elements revealed by lazy ants,” ACS Photonics 6, 2741–2748 (2019). [CrossRef]
2. J. Jiang and J. A. Fan, arXiv preprint arXiv:1906.07843 (2019).
3. J. Jiang and J. A. Fan, arXiv preprint arXiv:1906.04157 (2019).
4. Z. Liu, L. Raju, D. Zhu, and W. Cai, arXiv preprint arXiv:1902.02293 (2019).
5. I. Malkiel, M. Mrejen, A. Nagler, U. Arieli, L. Wolf, and H. Suchowski, “Plasmonic nanostructure design and characterization via Deep Learning,” Light: Sci. Appl. 7(1), 60 (2018). [CrossRef]
6. D. Liu, Y. Tan, E. Khoram, and Z. Yu, “Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures,” ACS Photonics 5(4), 1365–1369 (2018). [CrossRef]
7. Z. Liu, D. Zhu, K.-T. Lee, A. S. Kim, L. Raju, and W. Cai, arXiv preprint arXiv:1907.03366 (2019).
8. L. Gao, X. Li, D. Liu, L. Wang, and Z. Yu, “A bidirectional deep neural network for accurate silicon color design,” Adv. Mater. 31, e1905467 (2019). [CrossRef]
9. X. Li, J. Shu, W. Gu, and L. Gao, “Deep neural network for plasmonic sensor modeling,” Opt. Mater. Express 9(9), 3857–3862 (2019). [CrossRef]
10. W. Ma, F. Cheng, and Y. Liu, “Deep-Learning-Enabled On-Demand Design of Chiral Metamaterials,” ACS Nano 12(6), 6326–6334 (2018). [CrossRef]
11. J. Peurifoy, Y. Shen, L. Jing, Y. Yang, F. Cano-Renteria, B. G. DeLacy, J. D. Joannopoulos, M. Tegmark, and M. Soljačić, “Nanophotonic particle simulation and inverse design using artificial neural networks,” Sci. Adv. 4(6), eaar4206 (2018). [CrossRef]
12. K.-F. Lin, C.-C. Hsieh, S.-C. Hsin, and W.-F. Hsieh, “Achieving high numerical aperture near-infrared imaging based on an ultrathin cylinder dielectric metalens,” Appl. Opt. 58(32), 8914–8919 (2019). [CrossRef]
13. S. An, C. Fowler, B. Zheng, M. Y. Shalaginov, H. Tang, H. Li, L. Zhou, J. Ding, A. M. Agarwal, C. Rivero-Baleine, K. A. Richardson, T. Gu, J. Hu, and H. Zhang, “A Deep Learning Approach for Objective-Driven All-Dielectric Metasurface Design,” ACS Photonics 6(12), 3196–3207 (2019). [CrossRef]
14. Z. Liu, D. Zhu, S. P. Rodrigues, K.-T. Lee, and W. Cai, “Generative Model for the Inverse Design of Metasurfaces,” Nano Lett. 18(10), 6570–6576 (2018). [CrossRef]
15. W. Ma, F. Cheng, Y. Xu, Q. Wen, and Y. Liu, “Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy,” arXiv preprint arXiv:1901.10819 (2019).
16. J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-Form Diffractive Metagrating Design Based on Generative Adversarial Networks,” ACS Nano 13(8), 8872–8878 (2019). [CrossRef]
17. S. An, B. Zheng, H. Tang, M. Y. Shalaginov, L. Zhou, H. Li, T. Gu, J. Hu, C. Fowler, and H. Zhang, arXiv preprint arXiv:1908.04851 (2019).
18. S. So and J. Rho, “Designing nanophotonic structures using conditional deep convolutional generative adversarial networks,” Nanophotonics 8(7), 1255–1261 (2019). [CrossRef]
19. X. Han, Z. Fan, C. Li, Z. Liu, and L. J. Guo, arXiv preprint arXiv:1912.03696 (2019).
20. H. P. Wang, Y. B. Li, H. Li, S. Y. Dong, C. Liu, S. Jin, and T. J. Cui, “Deep learning designs of anisotropic metasurfaces in ultrawideband based on generative adversarial networks,” Advanced Intelligent Systems, 2000068 (2020). [CrossRef]
21. L. Lewin, “The electrical constants of a material loaded with spherical particles,” J. Inst. Electr. Eng. 3 94(27), 65–68 (1947). [CrossRef]
22. B. A. Slovick, Z. G. Yu, and S. Krishnamurthy, “Generalized effective-medium theory for metamaterials,” Phys. Rev. B 89(15), 155118 (2014). [CrossRef]
23. P. R. Wiecha and O. L. Muskens, “Deep Learning Meets Nanophotonics: A Generalized Accurate Predictor for Near Fields and Far Fields of Arbitrary 3D Nanostructures,” Nano Lett. 20(1), 329–338 (2020). [CrossRef]
24. R. Lin, Y. Zhai, C. Xiong, and X. Li, “Inverse design of plasmonic metasurfaces by convolutional neural network,” Opt. Lett. 45(6), 1362–1365 (2020). [CrossRef]
25. Y. Kiarashinejad, M. Zandehshahvar, S. Abdollahramezani, O. Hemmatyar, R. Pourabolghasem, and A. Adibi, arXiv preprint arXiv:1909.07330 (2019).
26. R. Socher, D. Chen, C. D. Manning, and A. Ng, “Reasoning with neural tensor networks for knowledge base completion,” inAdvances in neural information processing systems (2013), pp. 926–934.
27. S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” Int. J. Robot. Res. 37(4-5), 421–436 (2018). [CrossRef]
28. A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol. 10(11), 937–943 (2015). [CrossRef]
29. P. Genevet, F. Capasso, F. Aieta, M. Khorasaninejad, and R. Devlin, “Recent advances in planar optics: from plasmonic to dielectric metasurfaces,” Optica 4(1), 139–152 (2017). [CrossRef]
30. L. Zhang, J. Ding, H. Zheng, S. An, H. Lin, B. Zheng, Q. Du, G. Yin, J. Michon, Y. Zhang, Z. Fang, M. Y. Shalaginov, L. Deng, T. Gu, H. Zhang, and J. Hu, “Ultra-thin high-efficiency mid-infrared transmissive Huygens meta-optics,” Nat. Commun. 9(1), 1481 (2018). [CrossRef]
31. K. E. Chong, I. Staude, A. James, J. Dominguez, S. Liu, S. Campione, G. S. Subramania, T. S. Luk, M. Decker, and D. N. Neshev, “Polarization-Independent Silicon Metadevices for Efficient Optical Wavefront Control,” Nano Lett. 15(8), 5369–5374 (2015). [CrossRef]
32. J. Jiang, R. Lupoiu, E. W. Wang, D. Sell, J. P. Hugonin, P. Lalanne, and J. A. Fan, “MetaNet: a new paradigm for data sharing in photonics research,” Opt. Express 28(9), 13670–13681 (2020). [CrossRef]