Machine learning has been widely adopted in various disciplines as they offer low-computational cost solutions to complex problems. Recently, deep learning-enabled methods for metasurface design have received increasing attention in the field of subwavelength electromagnetics. However, the previous metasurface design methods based on deep learning usually use huge datasets or complex networks to make deep neural networks achieve high prediction accuracy which results in more time for dataset establishment and network training. Here, we propose an expeditious and accurate scheme for designing phase-modulating dielectric metasurface through employing the transfer learning technology and genetic algorithm. The performance of the neural network is improved distinctly by migrating knowledge between real part and imaginary part spectrum-prediction tasks. Furthermore, the target meta-atoms can be optimized readily without increasing a large dataset through transfer learning. Finally, we design two deflectors and two metalenses as a proof-of-concept demonstration to validate the ability of our proposed approach. The scheme provides an efficient and promising design method for phase-modulating metasurface.
© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Metasurface, a two-dimension artificial subwavelength structure, can arbitrarily manipulate the amplitude, phase, and polarization of the electromagnetic (EM) wave. It has tremendous potential to replace traditional optical components due to their advantages of low profile and ease of integration [1–6]. The traditional design method of metasurface is a one-time and trial-and-error process with complicated electromagnetic simulations. The general design philosophy is that the optimization algorithm combines with the electromagnetic simulation tool (e.g. finite-element method (FEM) and rigorous coupled-wave analysis (RCWA)) to search for the best result under the guidance of the gradient or fitness [7,8]. For example, the topology optimization method obtains the optimal result by updating the parameters along the deepest-gradient direction, which is calculated through the corresponding adjoint problem [9–11]. Nevertheless, most traditional methods suffer from slow convergence, expensive full-wave simulation, and local optimization due to the unsuitable selection of the algorithm template, the initial population, and the step size.
Recently, with the continuous development of artificial intelligence and the successive update of computer hardware, a growing number of researchers use the deep learning-enabled method to design the subwavelength structure [12–17]. Deep learning is a data-driven algorithm based on the artificial neural network inspired by the connections and transmissions between neurons in the human brain , which has dramatically improved the state-of-the-art in image recognition , inverse sensing  and many other domains. According to the gradient calculated by the back-propagation algorithm, the deep neural network (DNN) can learn the nonlinear relationship between input data and output data by updating the weights of each neuron. Generally, DNN can be classified into the forward neural network (FNN) and the inverse neural network (INN). INN is often employed in the deep learning-enabled research of amplitude-modulated metasurface [21,22], which takes spectral responses as input and structural parameters as output for training. Then, the structural parameters can be obtained instantly by inputting the target spectrum into the well-trained neural network. The FNN takes structural parameters as input and corresponding spectral responses as output for training. The well-trained FNN can be considered as a specific electromagnetic simulator and be flexibly modified to suit various kinds of optical device design frameworks [23–27]. For example, physics-driven and data-driven neural networks are used for phase manipulation [28,29]. Most deep learning-enabled methods for metasurface devote to reducing the error of DNNs and learning the relationship between data efficiently. There are two main approaches at present. One way is to increase the dataset to learn more detailed information, and then reduce the training error and test error. Another is to utilize complex neural network structures to improve the learning ability, such as residual neural networks, tensor neural networks, and recurrent neural networks.
Despite the above two approaches can indeed improve the performance of DNNs, they have inherent shortcomings. The huge dataset is naturally accompanied by extensive calculations and lengthy dataset establishment process. Although the complex network structure can appropriately improve network accuracy, the process of tunning and training networks for each new task is complicated and time-consuming. Recently, transfer learning has attracted increasing attention, because it can significantly improve the performance in the target task by transferring the learned knowledge from the source task [30,31]. Qu et al. have demonstrated that the transfer learning can improve the ability to predict the transmittance of multilayer photonic film . In this paper, to efficiently design phase-modulating meta-devices, we propose a scheme based on the transfer learning and genetic algorithm (GA), which features several noteworthy advantages. Firstly, DNNs can rapidly achieve a high-performance spectral predicting ability for a new meta-structure in a small dataset via the transfer learning technology. Secondly, the performance of the proposed forward spectrum-prediction neural network (FPN) can be improved by transferring the knowledge from a similar task, for example from the real part FPN to the imaginary part FPN. The above two advantages can greatly shorten the time for designing phase-modulating meta-devices. Finally, to validate the ability of our proposed approach, we design and demonstrate two deflectors and two metalenses as a proof-of-concept application. The simulated results demonstrate our method provides an efficient and promising design approach for phase-modulating metasurface. Moreover, the scheme can be applied to design the multi-functional metasurface [33,34].
2. Theory and methodology
2.1 Data preparation
We choose the dielectric meta-structure to verify our proposed method due to their robust shape and low loss [35,36]. A dataset of rectangular-shaped meta-structures is generated by the commercial software CST Microwave Studio under normal incident y-polarized light with frequency of 100–300 THz. In the dataset, each meta-atom which consists of a silicon rectangular block with a square sapphire substrate is determined by four parameters length (l), width (w), height (h) and period (p) (Fig. 1(b)). The ranges of the four parameters are l∈[100, 400], w∈[100, 400], h∈[800, 1000] and p∈[500, 550] (all in nm). Different from the amplitude-modulated metasurface, designing phase-modulating metasurface needs to consider the amplitude and phase distributions simultaneously. The only phase-prediction DNNs demonstrated in previous research suffer from a relatively large average prediction error of 16° . To predict the phase responses accurately, the real part and imaginary part of the complex transmission coefficients are utilized to train the FPN, and the amplitude and phase can be calculated indirectly according to the following formula (1):
Therefore, our database is comprised of the meta-atom structure parameters (l, w, h, p) corresponding to the 4 inputs of the neural network and the real part and imaginary part of the complex transmission spectrum s (s1, s2…s51). The whole spectral data is down-sampled into 51 frequency points (from 100 to 300 THz) with a frequency step of 4THz, corresponding to the 51 outputs.
2.2 FPN based on transfer learning
As a data-driven algorithm, the essence of deep learning in metasurface can be concluded as using DNN to learn the forward and reverse mapping relationships between metasurface and corresponding electromagnetic responses. To realize the DNN-enabled optical design, a fully-connected neural network called the basic real part FPN is constructed at the beginning. It can predict the real part of complex transmission coefficients of meta-structures with high accuracy and plays an important role in designing meta-devices with transfer learning. The structure of FPN shown in Fig. 1(a) consists of six consecutive fully-connected hidden layers containing 1024, 512, 512, 512, 512, and 51 neurons, respectively. We can denote FPN as a function f that essentially performs the transform:
The loss function is defined as the mean square error (MSE) between the predicted spectrum and the simulated spectrum:
In existing researches, the neural network employed to predict the imaginary part spectrum needs to share the same network architecture with the real part spectrum-prediction FPN and to independently run the same number of training . It is known that the real part and imaginary part spectral data possess certain similarities, so transfer learning can be utilized to improve the prediction accuracy of FPN. Primarily, the basic real part FPN used to predict the real part spectrum of rectangular meta-atoms is trained with a total of 36000 data (75% training data and 25% testing data). It takes an average of 150 minutes per 10000 epochs of training on our personal computer equipped with an Nvidia RTX 2060 GPU. As the learning curve shown in Fig. 2(a), the overall test MSE is 0.00077 for the basic real part FPN after 20000 epochs. Then, the TransferFPN copies some layers from the basic real part FPN as the initialization weights and biases. Meanwhile, the entire network runs 2500 iterations of training on the same dataset of imaginary part samples. The differences of training results generated by transferring different layers are shown in Fig. 2(d). The error of transfer learning increases with the number of transferred layers. The best TransferFPN called the eventual imaginary part FPN has the lowest error of 0.00067 as shown in Fig. 2(b), and it only copies the first 3 layers of the basic real part FPN. This test error is lower than the error (0.00076, red line) of the un-transferred imaginary part FPN which has directly learned for 20000 epochs on the same dataset. The training time decreases from 4 hours to 20 minutes. The fractional errors of the amplitude and phase predicted by the basic real part FPN and the eventual transferred imaginary part FPN are 3.9% and 0.3%, respectively. The fractional errors are defined as:
The above results show that the forward prediction accuracy of the neural network can be improved through transfer learning, and the training time can be effectively reduced via a limited dataset.
To readily obtain the optimal meta-atom for each target EM response, we add the elliptical-shaped meta-atoms into our database. Each elliptical-shaped meta-atom is determined by four parameters m, n, h, p (m and n are the lengths of the major and minor axes of the ellipse, respectively). However, according to the traditional method, we need to create a new dataset with 36000 sets of samples, which is time-consuming and computationally intensive. Benefiting from the similarity of the parameters between the ellipse and the rectangle, although the real part and the imaginary part TransferFPNs for the elliptical meta-atoms trained 5000 iterations with a small training set of 4500 samples (the testing set contains 1500 samples), they can achieve high prediction accuracy after transferring some layers from the two well-trained FPNs above. Similarly, we compare the training results of the TransferFPNs when they transfer different layers in Fig. 2(e). From the results, we can tell that some middle layers transferred from the two well-trained FPNs are specific to the spectrum-prediction task of elliptical meta-atoms. These layers of the two well-trained FPNs would not help the target task learning at all, and can even be harmful to the final performance of the TransferFPNs when the number of transferred layers is above 4. Finally, the 3rd layers of the two well-trained FPNs for rectangular meta-atoms are transferred to the two eventual FPNs for elliptical meta-atoms. The learning curve of the eventual real part FPN for elliptical meta-atoms is shown in Fig. 2(c). After 5000 rounds of training, the test errors of the two best TransferFPNs are 0.00074 and 0.0008 respectively, and the fractional errors of the amplitude and phase are 2.3% and 0.6%. The results indicate that transfer learning not only improves the performance of the neural network but also greatly saves time for data establishment and network training, which provides more choices for our subsequent meta-devices design.
To better demonstrate the spectrum-prediction ability of FPN after transfer learning, we input the structural parameters of several rectangular and elliptical meta-atoms with different sizes into the corresponding FPNs. Several test results shown in Fig. 3 illustrate that the predicted spectra of FPN are agree well with the simulated spectra. The whole test results indicate that no matter how the structure changes within our proposed design ranges, the FPNs can calculate the corresponding EM performance with high accuracy. Remarkably, the FPNs only take several milliseconds to calculate the transmission spectrum for each meta-atom, which indicates that FPN and transfer learning have great prospects in the field of metasurface EM performance prediction. According to the test results in Fig. 3, the meta-atoms are not able to tailor the incident wave with 2π phase coverage within the frequency range [100 THz, 150 THz]. By enlarging the range of the structure parameters of meta-atoms, it could manipulate the incident wave within the frequency range [100 THz, 300 THz]. Therefore, the deep learning-enabled design approach depends on the quality of the dataset, which is related to the physical structure model and the range of the structure parameters. A potential way to overcome this limitation is to build a dataset with the arbitrary shape of meta-atoms and expand the dataset by the generative adversarial networks .
2.3 GA on the geatpy computation framework
In essence, the trained FPN is regarded as an EM simulator to take the place of the traditional electromagnetic simulation software in the metasurface design method. Compared with traditional simulation software, the calculation speed of FPN increases dozens of times. According to different goals, FPN can be combined with GA to design various meta-devices flexibly. The workflow of our scheme is illustrated in Fig. 4: 1) Dividing the objective meta-device into several pixels, each pixel corresponds to a meta-atom with one target phase. 2) For each target phase, initializing a parent population and utilizing the FPN to predict the spectrum of each individual in the population. 3) Calculating the fitness of the population by comparing the target spectrum with the objective spectrum. 4) Under the guidance of fitness, performing the selection, crossover, and mutation operations on the parent population, to obtain the offspring population. Repeating steps 2), 3), and 4) to obtain the optimal meta-atom on each pixel. 5) Arranging these meta-atoms to get the final meta-device. The objective function is:
The population size is 200, the mutation and crossover probability are both 0.6, and the maximum number of iterations is 2000. It takes 20 seconds on average to optimize a meta-atom for each target phase on our personal computer. The geatpy  evolutionary computation framework is employed in our method.
2.4 Designing phase-modulating meta-device
To verify the reliability of this method, we quickly designed two metasurface deflectors through this method. Such as the 1013.5 nm (296 THz) deflector, its unit cell contains 8 meta-atoms, and there is a 45° phase difference between two adjacent meta-atoms. The target phase distribution of the 8 meta-atoms is selected as [−5°, 40°, 85°, 130°, 175°, −140°, −95°, −40°]. Each meta-atom with the highest amplitude coefficient under the target phase is optimized by GA in turn. Finally, the optimized 8 meta-atoms are arranged one by one to generate the target deflector. The phase delays and the amplitude responses of the 8 basic meta-atoms for the designed 1013.5 nm are illustrated in Fig. 5(a), and the inset is the top view of the unit cell. The CST full-wave simulation results of the designed deflector are shown in Figs. 5(b) and 5(c), which indicates that the deflecting angle is −14.4°, and the deflection efficiency is 91.6%. Another 1500 nm deflector which has a deflection efficiency of 76.2% at −36.3° is designed by this method as well, and the simulation results are shown in Figs. 5(d-f). Certainly, other deflectors with different deflecting angles in the range of 100 THz to 300 THz can be designed by this method conveniently. The average design time for each deflector is around 5 minutes, and the maximum deflection efficiency can reach 91.6%.
Subsequently, one typical metalens is designed to further verify the universality of our method. Similar to the metasurface deflector, the focusing function of the metalens can be implemented through manipulating the electromagnetic wavefront by a group of meta-atoms with different shapes. The ideal phase distribution can be expressed as:5(k) and 5(l). Here, 30 meta-atoms with the maximum amplitude at 300 THz are optimized by our method, and the same meta-atom is placed on each circle with the same radius over the whole metalens surface. The solid red line represents the theoretical phase distribution of the metalens, and the red dots represent the simulated phases of the 31 meta-atoms along the x-axis in Fig. 5(g). Figures 5(h-j) show the CST full-wave simulation results of the metalens. The simulation focus length is 14.38 µm, and the focusing efficiency is 49.9%. The focusing efficiency is defined as the ratio of the energy concentrated into the central spot to the incident energy. It takes 20 minutes to optimize the metalens.
To further show the flexibility of wavefront manipulation with our method, we have designed the polarized-insensitive meta-lens by adding the constraint condition “l (m) = w (n)” in the GA. This restriction can ensure that polarization-independent structures are adopted in metasurface design. A 1.5 µm (200THz) polarized-insensitive metalens with a target focal length of 18 µm and radius of 16 µm is designed. The schematic and simulation results of polarization-insensitive metalens are shown in Figs. 6(a-e). The solid blue line and the red dots in Fig. 6(a) represent the target phase distributions and the optimized phase distributions along the x-axis, respectively. The simulation focus length and the focusing efficiency are 16.5 µm and 69.6% under the x-polarized and y-polarized incident light. The simulated results in Fig. 5 and Fig. 6 indicate that our method can be employed to design polarized-sensitive and polarized-insensitive phase-modulating meta-devices.
3. Conclusion and discussion
In summary, a simple and fast scheme for designing high-efficiency phase-modulating meta-devices is proposed in this paper, which features several advantages as below. Firstly, the real part or imaginary part FPN can achieve higher prediction accuracy in a short time through transfer learning. It not only improves the performance of the neural network without increasing the dataset or employing complex architecture but also avoids a lot of extra time and expensive calculations for training. Secondly, we can take the advantage of similarities in physical structures, and transfer some layers of the trained FPN for the original meta-structure to the un-trained FPN for task learning. This transfer learning method makes the process to obtain the optimal meta-atom with target EM performance easier, which remarkably reduces the time for generating a new dataset and training a new FPN. Finally, the deflection efficiency of the designed deflectors can reach 91.6% and the simulation results of the designed metalens are agree well with our preset target. The successful application of transfer learning in our method indicates that the FPN has indeed learned the physical relationship between the meta-atoms and corresponding EM responses. Our proposed method has potential in further study of the essential physical relationship and can be modified to design other multi-functional meta-devices in the future.
National Natural Science Foundation of China (61705233, 61975210).
The authors declare no conflicts of interest.
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
1. X. Luo, “Principles of electromagnetic waves in metasurfaces,” Sci. China: Phys., Mech. Astron. 58(9), 594201 (2015). [CrossRef]
2. N. Yu and F. Capasso, “Flat optics with designer metasurfaces,” Nat. Mater. 13(2), 139–150 (2014). [CrossRef]
3. Y. Wang, Q. Fan, and T. Xu, “Design of high efficiency achromatic metalens with large operation bandwidth using bilayer architecture,” Opto-Electron. Adv. 4(1), 20000801 (2021). [CrossRef]
4. A. Nemati, Q. Wang, M. Hong, and J. Teng, “Tunable and reconfigurable metasurfaces and metadevices,” Opto-Electron. Adv. 1(5), 18000901 (2018). [CrossRef]
5. X. Lu, Y. Guo, M. Pu, Y. Zhang, Z. Li, X. Li, X. Ma, and X. Luo, “Broadband achromatic metasurfaces for sub-diffraction focusing in the visible,” Opt. Express 29(4), 5947 (2021). [CrossRef]
6. F. Zhang, M. Pu, X. Li, P. Gao, X. Ma, J. Luo, H. Yu, and X. Luo, “All-dielectric metasurfaces for simultaneous giant circular asymmetric transmission and wavefront shaping based on asymmetric photonic spin-orbit interactions,” Adv. Funct. Mater. 27(47), 1704295 (2017). [CrossRef]
7. D. C. Kim, A. Hermerschmidt, P. Dyachenko, and T. Scharf, “Inverse design and demonstration of high-performance wide-angle diffractive optical elements,” Opt. Express 28(15), 22321 (2020). [CrossRef]
8. H. Hao, Z. Tingting, S. Qiang, and Y. Xiaodong, “Wide angle 2D beam splitter design based on vector diffraction theory,” Opt. Commun. 434, 28–35 (2019). [CrossRef]
9. S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vuckovic, and A. W. Rodriguez, “Outlook for inverse design in nanophotonics,” Nat. Photonics 12(11), 659–670 (2018). [CrossRef]
10. Z. Lin, B. Groever, F. Capasso, A. W. Rodriguez, and M. Lončar, “Topology optimized multi-layered meta-optics,” Phys. Rev. Applied 9(4), 044030 (2018). [CrossRef]
11. A. Y. Piggott, J. Lu, K. G. Lagoudakis, J. Petykiewicz, T. M. Babinec, and J. Vučković, “Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer,” Nat. Photonics 9(6), 374–377 (2015). [CrossRef]
12. I. Malkiel, M. Mrejen, A. Nagler, U. Arieli, L. Wolf, and H. Suchowski, “Plasmonic nanostructure design and characterization via deep learning,” Light: Sci. Appl. 7(1), 60 (2018). [CrossRef]
13. D. Liu, Y. Tan, E. Khoram, and Z. Yu, “Training deep neural networks for the inverse design of nanophotonic structures,” ACS Photonics 5(2), 13 (2018). [CrossRef]
14. S. So, T. Badloe, J. Noh, J. Bravo-Abad, and J. Rho, “Deep learning enabled inverse design in nanophotonics,” Nanophotonics 9(5), 1041–1057 (2020). [CrossRef]
15. R. S. Hegde, “Deep learning: a new tool for photonic nanostructure design,” Nanoscale Adv. 2(3), 1007–1023 (2020). [CrossRef]
16. T. Zhang, J. Wang, Q. Liu, J. Zhou, J. Dai, X. Han, Y. Zhou, and K. Xu, “Efficient spectrum prediction and inverse design for plasmonic waveguide systems based on artificial neural networks,” Photonics Res. 7(3), 368 (2019). [CrossRef]
17. J. Jiang and J. A. Fan, “Global optimization of dielectric metasurfaces using a physics-driven neural network,” Nano Lett. 19(8), 5366–5372 (2019). [CrossRef]
18. M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspectives, and prospects,” Science 349(6245), 255–260 (2015). [CrossRef]
19. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521(7553), 436–444 (2015). [CrossRef]
20. L. Chen, Y. Yin, Y. Li, and M. Hong, “Multifunctional inverse sensing by spatial distribution characterization of scattering photons,” Opto-Electron. Adv. 2(9), 19001901 (2019). [CrossRef]
21. J. Ma, Y. Huang, M. Pu, D. Xu, J. Luo, Y. Guo, and X. Luo, “Inverse design of broadband metasurface absorber based on convolutional autoencoder network and inverse design network,” J. Phys. D: Appl. Phys. 53(46), 464002 (2020). [CrossRef]
22. W. Ma, F. Cheng, and Y. Liu, “Deep-learning-enabled on-demand design of chiral metamaterials,” ACS Nano 12(6), 6326–6334 (2018). [CrossRef]
23. S. Inampudi and H. Mosallaei, “Neural network based design of metagratings,” Appl. Phys. Lett. 112(24), 241102 (2018). [CrossRef]
24. S. An, C. Fowler, B. Zheng, M. Y. Shalaginov, H. Tang, H. Li, L. Zhou, J. Ding, A. M. Agarwal, C. Rivero-Baleine, K. A. Richardson, T. Gu, J. Hu, and H. Zhang, “A deep learning approach for objective-driven all-dielectric metasurface design,” ACS Photonics 6(12), 3196–3207 (2019). [CrossRef]
25. Z. Liu, D. Zhu, K. Lee, A. S. Kim, L. Raju, and W. Cai, “Compounding meta-atoms into metamolecules with hybrid artificial intelligence techniques,” Adv. Mater. 32(6), 1904790 (2020). [CrossRef]
26. J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-form diffractive metagrating design based on generative adversarial networks,” ACS Nano 13(8), 8872–8878 (2019). [CrossRef]
27. S. An, B. Zheng, H. Tang, M. Y. Shalaginov, L. Zhou, H. Li, M. Kang, K. A. Richardson, T. Gu, J. Hu, C. Fowler, and H. Zhang, “Multifunctional metasurface design with a generative adversarial network,” Adv. Opt. Mater. 9(5), 2001433 (2021). [CrossRef]
28. L. Jiang, X. Li, Q. Wu, L. Wang, and L. Gao, “Neural network enabled metasurface design for phase manipulation,” Opt. Express 29(2), 2521–2528 (2021). [CrossRef]
29. I. Tanriover, W. Hadibrata, and K. Aydin, “Physics-based approach for a neural networks enabled design of all-dielectric metasurfaces,” ACS Photonics 7(8), 1957–1964 (2020). [CrossRef]
30. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” arXiv e-prints arXiv:1411.1792 (2014).
31. A. A. Rusu, M. Vecerik, T. Rothörl, N. Heess, R. Pascanu, and R. Hadsell, “Sim-to-real robot learning from pixels with progressive nets,” arXiv e-prints arXiv:1610.04286 (2016).
32. Y. Qu, L. Jing, Y. Shen, M. Qiu, and M. Soljacic, “Migrating knowledge between physical scenarios based on artificial neural networks,” ACS Photonics 6(5), 1168–1174 (2019). [CrossRef]
33. B. Groever, N. A. Rubin, J. P. B. Mueller, R. C. Devlin, and F. Capasso, “High-efficiency chiral meta-lens,” Sci. Rep. 8(1), 7240 (2018). [CrossRef]
34. X. Ma, M. Pu, X. Li, Y. Guo, and X. Luo, “All-metallic wide-angle metasurfaces for multifunctional polarization manipulation,” Opto-Electron. Adv. 2(3), 18002301 (2019). [CrossRef]
35. Y. F. Yu, A. Y. Zhu, R. Paniagua-Domínguez, Y. H. Fu, B. Luk’yanchuk, and A. I. Kuznetsov, “High-transmission dielectric metasurface with 2π phase control at visible wavelengths,” Laser Photonics Rev. 9(4), 412–418 (2015). [CrossRef]
36. S. M. Kamali, A. Arbabi, E. Arbabi, Y. Horie, and A. Faraon, “Decoupling optical function and geometrical form using conformal flexible dielectric metasurfaces,” Nat. Commun. 7(1), 11618–7 (2016). [CrossRef]
37. Jazzbin, et.al. “geatpy: The genetic and evolutionary algorithm toolbox with high performance in python,” http://www.geatpy.com/ (2020)