Open-loop adaptive optics is a technique in which the turbulent wavefront is measured before it hits the deformable mirror for correction. We present a technique to model a deformable mirror working in open-loop based on multivariate adaptive regression splines (MARS), a non-parametric regression technique. The model’s input is the wavefront correction to apply to the mirror and its output is the set of voltages to shape the mirror. We performed experiments with an electrostrictive deformable mirror, achieving positioning errors of the order of 1.2% RMS of the peak-to-peak wavefront excursion. The technique does not depend on the physical parameters of the device; therefore it may be included in the control scheme of any type of deformable mirror.
©2010 Optical Society of America
Open-loop adaptive optics (AO) is a technique devised for multi-object adaptive optics (MOAO), which is one of the types of AO proposed to widen the limited field-of-view of ‘classical’ closed-loop astronomical AO systems [1,2]. In MOAO, wavefront sensors are locked on natural and/or laser guide stars in the field, measuring uncorrected, turbulent wavefronts. One or more pick-off mirrors are placed at science target locations in the image plane, directing the light to deformable mirrors (DMs) in the corresponding pupil planes. The turbulence is thus corrected for the specific line-of-sight to the science object, using estimations based on the measured guide star wavefronts. In open-loop AO, the DM shape cannot be controlled by the wavefront sensor as in closed-loop AO. In Guzmán et al , the problem has been tackled using a dedicated wavefront sensor measuring the DM shape and closing a local control loop on it. In this paper a mathematical model is presented to control an electrostrictive DM in open-loop, using multivariate adaptive regression splines (MARS). The use of MARS to build a DM model presents some advantages compared to more traditional techniques such as multiple linear regression and neural networks. Possibly the main advantage is the fact that MARS is able to describe a given response (e.g. actuator voltage) starting from a large number of predictors (e.g. DM facesheet positions), from which the best predictors are automatically selected, therefore the variables space is kept under control. Compared to neural networks and partial least squares (PLS) , MARS models are easier to interpret, since the original variables can be directly found in the resulting model and even interactions between the variables are indicated. Thus, MARS is able to build flexible models without the disadvantages of the more ‘black-box’ methods, as PLS and neural networks are sometimes called.
Applying voltages to a grid of actuators underneath the mirror deforms the continuous facesheet of the DM. The final shape of the mirror depends on factors such as the rigidity of the facesheet and the relationship between voltage applied and actuator displacement. The position of the facesheet at any given point depends upon the displacement of all actuators in its neighborhood as well as other parameters such as hysteresis of the actuators and operational temperature. As it will be seen in this paper, the final shape is not a linear combination of actuator displacements; therefore it is not straightforward to implement a simple model of a DM. Previous works modeling deformable mirrors have explored the physics behind the actuators and the mirror facesheet. Hom et al  presents a non-linear model of an electrostrictive DM, incurring a 13% underestimation error with respect to the real mirror position in a static test and ~40 nm rms errors when canceling a given wavefront. Andersen et al  and Laag et al  report results with a voice-coil actuator DM made by ALPAO (France), achieving errors of the order of 10-20 nm rms. Laag et al used a simple linear superposition of actuators to determine the final shape of the mirror. As stated by Laag et al, the ALPAO mirror design considerations included reduction of nonlinearities and cross-coupling; thereby resulting in a mirror that is easier to control in open-loop. MEMS DMs have become popular in AO because of their high density of actuators and negligible hysteresis [8, 9]. Electrostrictive actuators, as well as MEMS actuators, have a quadratic response to voltage [8, 12]. Morzinski et al  and Stewart et al  have developed open-loop models for MEMS DMs based on a physical model of the electrostatic force for the actuator and a thin plate equation for the facesheet; both groups report errors on the order of 15 nm rms.
Our MARS model was trained with DM surface measurements taken with a “Fisba” Twyman-Green interferometer, which works with monochromatic light at 633 nm, displaying the surface of the mirror in a 512 x 512 pixels phase map in Angstroms.
The mirror we modeled is a ‘Xinetics’ with 97 electrostrictive actuators. Once trained, the model has a set of non-recursive equations that can be easily implemented in real-time. Using this technique, we are proposing a general technique to operate a DM in open loop and in real-time.
We stress that hysteresis, which is inherent to the behavior of electrostrictive actuators (and of clear importance in open loop control), was present during both the training and the testing of our model. It is true that the level of hysteresis demonstrated by this type of actuator increases strongly if the temperature is reduced towards the Curie point, whereas our DM was operated around 20 Celsius where the maximum hysteresis-induced error is limited to around 2-4% of the total stroke. However, we argue that an electrostrictive DM employed at temperatures where hysteresis could greatly exceed this range would be a very odd choice for open-loop control. There are several alternative actuator technologies which can provide similar or better hysteresis performance at lower temperatures; these include harder piezoelectric ceramics, electrostatic MEMS DMs and magnetic-based actuators. We therefore believe that our model has been derived and tested with hysteresis present at the maximum levels likely to be present in any open loop control design.
2. Multivariate adaptive regression splines
Multivariate Adaptive Regression Splines, MARS, is a multivariate nonparametric regression technique introduced by Friedman [4, 13] in 1991. The space of the predictors (e.g. DM voltages) is split into several (overlapping) regions in which spline functions are fit. The main purpose of the MARS model is to predict the values of a continuous dependent variable, , from a set of independent explanatory variables, . The MARS model can be represented as in Eq. (1):
MARS can be considered as a generalization of Classification and Regression Trees (CART), and is able to overcome some limitations of CART  Due to its advantages, its use has become popular in the last years for different purposes . MARS does not require any a priori assumptions about the underlying functional relationship between dependent and independent variables. Instead, this relation is uncovered from a set of coefficients and piecewise polynomials of degree q (basis functions) that are entirely driven from the regression data (y, X). The MARS regression model is constructed by fitting basis functions to distinct intervals of the independent variables. Generally, piecewise polynomials, also called splines, have pieces smoothly connected together. In MARS terminology, the joining points of the polynomials are called knots, nodes or breakdown points. These will be denoted by the small letter t. For a spline of degree q each segment is a polynomial function. MARS uses two-sided truncated power functions as spline basis functions, described by Eqs. (2) and 3:
When , which is the case in this study, only simple linear splines are considered. A pair of splines for at the knot is presented in Fig. 1 .
The solid line represents the right-sided spline, , , which is positive for all objects located at the right side of the knot t. The dashed line represents the left-sided spline, , , which is positive for all objects located at the left side of the knot t. The two-sided truncated functions of the dependent variable are basis functions, linear or nonlinear, that describe the underlying phenomena. The MARS model of a dependent variable y with M basis functions (terms) can be written as Eq. (4):
Both the variables to be introduced into the model and the knot positions for each individual variable have to be optimized. For a data set X containing n objects and p explanatory variables, there are pairs of spline basis functions, given by (2) and (3), with knot locations (i=1, 2, …, n; j=1, 2,…, p).
A two-step procedure is followed to construct the final model. First, in order to select the consecutive pairs of basis functions of the model, a two-at-a-time forward stepwise procedure is implemented . This forward stepwise selection of basis functions leads to a very complex and overfitted model. Such a model, although it fits the data well, has poor predictive abilities for new objects. To improve the prediction, the redundant basis functions are removed one at a time using a backward stepwise procedure. To determine which basis functions should be included in the model, MARS utilizes the generalized cross-validation  (GCV). The GCV is the mean squared residual error divided by a penalty dependent on the model complexity. The GCV criterion is defined as in Eq. (5):Eq. (6):Eq. (4), and the parameter d is a penalty for each basis function included into the model. It can be also regarded as a smoothing parameter. Large values of d lead to fewer basis functions and therefore smoother function estimates. For more details about the selection of the d parameter, see . In our studies, the parameter d equals 2, and the maximum interaction level of the spline basis functions is restricted to 3.The main steps of the MARS algorithm as applied here can be summarised as follows:
- 1. Start with the simplest model, i.e. with the constant coefficient only.
- 2. Explore the space of the basis functions for each explanatory variable.
- 3. Determine the pair of basis functions that minimises the prediction error, and include them in the model.
- 4. Go to step 2 until a model with predetermined complexity is derived.
- 5. Search the entire set of basis functions (excluding the constant) and delete from the model the one that contributes least to the overall goodness of fit using the GCV criterion.
- 6. Repeat 5 until GCV reaches its minimum.
The predetermined complexity of MARS model in step 3 should be considerably larger than the optimal (minimal GCV) model size M*. Choosing the predetermined complexity of the model as more than 2M* is enough in general . In our case, it was equal to 3025 and this value is called Mmax.
2.1. ANOVA decomposition of the MARS model
It is possible to analyse a MARS model using surface plots that visualise the interactions and effects between the basis functions. To illustrate this, some definitions will be introduced.
Let be the set of all single variable basis functions, i.e., basis functions that contain only . Similarly, let be the set of all two-variable basis functions that contain the pairs of variables and , and the set of all three-variable basis functions that contain the triplets of variables , and . The MARS model can be rewritten as in Eq. (7):Equation (7) is called the ANOVA decomposition due to its similarity to the decomposition by ANOVA of experimental design . The two-variables interaction of a MARS model, , is given by Eq. (8):
The graphical presentation of the ANOVA decomposition facilitates the interpretation of the MARS model. The effect of a one-variable basis function can be viewed by plotting against . Two-variable interaction can be viewed by plotting against and in a surface plot.
2.2. Prediction ability of the MARS model
The prediction ability of the MARS model can be evaluated in terms of the Root Mean Squared Error of Cross-Validation (RMSECV) and the squared leave-one-out correlation coefficient (). To compute RMSECV, one object is left out from the data set and the model is constructed for the remaining n-1 objects. Then the model is used to predict the value for the object left out. When all objects have been left out once, RMSECV is given by Eq. (9):
The value of is given as in Eq. (10):
3. Nonlinear behavior of the DM
The behavior of the facesheet of our DM is non-linear, as it is in MEMS DMs . The simplest approach of a linear combination of individual actuator deformations produces large errors when trying to represent the final shape of the facesheet. An example of this non-linearity is presented in Fig. 2 : we experiment with a sector of 9 x 5 actuators, which are centrally located on the DM. We applied a half-range offset to all actuators to place them at a nominal condition for an AO system and then we exercise each actuator to +12 volts with respect to this offset. The data was taken with our interferometer (please see section 5 for details on our experimental setup), whilst subtracting the half-range offset reference.
We obtained the sum of all individual actuators (top-left panel) and then we applied +12 volts to all actuators at once (“joint poke”, top-right panel). We evaluated the difference between both results, as an image (bottom-left panel) and as a slice of that image (bottom-right panel). This simple exercise shows that the sum of actuators produces an excessive result with respect to the joint poke of about 200 nm. Therefore, modeling the final shape of a DM is not as simple as combining the individual contributions of the actuators.
4. Actuator’s area of influence
Traditionally, the ‘influence function’ is the concept used to describe the shape of the DM facesheet around an actuator being poked. However, the final shape of the mirror surface depends on the positions of the neighbor actuators. Depending on the physical characteristics of the facesheet, this effect may or may not extend beyond adjacent neighbor actuators. Knowing the extension of this ‘area of influence’ is important for our purposes, since it determines the complexity of the model we are attempting to produce, because surrounding actuator positions (defined by their applied voltage) can ultimately affect the final position of a central actuator. For a complete characterization of the area of influence of an actuator, we chose to exercise all neighbor actuators and not to limit the study to the traditional influence function.
For this experiment, we poke a 5x5 actuator sector with random values, using similar exercising parameters to the ones used in the training data to feed our MARS model (see next section for details).
To determine the area of influence of an actuator in the presence of neighbor actuators being exercised, we computed the correlation coefficients matrix R (or ‘cross-correlation’), which is calculated as follows.
If C is the covariance matrix of the set of data, the cross correlation R is defined as in Eq. (11):
In practical terms, each row and column of R represents one actuator, and the values at positions (i,j) in R correspond to the correlation between actuators i and j Using this definition, it follows naturally that the diagonal elements of R will be unitary.
Figure 3 shows the results of this experiment, where the most important conclusion is that only the closest neighbors are significant when defining the final position of an actuator. The ‘area of influence’ of an actuator spans its immediate neighbors, in particular the closest ones (the ones which form a ‘cross’ together with the one under study). Counter-intuitively, the corner actuators do not contribute to the final shape of the central actuator, which might be related to the rigidity of the facesheet. The right panel in Fig. 3 compares the correlation coefficients with a normalized, traditional influence function for the actuator. The height of the continuous DM facesheet was measured only at actuator coordinates to make the comparison valid: this is why the plots appears not to be continuous. Figure 3 shows that correlation coefficients gives slightly different results with respect to the influence function, at one and two actuators away from the one under study. We believe this method is better suited to describe the trul behavior of the DM membrane.
This result may be different for other types of deformable mirror (for instance, for MEMS DMs, which have a much thinner membrane), but the result of this section is useful in order to have a priori knowledge of the complexity of a DM model.
5. Data taking methodology
Our experimental setup consists of a commercial Twymann-Green interferometer (a ‘Fisba μPhse 2 HR’) coupled to a 100 mm telecentric lens, installed in front of the DM. The light source used by the interferometer is a temperature-stabilized 632 nm laser. The complete 75 mm DM aperture can be sampled by the interferometer, with each pixel corresponding to 0.2 x 0.2 mm of the DM pupil. The Z coordinate (DM membrane deformation) is measured by the interferometer and measured in Angstrom (Å).
The typical deformation we applied to the DM was around ± 27000 Å = ± 2700 nm = ± 2.7 μm. The maximum excursion we can measure is limited by the ability of the interferometer to unwrap the phase information from the interference fringes, which in turn is limited by the spatial resolution of the fringe sampling. This maximum excursion is reasonable for typical AO corrections . We operate the DM at half its range (Voffset = 50 volts), applying positive and negative voltages around this plateau voltage, in order to model the DM in accordance with a typical AO system. Our interferometer was calibrated for each run with this plateau position, which was subtracted automatically for all phase maps. This is why our data has a zero mean value, around which there are positive and negative excursions.
A phase map with the Fisba takes approximately 15 seconds to be computed, therefore data taking is a long process, usually spanning weeks. Throughout any given run, there were variations in the environment, which produced small long-term drifts in the measurements we were gathering. We settled on modeling the main central area of our deformable mirror (11 x 5 actuators), leaving two static and symmetric areas in the DM around the modeled area, which allows us to measure the drifts in the measured position of the DM surface. This method would still model a large area of the mirror (55 actuators), while overcoming a limitation in our measurement equipment. Figure 4 presents the area of study and the positions we are using in the surrounding area to measure these drifts. We obtained the actuator positions (within the interferometer phase map) by doing individual pokes for each actuator and recording its phase map. Then we measured a typical full-width at half-maximum (FWHM) of the influence function for the actuators and used this value as a parameter for a two-dimensional Gaussian influence function. The Gaussian was used with a least-squares fitting procedure to find the coordinates of each individual poke. The coordinates of the Gaussian are the ‘measured position’ of each actuator in phase map space (where X and Y coordinates are pixel numbers). Edge actuators have largely asymmetrical influence functions, unlike the rest of the actuators; therefore the fitting procedure with the Gaussian did not produce satisfactory results for the edge actuators. We decided to identify the actuator positions manually for the edge actuators, assuming they are located under the ‘tallest’ area of the influence function (where the gradient changes sign). As can be seen in Fig. 4, the position of these actuators do not comply with a square grid as the rest of actuators do: this is a result of the boundary conditions of the membrane; but it is not relevant that it is not a square grid, as long as we sample the height of the actuators at coordinates that respond to the different positions that the actuators can adopt.
In summary, we are sampling the Z coordinate of the phase maps at actuator positions and we use these values to represent the DM surface. We are also sampling six points of the surface in areas of the DM we left static, which we use to fit a plane that represents the position of the DM membrane not deformed by the actuators. The position of this plane accounts for any drifts throughout the run and is subtracted from the Z coordinate measured at actuator positions. We found this method to be simple and give consistent results. This is the reason one sees a tilted surrounding in Fig. 2, given the fact that the 9 x 5 actuators were poked one by one and it took time to complete the run. During this period, there were drifts in the measured value of the DM surface, which were compensated in the mirror data (this is why the aperture circle is not tilted), showing the effect of the compensation in the outer areas of the phase maps, which otherwise have a null value.
Our Xinetics DM is fed by high voltage amplifiers able to output 0 to 100 volts, for a 0 to 10 volts input. A typical voltage/stroke plot is presented in Fig. 5 .
Our MARS model receives 55 height (Z coordinate) inputs to produce 55 voltage outputs. We exercise the DM to its limits, in order to gather realistic data for an AO system: the DM is first ‘raised’ to half its range and from that point the 55 actuators are given random voltages, to shape the DM surface to a completely random phase. The range of the random values is the maximum permitted without damaging the surface of the mirror. For our Xinetics DM this is achieved by applying +/− 12 volts to the actuators on top of the raised half range height. This is similar to an AO system that needs to compensate positive and negative phases. As an example, Fig. 6 presents the first 9 random positions of one of our runs. It can be seen that there is an average value of 0 Angstrom (because the interferometer subtracts the purely raised mirror as its reference) and positive (redder) and negative (bluer) ‘bumps’.
6. Results and analysis
We trained our MARS model with 6000 random positions, such as the ones depicted in Fig. 5.
To test the ability of our model to generate the correct voltages to shape the facesheet, we experimented with two types of tests:
- • Qualitative test: we generate some arbitrary combination of Zernike polynomials and feed the model with the values of the combination of polynomials evaluated at actuator coordinates. The model outputs the voltages for the 55 actuators, which we run with the DM, measuring the surface shape with our interferometer
- • Quantitative test: we produced an additional 1000 new random positions with the same parameters as the ones used for training and feed the model with them. The model produces the predicted voltages to achieve such random shapes, which we run with the DM, acquiring phase maps.
We call them quantitative and qualitative tests because in the latter case, the ability of the DM to reproduce a certain Zernike polynomial is limited by its spatial resolution, incurring in what is usually known as ‘fitting error’, therefore we only present qualitative results. The quantitative case does not have that problem, allowing us to obtain a positioning or ‘GoTo’ error, or the difference in position between the desired shape and the achieved phase by the mirror. We decided on using purely random voltages to test our model to its limits, since the spatial frequency response of uniformly distributed random data is more stringent at high spatial frequencies compared to Kolmogorov turbulence for example.
Once the MARS model is calculated it can be implemented in any computer as a set of equations [19–21]. For an Intel Core 2 Duo, at 2,26GHZ, with 4GB of RAM (standard desktop computer), it takes 190 microseconds to perform a prediction for 55 actuators. Typical operation cycle of AO computers is 300-500 Hz (or 2-3 milliseconds), therefore our model would not add any significant delay to an AO computer.
The Zernike polynomials selected for Fig. 7 were chosen somewhat arbitrarily, but the selection criterion was to use Zernike that have more information along the Y-axis (where the majority of the actuators modeled are) and incorporate spatial frequencies that would be too high for this mirror to shape. The results of this experiment are clearly seen along the right panels of Fig. 7. For the first polynomial (at the top of the figure), the difference between theory and experiment is fairly small, but when increasing the order of the polynomial and thus the spatial bandwidth, the fitting error becomes significant, although the areas with limited spatial frequencies are still well modeled (see for instance the central part of the plot at the bottom-right panel). It is interesting to note that these results were obtained with our MARS model, which was never trained with Zernike polynomials. This allows us to establish that our training method is general and would be appropriate for a mirror in an AO system.
The positioning error or ‘GoTo’ error was obtained with a new 1000 trials random run. The MARS model was fed with the facesheet positions from this run and the voltages produced were used for a new run. The phase maps of both runs were subtracted (at actuator coordinates), to produce the results in Figs. 8 and 9 . Figure 8 shows the maxima and minima of each trial, to confirm the large span in the data. Figure 9 is the main result of this paper, presented as a GoTo error in Angstrom (RMS value for the 55 actuators) and as a percentage of the full-range of actuators excursion (from Fig. 8)
The mean value of the plot at the bottom panel of Fig. 9 is 1.2%, which we believe is a remarkable result for an electrostrictive (e.g. with non-negligible hysteresis) mirror such as our Xinetics .
We have presented a model to predict the voltages to apply to a DM in order to achieve a desired position on its facesheet, using a multivariate adaptive regression splines model. The model is purely mathematical and is trained using a data set of interferometric phase maps. Its performance was verified using a DM in real conditions, reaching a GoTo error of 1.2% of the full-range of actuator positions. This model has the benefit of not developing a physical model of the DM; therefore the DM modeling strategy can be the same, regardless of the type of mirror in operation. MARS produced a series of simple equations, involving only sums and multiplications – they are fast to compute, unlike an iterative solution, so it should not add any significant latency to a real-time AO computer.
We appreciate fruitful discussions with Prof. Ray Sharples and Dr. Tim Morris, from Durham University. We thank Prof. Robert Tyson for useful suggestions to an early draft of this paper. D. Guzman appreciates support from the Science and Technology Facilities Council (STFC), through the ‘Dorothy Hodgkin’ postgraduate studentships program. This work was funded by STFC, grant PP/E007651/1.
References and links
1. F. Hammer, F. Sayede, E. Gendron, T. Fusco, D. Burgarella, V. Cayatte, J. M. Conan, F. Courbin, H. Flores, I. Guinouard, L. Jocou, A. Lancon, G. Monnet, M. Mouhcine, F. Rigaud, D. Rouan, G. Rousset, V. Buat, and F. Zamkotsian, “The FALCON Concept: Multi-Object Spectroscopy Combined with MCAO in Near-IR,” Proc. ESO Workshop (2002).
2. F. Assémat, E. Gendron, and F. Hammer, “The FALCON concept: multi-object adaptive optics and atmospheric tomography for integral field spectroscopy - principles and performance on an 8-m telescope,” Mon. Not. R. Astron. Soc. 376(1), 287–312 (2007). [CrossRef]
3. D. Guzmán, A. Guesalaga, R. Myers, R. Sharples, T. Morris, A. Basden, C. Saunter, N. Dipper, L. Young, L. Rodríguez, M. Reyes, and Y. Martin, “Deformable mirror controller for open-loop adaptive optics” Proc. SPIE 7015, 70153X–70153X–12 (2008).
4. J. Friedman, “Multivariate adaptive regression splines,” Ann. Stat. 19(1), 1–67 (1991). [CrossRef]
5. C. Hom, P. Dean, and S. Winzer, “Simulating electrostrictive DM: I nonlinear static analysis,” Smart Mater. Struct. 8(5), 691–699 (1999). [CrossRef]
6. D. Andersen, M. Fischer, R. Conan, M. Fletcher, and J. P. Veran, “VOLT: the Victoria Open Loop Testbed” Proc. SPIE 7015, 7015OH-7015OH-11 (2008).
7. E. Laag, D. Gavel, and M. Ammons, “Open-loop woofer-tweeter control on the LAO multi-conjugate adaptive optics testbed” in Adaptive optics for industry and medicine, C. Dainty. (Imperial College Press, 2008), pp. 143–148.
8. T. Bifano, P. Bierden, H. Zhu, S. Cornelissen, and J. Kim, “Megapixel wavefront correctors,” Proc. SPIE 5490, 1472–1481 (2004). [CrossRef]
9. C. Blain, O. Guyon, R. Conan, and C. Bradley, “Simple iterative method for open-loop control of MEMS deformable mirrors”, Proc. SPIE 7015, 701534–701534–8 (2008).
10. K. Morzinski, K. Harpsoe, D. Gavel, and M. Ammons, “The open-loop control of MEMS: modeling and experimental results”, Proc. SPIE 6467, 6467OG-6467OG-10 (2007).
11. J. Stewart, A. Diouf, Y. Zhou, and T. Bifano, “Open-loop control of a MEMS deformable mirror for large-amplitude wavefront control,” J. Opt. Soc. Am. A 24(12), 3827–3833 (2007). [CrossRef]
12. J. Hardy, “Wavefront Correctors” in Adaptive Optics for Astronomical Telescopes (Oxford 1998), pp. 176–212.
13. S. Sekulic and B. R. Kowalski, “MARS: a tutorial,” J. Chemometr. 6(4), 199–216 (1992). [CrossRef]
14. L. Breiman, J. H. Friedman, R. A. Olshen, and C. G. Stone, Classification and Regression Trees., Wadsworth International Group, Belmont, CA (1984)
15. Q.-S. Xu, M. Daszykowski, B. Walczak, F. Daeyaert, M. R. de Jonge, J. Heeres, L. M. H. Koymans, P. J. Lewi, H. M. Vinkers, P. A. Janssen, and D. L. Massart, “Multivariate adaptive regression splines - studies of HIV reverse transcriptase inhibitors,” Chemom. Intell. Lab. Syst. 72(1), 27–34 (2004). [CrossRef]
16. P. Craven and G. Wahba, “Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation,” Numer. Math. 31, 317–403 (1979).
17. D. L. Massart, B. Vandeginste, L. Buydens, S. De Jong, P. Lewi, and J. Smeyers-Verbeke, In: “Handbook of Chemometrics and Qualimetrics” vol. 20 A., Elsevier, Amsterdam (1997)
18. J. W. Evans, B. Macintosh, L. Poyneer, K. Morzinski, S. Severson, D. Dillon, D. Gavel, and L. Reza, “Demonstrating sub-nm closed loop MEMS flattening,” Opt. Express 14(12), 5558–5570 (2006). [CrossRef] [PubMed]
19. Y.F. Li, S.H. Ng, M. Xie, T.N. Goh. “A systematic comparison of metamodeling techniques for simulation optimization in Decision Support Systems”. Applied Soft Computing, In Press, Corrected Proof, Available online 24 December 2009. doi:10.1016/j.asoc.2009.11.034
20. M. Carlin, T. Kavli, and B. Lillekjendlie, “A comparison of four methods for non-linear data modelling,” Chemom. Intell. Lab. Syst. 23(1), 163–177 (1994). [CrossRef]
21. E. Deconinck, M. H. Zhang, F. Petitet, E. Dubus, I. Ijjaali, D. Coomans, and Y. Vander Heyden, “Boosted regression trees, multivariate adaptive regression splines and their two-step combinations with multiple linear regression or partial least squares to predict blood–brain barrier passage: A case study,” Anal. Chim. Acta 609(1), 13–23 (2008). [CrossRef] [PubMed]
22. B. R. Oppenheimer, D. Palmer, R. Dekany, A. Sivaramakrishnan, M. Ealey, and T. Price, “Investigating a Xinetics Inc. deformable mirror,” Proc. SPIE 3126, 569–579 (1997). [CrossRef]