Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group
Journal Home About Tutorials Issues in Progress Current Issue All Issues Early Posting Feature Issues

Reproducing Kernel Hilbert spaces for wave optics: tutorial

Franco Gori and Rosario Martínez-Herrero

Open Access Open Access

Abstract

An introduction to the Hilbert spaces that are endowed with a reproducing kernel is presented on using the mathematical tools of Fourier optics and coherence theory. After giving the basic definition of such spaces, some examples are worked out to show how the inner product can take different forms depending on the particular function space one works with. The basic rule to build a reproducing kernel Hilbert space (RKHS) is then presented together with the basic properties of those spaces. Eigenfunctions and eigenvalues of the reproducing kernel are then illustrated and lead to the important integral representation of the reproducing kernel. The latter is used to present pseudomodal expansions and generalized forms of sampling. The concluding section offers some thoughts on the applications of RKHSs in wave optics. An appendix presents an introduction to treatments using more advanced concepts of functional analysis.

© 2021 Optical Society of America

1. INTRODUCTION

In recent years, concepts and methods derived from the theory of reproducing kernel Hilbert spaces (RKHSs) have found applications in coherence theory as well as in coherent optical processing. The theory of RKHSs itself though remained scarcely appreciated. A better knowledge of such a theory would help to widen horizons and to suggest further applications. Before delving into specific textbooks on RKHSs, a scientist working in optics can be aided by an introduction where the basic concepts are presented, keeping the mathematics at a level comparable to those needed for Fourier optics and coherence theory. The tutorial to follow aims at this kind of presentation. Throughout the paper, examples are given (generally, in the last part of each section) to help readers get a better grasp of the subject. Introductory elements of functional analysis are collected in a dedicated appendix as an aid for readers interested in deepening their understanding of the subject with more advanced papers.

2. SOME FAMILIAR CASES

Before defining RKHSs, it is worthwhile to discuss some cases of these spaces that are well known by virtue of Fourier optics [1,2] and coherence theory [3]. Let us begin with the Hilbert space, say, ${\cal H}$, of Fourier transformable band-limited functions. Denoting by $f(x)$ a typical function of this set and by $\tilde f(\nu)$, or by ${\cal F}\{f(\cdot)\} (\nu)$, its Fourier transform, i.e.,

$${\cal F}\{f(\cdot)\} (\nu) = \tilde f(\nu) = \int_{- \infty}^\infty f(x)\exp (- 2\pi i\nu x){\rm d}x,$$
the characterizing property of $f$ is
$$\tilde f(\nu) = 0 \quad {\rm{for}}\;\;|\nu | \ge {\nu _M} \gt 0,$$
meaning that $\tilde f(\nu)$ can be different from zero only for ${-}{\nu _M} \lt \nu \lt {\nu _M}$. The following identity then holds,
$$\tilde f(\nu)\;{\rm{rect}}\left({\frac{\nu}{{2{\nu _M}}}} \right) = \tilde f(\nu),$$
where rect($t$) = 1 if $|t| \le 1/2$, = 0 otherwise. Performing the inverse Fourier transform, namely,
$$f(x) = \int_{- \infty}^\infty \tilde f(\nu)\exp (2\pi i\nu x){\rm d}\nu ,$$
on both sides of Eq. (3), and using the convolution theorem [1], we obtain
$$2{\nu _M}\int_{- \infty}^\infty f(y)\;{{\rm sinc}}[2{\nu _M}(x - y)]{\rm d}y = f(x),$$
where sinc is the even function
$${\rm{sinc}}(x) = \frac{{\sin (\pi x)}}{{\pi x}}.$$
This means that the convolution between any function $f(x) \in {\cal H}$ and the kernel $2{\nu _M}{\rm{sinc}}(2{\nu _M}x)$ reproduces $f(x)$. Let us use the notation
$$K(x,y) = 2{\nu _M}{\rm{sinc}}[2{\nu _M}(x - y)],$$
and assume the inner (or scalar) product of two typical functions $f(y)$ and $g(y)$ in ${\cal H}$, to be denoted by ${\langle f(\cdot),g(\cdot)\rangle _{\cal H}}$, has the well-known expression
$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \int_{- \infty}^\infty f(y){g^*}(y){\rm d}y,$$
where the star denotes the complex conjugate. Then, Eq. (5) can be written as
$${\langle f(\cdot),K(\cdot ,x)\rangle _{\cal H}} = f(x).$$
Accordingly, we can say that in the present ${\cal H}$ space there exists a reproducing kernel $K(x,y)$ such that Eq. (9) holds true. The above property sounds evident in the Fourier domain
$$\tilde f(\nu)\tilde K(\nu) = \tilde f(\nu),$$
which is again Eq. (3) written with a different symbol for rect. This is a first example of Hilbert space endowed with a reproducing kernel $K$. The characterizing property is expressed by Eq. (9).

Band-limited functions exhibit certain regularity features. For example, they cannot take on arbitrarily large values or vary at will passing from a point to another. In fact, the following upper bounds can be established [2] $\forall x$:

$$|f(x)| \le \sqrt {2{\nu _M}E} ,$$
$$|f(x + D) - f(x)| \le \sqrt {4{\nu _M}E[1 - {\rm{sinc}}(2{\nu _M}D)]} ,$$
where $D$ is a real constant and
$$E = \int_{- \infty}^\infty |f(x{)|^2}{\rm d}x.$$
Furthermore, for such functions the well-known sampling theorem holds [1,2].

As another example we can consider the set of trigonometric polynomials with a finite number of Fourier components. Assume the period to be $P$, and let the polynomials possess a number $2N + 1$ of Fourier components with indexes between ${-}N$ and $N$. In other words, consider the set of trigonometric polynomials expressed by the formula

$$f(x) = \sum\limits_{n = - N}^N {f_n}{e^{2\pi inx/P}},$$
where
$${f_n} = \frac{1}{P}\int_0^P f(x){e^{- 2\pi inx/P}}{\rm d}x.$$
The scalar product of two typical functions $f$ and $g$ over one period gives
$$\frac{1}{P}\int_0^P f(x){g^*}(x){\rm d}x = \sum\limits_{n = - N}^N {f_n}g_n^*,$$
where the last expression is obtained on writing $f$ and $g$ as sums of the form in Eq. (14) and exploiting the orthonormality of the exponential functions. With this scalar product, the set of functions specified by Eq. (14) constitutes a RKHS having a reproducing kernel given by the Dirichlet kernel [4]
$$K(x - y) = \frac{{\sin [\pi (2N + 1)(x - y)/P]}}{{\sin [\pi (x - y)/P]}},$$
well known from the theory of Fourier series. The check that $K$ is a reproducing kernel is easily performed by taking into account the following expansion,
$$\frac{{\sin [\pi (2N + 1)t]}}{{\sin (\pi t)}} = \sum\limits_{n = - N}^N {e^{2\pi {\rm{int}}}},$$
and using again the orthonormality of the exponential functions.

Our last example deals with the well-known system of orthonormal polynomials ${P_n}(x)$ with $n = 0,1,\ldots$ defined on a set $X$ (e.g.,  Jacobi, Hermite, Laguerre) [5,6]. A typical polynomial $p(x)$ of order $N$ can be expressed as a linear combination of ${P_n}$ with ($n = 0,1,\ldots,N$), i.e.,

$$p(x) = \sum\limits_{n = 0}^N {c_n}{P_n}(x),$$
where the typical ${c_m}$ coefficient is evaluated as the inner product of $p(x)$ with the $m$th polynomial ${P_m}(x)$, defined as [5,6]
$$\begin{split}{c_m}& = {{{\langle p(\cdot),{P_m}(\cdot)\rangle}_{\cal H}} }\\&={\int_X p(x){P_m}(x)w(x){\rm d}x,\quad (m = 0,1,\ldots,N),}\end{split}$$
where $w(x) \ge 0$ is the weight function. Note that both $X$ and $w(x)$ depend on the chosen system of polynomials.

On inserting Eq. (19) into Eq. (20), and exploiting the orthonormality of the ${P_n}$, we have

$$\begin{split}&{\int_X p(x){P_m}(x)w(x){\rm d}x}\\ &\; ={\sum\limits_{n = 0}^N {c_n}\int_X {P_n}(x){P_m}(x)w(x){\rm d}x = {c_m},\quad(m = 0,1,\ldots,N),}\end{split}$$
as we wanted to prove.

Let us now define the kernel

$${K_N}(x,y) = \sum\limits_{n = 0}^N {P_n}(x){P_n}(y)$$
and evaluate the inner product
$$\begin{split}&{\int_X p(x){K_N}(x,y)w(x){\rm d}x}\\&\; ={\sum\limits_{n = 0}^N {P_n}(y)\int_X p(x){P_n}(x)w(x){\rm d}x = \sum\limits_{n = 0}^N {c_n}{P_n}(y) = p(y).}\end{split}$$
We then see that Eq. (22) is the reproducing kernel for this space of polynomials, in that the inner product between $p(\cdot)$ and ${K_N}(\cdot ,y)$ reproduces $p(y)$.

It is worthwhile to note that the kernel ${K_N}(x,y)$ can be given a closed form through the Christoffel–Darboux formula [5,7,8]

$${{K_N}(x,y) = \frac{{{a_N}}}{{{a_{N + 1}}}}\frac{{{P_{N + 1}}(x){P_N}(y) - {P_{N + 1}}(y){P_N}(x)}}{{x - y}},}$$
where ${a_N}$ and ${a_{N + 1}}$ are the coefficients of the highest $x$ power in the polynomials ${P_N}$ and ${P_{N + 1}}$, respectively.

Each of the previous examples specifies a Hilbert space endowed with its own reproducing kernel and its own inner product.

3. DEFINING REPRODUCING KERNEL HILBERT SPACES

We now give the definition of a RKHS, meaning a Hilbert space endowed with a reproducing kernel. Let ${\cal H}$ be a Hilbert space formed by functions, say, $f(x)$, defined on a set $X$. (From now on we shall follow the convention of mathematical papers on reproducing kernels where scalar symbols, like $x,y,\ldots$ may even refer to vectorial quantities. Furthermore, dimensional parameters will be omitted except for some examples.) It is then defined an inner or scalar product between two typical elements $f$ and $g$. It will be denoted by

$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}},$$
and it will induce a norm
$$\Vert f \Vert= {\sqrt {\langle f(\cdot),f(\cdot)\rangle} _{\cal H}}.$$
It is worthwhile to recall that the properties of the inner product are the following (using a synthetic notation and assuming they hold $\forall f,g,h \in {\cal H}$, $\forall \alpha ,\beta \in {\mathbb C}$):
$${\langle f,g\rangle _{\cal H}} = \langle g,f\rangle _{\cal H}^*,$$
$${\langle \alpha f + \beta g,h\rangle _{\cal H}} = \alpha {\langle f,h\rangle _{\cal H}} + \beta {\langle g,h\rangle _{\cal H}},$$
$${\langle f,f\rangle _{\cal H}} \ge 0,$$
$${\langle f,f\rangle _{\cal H}} = 0\;{\rm{iff}} f = 0.$$
Notice that from Eqs. (27) and (28) it follows that
$$\begin{split}{{\langle f,\alpha g + \beta h\rangle}_{\cal H}} &= \langle \alpha g + \beta h,f\rangle _{\cal H}^* \\&={{\alpha ^*}\langle f,g\rangle + {\beta ^*}\langle f,h\rangle .}\end{split}$$

The space is said to be a RKHS [4,7,9–12] if there exists a kernel $K(x,y)$, defined over $X \times X$, such that

$$A)\;\;\forall y \in X\; \Rightarrow \;K(\cdot ,y) \in {\cal H};$$
$$B)\;\;\forall f \in {\cal H}\quad{\rm{and}}\quad \forall y \in X\; \Rightarrow \;{\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}} = f(y).$$
The kernel $K$ is called the reproducing kernel. Property $B)$ accounts for the name given to ${\cal H}$. We see that, in the spaces under consideration, $K$ plays a role similar to that of a Dirac delta function. One may wonder whether the Dirac delta itself can be used as a reproducing kernel. The answer is negative because of condition $A)$. In fact, $K(\cdot ,x)$ must be a function belonging to ${\cal H}$, whereas $\delta$ is a distribution [4,7]. (Extensions to theories where the $\delta$ can be included exist [7].)

By virtue of condition $A)$, $f(\cdot)$ can be replaced by $K(\cdot ,x)$ so that, using condition $B)$, we have

$${\langle K(\cdot ,x),K(\cdot ,y)\rangle _{\cal H}} = K(y,x).$$
In particular, for $x = y$ we obtain
$${\langle K(\cdot ,x),K(\cdot ,x)\rangle _{\cal H}} = \Vert K(\cdot ,x{) \Vert ^2} = K(x,x).$$
Note that if $c$ is a complex constant, we have
$${\langle cK(\cdot ,x),K(\cdot ,y)\rangle _{\cal H}} = cK(y,x)$$
and
$${\langle K(\cdot ,x),cK(\cdot ,y)\rangle _{\cal H}} = {c^*}K(y,x).$$
Furthermore, we have [see Eq. (27)]
$${\langle K(\cdot ,x),K(\cdot ,y)\rangle _{\cal H}} = [{\langle K(\cdot ,y),K(\cdot ,x)\rangle _{\cal H}}{]^*},$$
from which [see Eq. (34)] the relation follows
$$K(y,x) = K{(x,y)^*},$$
which shows that a reproducing kernel is Hermitian (in the real case, symmetric).

Let us now show that the reproducing kernel is non-negative definite, i.e., that once fixed at an arbitrary number $N$ of points ${x_1},{x_2},\ldots,{x_N}$ and of complex constants ${c_1},{c_2},\ldots,{c_N}$, the quadratic form $Q$, defined as follows,

$$Q = \sum\limits_{i = 1}^N \sum\limits_{j = 1}^N K({x_i},{x_j})c_i^*{c_j},$$
satisfies the condition
$$Q \ge 0,$$
or equivalently that the matrix $\{K({x_i},{x_j})\} _{i,j = 1}^N$, known as the Gram matrix [4], is non-negative. In fact, on exploiting Eq. (34), we can write
$$Q = \sum\limits_{i = 1}^N \sum\limits_{j = 1}^N c_i^*{c_j}{\langle K(\cdot ,{x_j}),K(\cdot ,{x_i})\rangle _{\cal H}},$$
and, on recalling Eqs. (36) and (37),
$$\begin{split}Q &= {{{\left\langle {\sum\limits_{j = 1}^N {c_j}K(\cdot ,{x_j}),\sum\limits_{i = 1}^N {c_i}K(\cdot ,{x_i})} \right\rangle}_{\cal H}} }\\&={{{\left| {\left| {\sum\limits_{i = 1}^N {c_i}K(\cdot ,{x_i})} \right|} \right|}^2} \ge 0.}\end{split}$$
In particular, for $N = 1,{c_1} = 1$, and ${x_1} = x$, Eqs. (40) and (41) give [in agreement with Eq. (35)]
$$K(x,x) \ge 0,$$
so that the kernel $K$ takes on non-negative values along the so-called principal diagonal.

A fundamental theorem is that of Moore–Aronszajn [7,9,10]. The theorem states that to any RKHS a non-negative definite reproducing kernel is associated (as we have just seen), and, conversely, that to any non-negative definite kernel $K$ we can associate a RKHS for which $K$ is the reproducing kernel.

Something has to be added about terminology. Many authors use the term positive definite for a kernel satisfying Eq. (41) and specify strictly positive definite if Eq. (41) holds for inequality only. Others speak of a positive semi-definite kernel, limiting the use of positive definite to the case in which Eq. (41) holds for inequality only. In this paper, we shall use the term non-negative when Eq. (41) holds and the term positive if Eq. (41) holds for inequality only.

4. EXAMPLES

Before considering general properties of RKHSs, it is worthwhile to see a few examples.

A. Shift-Invariant Kernels

Let us consider a typical shift-invariant kernel $K(x - y)$. Particular cases were already considered in Section 2. We shall now refer to a more general case. We recall Bochner’s theorem [13], according to which a function $K(x - y)$ is non-negative definite if and only if the Fourier transform of $K(\cdot)$ is real non-negative. For simplicity, we assume $\tilde K(\nu)$ to be positive within a given interval (possibly of infinite extent). In general, Eq. (10) does not hold, and $K$ is not a reproducing kernel. However, by virtue of Moore–Aronszajn theorem, a reproducing kernel exists. We can find it on exploiting the definition of the inner product. To this end we define the inner product of two Fourier transformable functions $f(x)$ and $g(x)$ as follows:

$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \int \frac{{\tilde f(\nu){{\tilde g}^*}(\nu)}}{{\tilde K(\nu)}}{\rm d}\nu ,$$
where the integration interval will be chosen by taking the extension of $\tilde K$ into account. In particular, for band-limited functions, the integration interval will coincide with the band extent. It can be seen [4,7,10] that the above integral defines a bona fide inner product. The space ${\cal H}$ of functions $f(x)$, for which
$$\Vert f{\Vert ^2} = {\langle f(\cdot),f(\cdot)\rangle _{\cal H}} = \int \frac{{|\tilde f(\nu {{)|}^2}}}{{\tilde K(\nu)}}{\rm d}\nu \lt + \infty ,$$
is a RKHS. In fact, thanks to the properties of the Fourier transform (FT) [1,2], we have
$${\cal F}\{K(\cdot - y)\} (\nu) = \tilde K(\nu){e^{- 2\pi i\nu y}}.$$
If we now evaluate the inner product (in ${\cal H}$) of $f(\cdot)$ and $K(\cdot - y)$ by using Eqs. (45) and (47), we obtain
$${\langle f(\cdot),K(\cdot - y)\rangle _{\cal H}} = \int \frac{{\tilde f(\nu)\tilde K(\nu){e^{2\pi i\nu y}}}}{{\tilde K(\nu)}}{\rm d} \nu ,$$
where the reality of $\tilde K$ was taken into account. By virtue of Eq. (4) the last integral is exactly $f(y)$. Hence the reproduction property is satisfied.

For a simple example, let us consider the kernel

$$K(x - y) = \frac{1}{2}{e^{- |x - y|}},$$
whose FT is the non-negative function [1,2]
$$\tilde K(\nu) = \frac{1}{{1 + 4{\pi ^2}{\nu ^2}}}.$$
According to Eq. (45), the inner product of two functions $f$ and $g$ is then
$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \int_{- \infty}^\infty \tilde f(\nu){\tilde g^*}(\nu)(1 + 4{\pi ^2}{\nu ^2}){\rm d}\nu .$$
Now it is enough to remember that $2\pi i\nu \tilde f(\nu)$ is the FT of the first derivative $f^\prime (x)$ [1,2] to see that Eq. (51) gives, in the $x$ domain,
$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \int_{- \infty}^\infty [f(x){g^*}(x) + f^\prime (x){g^{\prime*}}(x)]{\rm d}x.$$

B. Real Functions in [0,1]

Let us consider the set of real functions that are continuous in the interval [0,1], vanishing at the origin and endowed with continuous first derivative in [0,1] (possibly except for a finite number of points). We assume that on such a set the following kernel is acting,

$$K(x,y) = {\min}(x,y),$$
where min($x,y$) denotes the smaller value between $x$ and $y$. For any fixed $y$, the graph of $K$ versus $x$ is the bisecting line of the first quadrant for $x \le y$ and becomes a line parallel to the $x$ axis as soon as $x \ge y$. We want to find the expression of the inner product. More explicitly, we consider the expression
$$\begin{split}&{{{\langle f(\cdot),K(\cdot ,y)\rangle}_{\cal H}} }\\&\quad ={\int_0^1 U\{f(x)\} U\{K(x,y)\} {\rm d}x = f(y),}\end{split}$$
where $U$ denotes a suitable operation performed on the function in curly braces. Notice that, by virtue of the hypothesis about $f(0)$, the last side of previous equation can be written
$$\begin{split}{f(y) - f(0) = \int_0^y f^\prime (x){\rm d}x.}\end{split}$$

Since the derivative of $K$ with respect to $x$ equals 1 for $0 \le x \le y$ and 0 elsewhere, we easily conclude that $U$ is nothing else than the first derivative. Then the inner product between two functions $f$ and $g$ has to be defined as

$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \int_0^1 f^\prime (x)g^\prime (x){\rm d}x.$$
Again, it could be proved that Eq. (56) defines a bona fide inner product.

For a further example, we take the space of real continuous functions defined in the interval [0,1], there possessing a continuous first derivative except at a finite number of points and vanishing at the end points. Using again Eq. (56) as a definition of inner product, it is easy to check that the space ${\cal H}$ thus obtained is a RKHS, whose reproducing kernel is given by

$$K(x,y) = {\min}(x,y) - xy.$$
This kernel equals $x(1 - y)$ for $x \le y$ and $y(1 - x)$ for $x \ge y$. Then, for any $y$, the graph versus $x$ has a triangular shape. For this reason, the kernel is called triangular [14].

C. Szegö’s Kernel

Both this example and the one to follow show the use of reproducing kernels for functions of complex variables. Let $f(z)$ and $g(z)$ be two functions of the complex variable $z = r{e^{{i\theta}}}$ representable as a convergent power series for $r \le 1$:

$$f(z) = \sum\limits_{n = 0}^\infty {a_n}{z^n}, \quad g(z) = \sum\limits_{n = 0}^\infty {b_n}{z^n}.$$
Let us define their inner product as the integral along the circle $r = 1$:
$$\begin{split}{\langle f,g\rangle = \frac{1}{{2\pi}}\sum\limits_{n = 0}^\infty \sum\limits_{m = 0}^\infty {a_n}b_m^*\int_0^{2\pi} {e^{i(n - m)\theta}}{\rm d}\theta = \sum\limits_{n = 0}^\infty {a_n}b_n^*.}\end{split}$$
We want to find the expression of the reproducing kernel $K$ (if any). To this aim we note that on computing the inner product between $f$ and $K$ we have to find
$${\langle f({z_1}),K({z_1},{z_2})\rangle _{\cal H}} = f({z_2}) = \sum\limits_{n = 0}^\infty {a_n}z_2^n.$$
As a function of ${z_1}$, $K$ will have the form [see Eq. (58)]
$$K({z_1},{z_2}) = \sum\limits_{n = 0}^\infty {b_n}({z_2})z_1^n.$$
According to Eq. (59), we have
$$\langle f({z_1}),K({z_1},{z_2})\rangle = \sum\limits_{n = 0}^\infty {a_n}b_n^*({z_2}).$$
Equaling the last sides of Eqs. (60) and (62), we find
$$\sum\limits_{n = 0}^\infty {a_n}z_2^n = \sum\limits_{n = 0}^\infty {a_n}b_n^*({z_2}),$$
from which we derive
$${b_n}({z_2}) = (z_2^*{)^n}.$$
On inserting this result into Eq. (61), we obtain
$$K({z_1},{z_2}) = \sum\limits_{n = 0}^\infty z_1^n{(z_2^*)^n}.$$

Since ${z_1},{z_2}$ are on the unit disk of the complex plane, the sum defining $K$ converges to

$$K({z_1},{z_2}) = \frac{1}{{1 - {z_1}z_2^*}},$$
which is Szegö’s kernel [4].

D. Bergman’s Kernel

Let $f(z)$ and $g(z)$ be two functions of the complex variable $z = r{e^{{i\vartheta}}}$, expressible as the convergent series for $r \le 1$:

$$f(z) = \sum\limits_{n = 0}^\infty {a_n}{z^n}; \quad g(z) = \sum\limits_{n = 0}^\infty {b_n}{z^n}.$$
We define the inner product as follows:
$$\begin{split}\langle f(z),g(z)\rangle &= \int f(z){g^*}(z){{\rm d}^2}z\\&={\sum\limits_{n = 0}^\infty \sum\limits_{m = 0}^\infty {a_n}b_m^*\int_0^1 \int_0^{2\pi} {r^{n + m}}{e^{i(n - m)\vartheta}}r{\rm d}r{\rm d}\vartheta }\\&={\pi \sum\limits_{n = 0}^\infty {a_n}b_n^*\int_0^1 {t^n}{\rm d}t = \pi \sum\limits_{n = 0}^\infty \frac{{{a_n}b_n^*}}{{n + 1}}.}\end{split}$$
Let us now compute $\langle f({z_1}),K({z_1},{z_2})\rangle$, where $K({z_1},{z_2})$ as a function of ${z_1}$ is written
$$K({z_1},{z_2}) = \sum\limits_{n = 0}^\infty {k_n}({z_2})r_1^n{e^{in{\vartheta _1}}}.$$
Using Eq. (68) we then find
$$\langle f({z_1}),K({z_1},{z_2})\rangle = \pi \sum\limits_{n = 0}^\infty \frac{{{a_n}k_n^*({z_2})}}{{n + 1}},$$
and since this has to equal $f({z_2}) = \sum\nolimits_n {a_n}z_2^n$, we obtain
$${k_n}({z_2}) = \frac{{n + 1}}{\pi}r_2^n{e^{- in{\vartheta _2}}}.$$
On inserting Eq. (71) into Eq. (69), we obtain
$$K({z_1},{z_2}) = \frac{1}{\pi}\sum\limits_{n = 0}^\infty (n + 1)z_1^nz_2^{*n} = \frac{1}{\pi}\frac{1}{{{{(1 - {z_1}z_2^*)}^2}}},$$
where we exploited knowledge of the sum $\sum\nolimits_{n = 0}^\infty (n + 1){x^n} = 1/(1 - x{)^2}$ for $|x| \lt 1$. This is the closed-form expression of Bergman’s kernel on the unit disk of the complex plane [4].

5. CONSTRUCTION OF RKHSS

Apart from the ad hoc procedures exemplified in Section 4, a general problem to be faced is: Given a non-negative definite kernel $K(x,y)$, how do we construct the associated RKHS?

Let us present the standard approach of Aronszajn based on linear combinations of kernel functions. Take $N$ points ${x_1},{x_2}, \ldots ,{x_N}$ and $N$ complex constants ${a_1},{a_2}, \ldots ,{a_N}$, where $N$ is an integer. Consider now functions of the form

$$f(x) = \sum\limits_{n = 1}^N {a_n}K(x,{x_n}).$$
The inner product of two functions $f$ and $g$ (with coefficients ${b_n}$) is, by virtue of the linearity of the inner product,
$$\begin{split}&{{{\langle f(x),g(y)\rangle}_{\cal H}}}\\&\quad ={\sum\limits_{n = 1}^N \sum\limits_{m = 1}^N {a_n}b_m^*{{\langle K(x,{x_n}),K(x,{x_m})\rangle}_{\cal H}}}\\&\quad ={\sum\limits_{n = 1}^N \sum\limits_{m = 1}^N {a_n}b_m^*K({x_m},{x_n}),}\end{split}$$
where we used the reproducing property that $K$ must possess. With this inner product, $K$ behaves as reproducing kernel for any function of the form in Eq. (73). In fact, we have
$$\begin{split}{{\langle f(\cdot),K(\cdot ,x)\rangle}_{\cal H}}& = {\sum\limits_{n = 1}^N {a_n}{{\langle K(\cdot ,{x_n}),K(\cdot ,x)\rangle}_{\cal H}}}\\&={\sum\limits_{n = 1}^N {a_n}K(x,{x_n}) = f(x).}\end{split}$$

The space built in this way is not complete [9] (it is called a pre-Hilbertian space). It is dense, though [9], in the Hilbert space obtained by completion on adding the limits of the Cauchy sequences converging in norm and taking into account that, as will be seen in Section 6, the latter implies pointwise convergence.

Let us pause for a moment to put into evidence that the construction of a RKHS by means of a combination of kernels centered at distinct points, as in Eq. (73), gives an idea of the regularity properties of the functions belonging to the RKHS. Apparently, they are the same as those of the kernel itself. For a limiting example, let us refer to the kernel $K(x,y) = xy$ on the domain $[ 0,1 ] \times [ 0,1 ]$ whose non-negativeness is easily proved. It is clear that, no matter how the quantities ${a_n}$ and ${x_n}$ are chosen in Eq. (73), the resulting function will be of the form $ax$, where $a$ can possibly vanish. Hence, the RKHS associated to the present kernel includes one type of function only.

6. GENERAL PROPERTIES OF RKHSS

The examples of Section 4 could give the idea that we would have found different reproducing kernels on changing the search procedure. This is not the case. In fact, we now show that the reproducing kernel is unique. Arguing by contradiction, imagine that, in a typical RKHS endowed with a reproducing kernel $K$, another reproducing kernel, say, ${K_1}(x,y)$, exists. Then for some $y$ it should be

$$||K(\cdot ,y) - {K_1}(\cdot ,y{)||^2} \gt 0.$$
On the other hand, the left-hand side can be written
$$\begin{split}&{||K(\cdot ,y) - {K_1}(\cdot ,y{{)||}^2}}\\&\quad ={{{\langle K(\cdot ,y) - {K_1}(\cdot ,y),K(\cdot ,y) - {K_1}(\cdot ,y)\rangle}_{\cal H}},}\end{split}$$
and, for the linearity of the inner product, this gives
$$\begin{split}&{||K(\cdot ,y) - {K_1}(\cdot ,y{{)||}^2}}\\&\quad ={{{\langle K - {K_1},K\rangle}_{\cal H}} - {{\langle K - {K_1},{K_1}\rangle}_{\cal H}}.}\end{split}$$
The right-hand side is zero because both $K$ and ${K_1}$ are assumed to be reproducing kernels. Therefore, for any $y$, we have $||K(\cdot ,y) - {K_1}(\cdot ,y)|| = 0$, from which $K = {K_1}$ follows. (Remember that a norm is zero only for the zero vector.)

Let us now recall that in any Hilbert space the Cauchy–Schwarz inequality holds:

$$|{\langle f(\cdot),g(\cdot)\rangle _{\cal H}}| \le ||f||\;||g||.$$
On applying this inequality to the reproduction relation from Eq. (33) and using Eq. (35), we obtain
$$|f(y)| = |{\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}}| \le ||f||\sqrt {K(y,y)} ,$$
which shows that the values of $|f(y)|$ are limited by the norm of $f$ and by the values of $K$ along the principal diagonal.

Let us note that Eq. (80) entails $f(y) \equiv 0$ if $||f|| = 0$. We further note that on replacing $f$ with $K$ in Eq. (80) we have

$$\begin{split}&{|K(x,y)| }\\&\quad ={|{{\langle K(\cdot ,y),K(\cdot ,x)\rangle}_{\cal H}}| \le \sqrt {K(x,x)K(y,y)} .}\end{split}$$

Let us add a property about convergence in RKHSs. Let ${f_n}(\cdot)$ be a sequence that converges in norm. On applying the reproduction relation and inequality from Eq. (79), we find for a typical pair $n, m$

$$\begin{split}|{f_n}(x) - {f_m}(x)| &= {|{{\langle {f_n}(\cdot) - {f_m}(\cdot),K(\cdot ,x)\rangle}_{\cal H}}|}\\& \le{||K(\cdot ,x)|| ||{f_n} - {f_m}||,}\end{split}$$
or, thanks to Eq. (35),
$$|{f_n}(x) - {f_m}(x)| \le \sqrt {K(x,x)} ||{f_n} - {f_m}||.$$
Therefore, if $||{f_n} - {f_m}||$ tends to zero, the same occurs to $|{f_n}(x) - {f_m}(x)|$, $\forall x$. Hence, in RKHSs, convergence in norm implies pointwise convergence.

Let us finally show that the variations of a typical function belonging to a RKHS are limited by the kernel’s features. To this end, we consider the modulus of the difference of the values of $f$ at two points, say, $x$ and $y$. Taking the reproduction property into account we can write

$$\begin{split}|f(x) - f(y)| &={ |{{\langle f(\cdot),K(\cdot ,x)\rangle}_{\cal H}} - {{\langle f(\cdot),K(\cdot ,y)\rangle}_{\cal H}}|}\\&={|{{\langle f(\cdot),K(\cdot ,x) - K(\cdot ,y)\rangle}_{\cal H}}|,}\end{split}$$
from which by Schwarz inequality [Eq. (79)] we obtain
$$|f(x) - f(y)| \le ||f|| ||K(\cdot ,x) - K(\cdot ,y)||.$$
We now have
$$\begin{split}&{||K(\cdot ,x) - K(\cdot ,y{{)||}^2}}\\&\quad={{{\langle K(\cdot ,x) - K(\cdot ,y),K(\cdot ,x) - K(\cdot ,y)\rangle}_{\cal H}},}\end{split}$$
and hence
$$\begin{split}&{||K(\cdot ,x) - K(\cdot ,y{{)||}^2}}\\&\quad ={{{\langle K(\cdot ,x),K(\cdot ,x)\rangle}_{\cal H}} + {{\langle K(\cdot ,y),K(\cdot ,y)\rangle}_{\cal H}}}\\&\quad -{2\Re \{{{\langle K(\cdot ,x),K(\cdot ,y)\rangle}_{\cal H}}\} ,}\end{split}$$
or by the reproduction property from Eq. (33),
$$||K(\cdot ,x) - K(\cdot ,y{)||^2} = K(x,x) + K(y,y) - 2\Re \{K(x,y)\} .$$
In conclusion, the modulus of the difference $f(x) - f(y)$ is limited by
$$|f(x) - f(y)| \le ||f||\;d(x,y),$$
where we used the quantity
$$d(x,y) = \sqrt {K(x,x) + K(y,y) - 2\Re \{K(x,y)\}} ,$$
which is a feature of the pertinent Hilbert space independent of $f$.

The inequalities from Eqs. (80) and (89) generalize to any RKHS the limitations from Eqs. (11) and (12) that held specifically for band-limited functions.

It is rather spontaneous to wonder whether different RKs can be somehow combined to give new kernels. The simplest of such operations is the sum. The following theorem holds true [4]: “Let ${K_1}$ and ${K_2}$ be RKs for spaces ${{\cal H}_1}$ and ${{\cal H}_2}$, respectively, of functions on $X$ with corresponding norms ${||.||_{{{\cal H}_1}}}$ and ${||.||_{{{\cal H}_2}}}$. Then $K = {K_1} + {K_2}$ is the RK of the space of functions $f = {f_1} + {f_2}$ where ${f_1} \in {{\cal H}_1}$ and ${f_2} \in {{\cal H}_2}$ with the norm $||f||_{\cal H}^2 = {\min}\; (||{f_1}||_{{{\cal H}_1}}^2 + ||{f_2}||_{{{\cal H}_2}}^2$).”

The difference of two RKs is not necessarily a genuine RK, so a specific analysis is to be done case by case [15,16].

Lastly, a RK remains valid if multiplied by a constant and even if multiplied by a different (valid) RK.

The above hints can give an idea of how RKs can be extended by simple operations.

7. EIGENFUNCTIONS AND EIGENVALUES

Here we shall give the basis of an approach to RKHSs that differs from that used in Section 5.

The pertinent treatment is based on the homogeneous Fredholm integral equation of the second kind [13,14]:

$$\int_D \Phi (y)\;K(x,y){\rm d}y = \lambda \Phi (x),$$
where $D$ specifies the integration domain while $\Phi$ and $\lambda$ denote an eigenfunction and the corresponding eigenvalue. As we shall see such quantities play an important role in RKHSs.

Under suitable conditions, in particular if the following inequality,

$$\iint |K(x,y{)|^2}{\rm d}x{\rm d}y \lt + \infty,$$
is met, a discrete set of eigenfunctions ${\Phi _n}$ and eigenvalues ${\lambda _n}$ is found [13]. The eigenfunctions are (or can be made if degenerate eigenvalues exist) mutually orthogonal and normalized. Since the reproducing kernel $K$ is non-negative definite the eigenvalues ${\lambda _n}$ are non-negative. As a rule the eigenvalues are ordered in a non-increasing way.

A fundamental result is given by Mercer’s theorem [13], expressing the kernel $K$ as the following series in $x$ and $y$ (for the so-called Pincherle–Goursat kernels, the series is replaced by a sum [14]):

$$K(x,y) = \sum\limits_{n = 0}^\infty {\lambda _n}{\Phi _n}(x)\Phi _n^*(y),$$
where the convergence of the series is uniform with respect to both $x$ and $y$. Since the eigenfunctions are often interpreted as modes associated to $K$, the equality from Eq. (93) is also known as the modal expansion of the kernel.

Let us now discuss the role of eigenfunctions and eigenvalues in the inner product operation. Given a typical eigenfunction ${\Phi _j}$, the reproduction property holds (as for any function of the space),

$${\Phi _j}(y) = {\langle {\Phi _j}(\cdot),K(\cdot ,y)\rangle _{\cal H}},$$
or, according to Eq. (93),
$${\Phi _j}(y) = \sum\limits_n {\lambda _n}{\langle {\Phi _j}(\cdot),{\Phi _n}(\cdot)\Phi _n^*(y)\rangle _{\cal H}}.$$
Now, for any $n$ and any $y$, $\Phi _n^*(y)$ is a number. Then, it can be brought out of the scalar product sign by changing it into its complex conjugate. Equation (95) then becomes
$${\Phi _j}(y) = \sum\limits_n {\lambda _n}{\Phi _n}(y){\langle {\Phi _j}(\cdot),{\Phi _n}(\cdot)\rangle _{\cal H}}.$$
It is now sufficient to take the scalar product (in ordinary, ${L^2}$ sense) with a typical ${\Phi _k}(y)$ to obtain
$${\delta _{\textit{jk}}} = \sum\limits_n {\lambda _n}{\delta _{\textit{nk}}}{\langle {\Phi _j}(\cdot),{\Phi _n}(\cdot)\rangle _{\cal H}},$$
by virtue of the orthonormality of the ${\Phi _n}$. We further have
$${\delta _{\textit{jk}}} = {\lambda _k}{\langle {\Phi _j}(\cdot),{\Phi _k}(\cdot)\rangle _{\cal H}}.$$
The last relation gives
$${\langle {\Phi _j}(\cdot),{\Phi _k}(\cdot)\rangle _{\cal H}} = {\delta _{\textit{jk}}}/{\lambda _k}.$$

Let us now take two functions $f(x)$ and $g(x)$ and expand them in eigenfunction series:

$$f(x) = \sum\limits_j {f_j}{\Phi _j}(x),$$
$$g(x) = \sum\limits_k {g_k}{\Phi _k}(x).$$
Their inner product in ${\cal H}$ is
$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \sum\limits_j \sum\limits_k {f_j}g_k^*{\langle {\Phi _j}(\cdot),{\Phi _k}(\cdot)\rangle _{\cal H}},$$
which, by virtue of Eq. (99), gives
$${\langle f(\cdot),g(\cdot)\rangle _{\cal H}} = \sum\limits_k {f_k}g_k^*/{\lambda _k}.$$
This is the inner product in the RKHS as evaluated by means of eigenfunction and eigenvalue series. Notice that Eq. (103) is the discrete analog of Eq. (45).

It is easy to check, by using this expression of the inner product, that $K$ is the reproducing kernel. Let us, in fact, evaluate the inner product between $f(x)$ written according to Eq. (100) and $K(x,y)$ written according to Eq. (93). In the latter, the coefficient role is played by the terms ${\lambda _n}\Phi _n^*(y)$. Then, using Eq. (103), we find

$${\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}} = \sum\limits_k {f_k}{\lambda _k}{\Phi _k}(y)/{\lambda _k},$$
and this, according to Eq. (100), is just $f(y)$. Therefore, the reproducing property
$${\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}} = f(y)$$
holds as required.

Let us add a qualitative remark. According to Eq. (103), the squared norm of $f$ is given by

$$||f{||^2} = {\langle f(\cdot),f(\cdot)\rangle _{\cal H}} = \sum\limits_k |{f_k}{|^2}/{\lambda _k}.$$
Now, when $k$ increases, the ${\lambda _k}$ eigenvalues tend to zero [13]. In order to have a finite norm, the ${f_k}$ coefficients have to go to zero fast enough. Since the eigenfunctions ${\Phi _k}$ of high order are those that oscillate more rapidly, this implies that the functions of a RKHS have regularity features greater than those of ${L^2}$. In this sense it is sometimes said that a RKHS is smaller than ${L^2}$.

It will be noted that the first example of RKHS we saw, i.e., the one dealing with band-limited functions in (${-}{\nu _M},{\nu _M}$), whose reproducing kernel was $2{\nu _M}$ sinc[$2{\nu _M}(x - y)$], was not discussed in terms of eigenfunctions. This is because for such a kernel Eq. (92) does not hold. This is also true for all the shift-invariant kernels. Limiting ourselves to an intuitive hint, for all such kernels the eigenfunctions form a continuous set. This can be easily seen on writing the pertaining integral equation

$$\int_{- \infty}^\infty K(x - y)\Phi (y){\rm{d}}y = \lambda \Phi (x)$$
and passing to the Fourier domain. Then the convolution theorem gives
$$\tilde K(\nu)\tilde \Phi (\nu) = \lambda \tilde \Phi (\nu).$$
For a typical kernel $K$, Eq. (108) has Dirac delta functions as solutions. More explicitly, letting
$${\tilde \Phi _{{\nu _0}}}(\nu) = \delta (\nu - {\nu _0}),$$
with an arbitrary ${\nu _0}$, Eq. (108) becomes
$$\tilde K(\nu)\delta (\nu - {\nu _0}) \equiv \tilde K({\nu _0})\delta (\nu - {\nu _0}) = \lambda ({\nu _0})\delta (\nu - {\nu _0}),$$
where, in the first passage, we used a well-known property of the delta function. We then found $\lambda ({\nu _0}) \equiv \tilde K({\nu _0})$. The meaning is that any function ${\Phi _{{\nu _0}}}(x) = \exp (2\pi ix{\nu _0})$ is an eigenfunction of Eq. (107) with eigenvalue $\tilde K({\nu _0})$. Depending on the form of $\tilde K(\nu)$ there can be degeneracy of the eigenvalues. A limiting case is when $\tilde K(\nu) = {\rm{rect}}[\nu /(2{\nu _M})]$, in which only one eigenvalue ($\lambda = 1$) exists. There is no degeneracy when, e.g.,  $\tilde K(\nu)$ is a real strictly monotone function. Note that complex eigenvalues exist if $\tilde K(\nu)$ is not real.

Just to give an example, let us see how eigenfunctions and eigenvalues are evaluated for the kernel (Section 4)

$$K(x,y) = {\min}(x,y),$$
acting in the interval $(0,1)$ with the condition that $\Phi (x)$ vanishes in $x = 0$. To see how the integral equation is solved, note that the explicit form of Eq. (91) reads
$$\int_0^x y\Phi (y){\rm d}y + x\int_x^1 \Phi (y){\rm d}y = \lambda \Phi (x).$$
Differentiating with respect to $x$, we obtain
$$\int_x^1 \Phi (y){\rm d}y = \lambda \Phi ^\prime (x),$$
and with a further differentiation
$${-}\Phi (x) = \lambda \Phi ^{\prime \prime} (x).$$
Hence, the solutions of the harmonic oscillator equation come into play. Since the searched solutions have to vanish for $x = 0$, they are of the form
$$\Phi (x) = A\sin (\alpha x),$$
where the normalizing factor $A$ and $\alpha$ are to be determined. On inserting Eq. (115) into Eq. (112), we easily find
$${\alpha _n} = (2n + 1)\frac{\pi}{2},\quad {\lambda _n} = 1/\alpha _n^2,\quad (n = 0,1,2\ldots)$$
as well as $A = \sqrt 2$. All the eigenvalues are positive, and this implies that the kernel is positive definite [13,14]. Using now the Mercer theorem [Eq. (93)], we obtain
$$\begin{split}K(x,y)& ={ {\min}(x,y) }\\&={\frac{4}{{{\pi ^2}}}\sum\limits_{n = 0}^\infty \frac{1}{{{{(2n + 1)}^2}}}\sin \left({\frac{{2n + 1}}{2}\pi x} \right)\sin \left({\frac{{2n + 1}}{2}\pi y} \right)\!.}\end{split}$$

8. INTEGRAL REPRESENTATION

In this section we will be examining an integral representation of reproducing kernels. It was developed by Parzen [16] and imported into optics much later [17,18]. Let us consider kernels of the form

$$K(x,y) = \int_T A(t,y){A^*}(t,x)w(t){\rm d}t, \quad (x,y \in X),$$
where $w$ is non-negative over the integration domain $T$. The function $A$ is assumed to be square summable over $T$. It is easily seen that $K$ is non-negative definite. To this end, let us insert Eq. (118) into Eq. (40). This leads to
$$Q = \int_T {\left| {\sum\limits_i {c_i}A(t,{x_i})} \right|^2}w(t){\rm d}t \ge 0,$$
as predicted. Actually, in Eq. (119) the strict inequality sign holds (thus giving the kernel a strictly positive definite nature) unless the functions $A(t,{x_i})$ are linearly dependent [16].

It is found [16,19] that the space ${\cal H}$ associated to $K(x,y)$ is that of the functions $f(x)$ obtained through the linear transformation ${\cal L}$,

$$f(x) = {\cal L}\{F(\cdot)\} (x) = \int_T F(t){A^*}(t,x)w(t){\rm d}t,$$
where $F$ belongs to the closure of the linear span of $A$ and is uniquely associated to $f$. (A linear combination of an arbitrary number of functions $A(t,{x_i})$, where ${x_i}$ are distinct, fixed values of $x$.) Notice that, for given $y$, $K$ itself, see Eq. (118), has this structure [replacing $F(t)$ with $A(t,y)$], i.e.,
$$K(x,y) = {\cal L}\{A(\cdot ,y)\} (x).$$
We assume ${\cal L}$ to be invertible and denote by ${{\cal L}^{- 1}}$ the inverse operator, so that we have
$$F(t) = {{\cal L}^{- 1}}\{f(\cdot)\} (t);\;\;\;A(t,y) = {{\cal L}^{- 1}}\{K(\cdot ,y)\} (t).$$
We now define the inner product in ${\cal H}$ as follows:
$$\begin{split}{{\langle f(\cdot),g(\cdot)\rangle}_{\cal H}}& = {{{\left({{{\cal L}^{- 1}}\{f\} (\cdot),{{\cal L}^{- 1}}\{g\} (\cdot)} \right)}_{{L^2}}} }\\&={{{\left({F(\cdot),G(\cdot)} \right)}_{{L^2}}},}\end{split}$$
where the inner product specified by round brackets is the ordinary one on ${L^2}$, or, more explicitly,
$${\left({F(\cdot),G(\cdot)} \right)_{{L^2}}} = \int_T F(t){G^*}(t)w(t){\rm d}t.$$
In particular, the norm of $f$ is evaluated from
$$||f(\cdot {)||^2} = \int_T |F(t{)|^2}w(t){\rm d}t.$$
It is easily seen that, with the above inner product, ${\cal H}$ is a RKHS. In fact, taking Eqs. (122) and (123) into account, we find
$${\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}} = (F(\cdot),A(\cdot ,y{))_{{L^2}}},$$
or, by virtue of Eq. (120),
$${\langle f(\cdot),K(\cdot ,y)\rangle _{\cal H}} = f(y),$$
as we wanted to prove.

It will be noted that Eq. (120) represents an integral transform of $F(t)$, including, for a suitable choice of $A$ and of the domain $T$, the Fourier, Fresnel, and Hankel cases, to name a few.

We can give a very simple example. Let $T = [- {\nu _M},{\nu _M}]$, $A(t,x) = {e^{2\pi itx}}$, and $w(t) = 1$. Then, Eq. (118) gives

$$\begin{split}&{K(x,y) }\\&\quad ={\int_{- {\nu _M}}^{{\nu _M}} {e^{- 2\pi i(x - y)t}}{\rm d}t = 2{\nu _M} {\rm{sinc}}[2{\nu _M}(x - y)].}\end{split}$$
Accordingly, the $f$ functions are band-limited. Notice that in this case Eq. (123) is associated to the inner product theorem for Fourier transforms [1,2].

A more elaborate example is obtained on taking

$$T = (- \infty ,\infty); \quad A(t,x) = {e^{- \alpha {{(t - x)}^2}}}; \quad w(t) = {e^{- \gamma {t^2}}},$$
with $\alpha ,\gamma \gt 0$. The resulting kernel
$$K(x,y) = \int_{- \infty}^\infty {e^{- \alpha {{(t - x)}^2} - \alpha {{(t - y)}^2}}}{e^{- \gamma {t^2}}}{\rm{d}}t$$
is evaluated by completing the square in the exponent and by using the Gauss integral. After simple passages, we obtain (this is the well-known Gaussian–Schell model kernel of coherence theory [3])
$$\begin{split}&{K(x,y) }\\&\; ={\sqrt {\frac{\pi}{{2\alpha + \gamma}}} \exp \left[{- \frac{{\alpha \gamma}}{{2\alpha + \gamma}}({x^2} + {y^2}) - \frac{{{\alpha ^2}}}{{2\alpha + \gamma}}{{(x - y)}^2}} \right].}\end{split}$$
The associated ${\cal H}$ space is made of functions of the form of Eq. (120), which here reads
$$f(x) = \sqrt {\frac{\pi}{{2\alpha + \gamma}}} \int_{- \infty}^\infty F(t){e^{- \alpha {{(t - x)}^2} - \gamma {t^2}}}{\rm d}t.$$
It can be noted that $f$ is obtained by multiplying $F$ by a Gaussian function and on taking the convolution of the resulting function with another Gaussian function. Therefore, whatever $F$, its central part is emphasized [through multiplication by $\exp (- \gamma {t^2})$] and its structure is made smoother (through a convolution).

As a final example consider the following kernel,

$$K(x,y) = {K_0}\;{e^{\pi i\alpha ({x^2} - {y^2})}}{\rm{sinc}}[\beta (x - y)],$$
with constant ${K_0},\;\alpha \gt 0,\;\beta$. The kernel from Eq. (133) can be written
$$K(x,y) = {K_0}\frac{\alpha}{\beta}\int_{- \beta /(2\alpha)}^{\beta /(2\alpha)} \;{e^{- \pi i\alpha {{(y - t)}^2}}}\;{e^{\pi i\alpha {{(x - t)}^2}}}{\rm d}t,$$
which is of the form in Eq. (118) with
$$A(t,y) = \sqrt {\frac{{\alpha {K_0}}}{\beta}} {e^{- \pi i\alpha {{(y - t)}^2}}}.$$
Then, Eq. (120) takes on the form
$$f(x) = \sqrt {\frac{{\alpha {K_0}}}{\beta}} \int_{- \beta /(2\alpha)}^{\beta /(2\alpha)} F(t)\;{e^{\pi i\alpha {{(x - t)}^2}}}{\rm d}t,$$
which is (up to multiplicative factors) nothing else than the Fresnel transform [20].

9. PSEUDOMODAL EXPANSIONS

Let us consider a kernel of the form from Eq. (118) and the associated Hilbert space ${\cal H}$. Let $\{{F_n}(t)\} _{n = 0}^\infty$ be a basis in ${L^2}(T)$. Then the functions obtained through Eq. (120), say, $\{{f_n}(x)\} _{n = 0}^\infty$, form a basis in ${\cal H}$. To prove this let us first note that the functions $\{{f_n}(x)\}$ are orthonormal. In fact [see Eq. (123)],

$${\langle {f_n},{f_m}\rangle _{\cal H}} = ({F_n},{F_m}{)_{{L^2}}} = {\delta _{\textit{nm}}}.$$
Next note that if $f \in {\cal H}$ is such that ${\langle {f_n},f\rangle _{\cal H}} = 0,\;\;\forall n$ then $f = 0$. In fact,
$$0 = {\langle {f_n},f\rangle _{\cal H}} = ({F_n},F{)_{{L^2}}},\;\forall n.$$
Since ${F_n}$ is a basis in ${L^2}(T)$ Eq. (138) implies $F = 0$, which in turn gives $f = 0$. Then the functions ${f_n}$ form a basis in ${\cal H}$.

For any fixed $y = {y_0}$, $K(x,{y_0}) \in {\cal H}$ [see Eq. (32)]. Then

$$K(x,{y_0}) = \sum\limits_{n = 0}^\infty {K_n}({y_0}){f_n}(x),$$
where [Eq. (120)]
$$\begin{split}{K_n}({y_0}) &={ {{\langle K(x,{y_0}),{f_n}(x)\rangle}_{\cal H}} }\\&={\int_T F_n^*(t)A(t,{y_0}){\rm d}t = f_n^*({y_0}).}\end{split}$$
Hence,
$$K(x,{y_0}) = \sum\limits_{n = 0}^\infty f_n^*({y_0}){f_n}(x),$$
where the convergence is in ${\cal H}$. Then, for any fixed ${y_0}$, we have in norm convergence, which in RKHSs implies point convergence too (Section 6).

If we now fix $x = {x_0}$, we can proceed in a similar way and reach the expression

$$K({x_0},y) = \sum\limits_{n = 0}^\infty {f_n}({x_0})f_n^*(y).$$
Following a procedure analogous to the previous one, we obtain that the convergence in Eq. (142) is pointwise convergence in $y$ for fixed ${x_0}$.

In summary, we conclude that

$$K(x,y) = \sum\limits_{n = 0}^\infty {f_n}(x)f_n^*(y),$$
with pointwise convergence.

On the other hand, $\forall n$ we have that

$$||{f_n}||_{{L^2}}^2 = \int_X |{f_n}(x{)|^2}{\rm{d}}x$$
is finite, in fact,
$$\begin{split}|{f_n}(x{{)|}^2}& ={ {{\left| {\int_T A(t,x){F_n}(t){\rm d}t} \right|}^2} }\\&\le{\int_T |A(t,x{{)|}^2}{\rm d}t\int_T |{F_n}(t{{)|}^2}{\rm d}t,}\end{split}$$
so that using Eq. (118) we find
$$||{f_n}||_{{L^2}}^2 \le \int_X K(x,x){\rm d}x \lt \infty ,$$
since $||{F_n}{||_{{L^2}}} = 1$.

By using $||{f_n}||_{{L^2}}^2 \lt \infty ,\;\forall n$, we rewrite Eq. (143) as

$$K(x,y) = \sum\limits_n ||{f_n}||_{{L^2}}^2{\psi _n}(x)\psi _n^*(y),$$
where
$${\psi _n}(x) = \frac{{{f_n}(x)}}{{||{f_n}{{||}_{{L^2}}}}}.$$

It can be noted that Eq. (147) has the same form as Eq. (93), the so-called modal expansion. The functions ${\psi _n}$ though are (generally) different from the eigenfunctions. This is why Eq. (147) is called pseudomodal expansion [18]. The above procedure offers a rather simple way to construct such an expansion.

Let us work out an example using the following kernel,

$$K(x,y) = {h^*}(x)h(y){J_0}[b(x - y)],$$
where ${J_0}$ denotes the Bessel function of the first kind and zero order, $h(x) \in {L^2}(X)$, and $b$ is a positive constant. We can write [5]
$$K(x,y) = \frac{{{h^*}(x)h(y)}}{{2\pi}}\int_0^{2\pi} {e^{ib(x - y)\cos \varphi}}{\rm d} \varphi ,$$
i.e., $T = [0,2\pi]$ and $A(\varphi ,x) = h(x)\exp (ibx\cos \varphi)/\sqrt {2\pi}$.

In this case we can use as ${F_n}(t)$

$${F_n}(\varphi) = \frac{{{e^{{in\varphi}}}}}{{\sqrt {2\pi}}},$$
so that
$${f_n}(x) = \frac{{h(x)}}{{\sqrt {2\pi}}}\int_0^{2\pi} {e^{- ibx\cos \varphi}}{e^{{in\varphi}}}d\varphi = \frac{{h(x)}}{{\sqrt {2\pi}}}{i^{- n}}{J_n}(bx).$$
Then
$$K(x,y) = \frac{{{h^*}(x)h(y)}}{{2\pi}}\sum\limits_{n = 0}^\infty {J_n}(bx){J_n}(by).$$

10. GENERALIZED SAMPLING

Working in RKHS, it is possible to generalize the sampling theorem to a vast class of cases [21,22]. We give here a basic derivation. With reference to the integral representation, let us suppose a countable set of values ${t_n}$ exists for which the functions $A(t,{t_n})$ form a complete orthonormal system. Then the typical function $F(t)$ appearing in Eq. (120) can be expanded as follows:

$$F(t) = \sum\limits_{n = - \infty}^\infty {F_n}A(t,{t_n}).$$
As a consequence, the function $f(x)$ represented by Eq. (120) takes the form
$$\begin{split}f(x) &= {\sum\limits_{n = - \infty}^\infty {F_n}\int_T A(t,{t_n}){A^*}(t,x)w(t){\rm d}t }\\&={\sum\limits_{n = - \infty}^\infty {F_n}K(x,{t_n}),}\end{split}$$
where we used Eq. (118). On the other hand, because of Eq. (120), we have
$${F_n} = \int_T F(t){A^*}(t,{t_n})w(t){\rm d}t = f({t_n}),$$
so that the following sampling expansion is obtained:
$$f(x) = \sum\limits_{n = - \infty}^\infty f({t_n})K(x,{t_n}).$$

To see the connection with the traditional sampling, let us write $f(x)$ as

$$f(x) = \int_{- 1/2}^{1/2} F(t){e^{2\pi itx}}{\rm d}t.$$
This has the form of Eq. (120) with $A$ given by
$$A(x,t) = {e^{- 2\pi itx}}.$$
The functions $A(x,n)$ (for integer $n$) form a complete orthonormal set in (${-}{\rm{1/2}},\;{\rm{1/2}}$). The associated kernel (118) is nothing but sinc $(x - n)$, so that Eq. (157) gives the ordinary sampling theorem.

As an example, let us consider again the Fresnel transform [20], denoted for a typical function $f(t)$ with the symbol ${\hat f_\alpha}(x)$, which is defined as

$${\hat f_\alpha}(x) = \sqrt {- i\alpha} \int_{- \infty}^\infty f(t){e^{\pi i\alpha {{(t - x)}^2}}}{\rm d}t,$$
where $\alpha$ is a real non-zero number, from which we deduce
$$A(t,x) = \sqrt {- i\alpha} \;{e^{\pi i\alpha {{(t - x)}^2}}}; \quad w(t) = 1,$$
whose inversion formula reads
$$f(t) = \sqrt {i\alpha} \int_{- \infty}^\infty {\hat f_\alpha}(x){e^{- \pi i\alpha {{(t - x)}^2}}}{\rm d}x,$$
showing that the inverse Fresnel transform is simply obtained by changing $\alpha$ into ${-}\alpha$.

A function $f(\cdot)$ is said to be $(\alpha)$-Fresnel limited if its Fresnel transform ${\hat f_\alpha}(x)$ vanishes outside an interval $|x| \lt {\xi _0}$. Assuming $f$ to be $\alpha$-limited, let us evaluate $K(x,y)$, using for $A$, ${A^*}$ the expressions deducible from Eqs. (160) and (162):

$$\begin{split}K(x,y) &= \int_T A(t,y){A^*}(t,x){\rm{d}}t\\ &= |\alpha |{e^{- \pi i\alpha ({x^2} - {y^2})}}\int_{- {\xi _0}}^{{\xi _0}} {e^{2\pi i\alpha (x - y)t}}{\rm{d}}t\\ &= {e^{- \pi i\alpha ({x^2} - {y^2})}}{\rm{sinc}}[2\alpha {\xi _0}(x - y)],\end{split}$$
where we let $dm(t) = dt/(2|\alpha |{\xi _0})$. Since the zeros of $K$ are located at a mutual distance $1/(2|\alpha |{\xi _0})$, the following sampling theorem is obtained:
$$\begin{split}f(x)& = {{e^{- \pi i\alpha {x^2}}}\sum\limits_{n = - \infty}^\infty f\left({\frac{n}{{2|\alpha |{\xi _0}}}} \right)}\\ &\quad \times{\exp \left[{\pi i\alpha {{\left({\frac{n}{{2|\alpha |{\xi _0}}}} \right)}^2}} \right]{\rm{sinc}}(2|\alpha |{\xi _0}x - n).}\end{split}$$
Notice that, while superficially similar to the ordinary sampling theorem, this equation does not apply to band-limited functions. It can be seen in fact that a Fresnel-limited function cannot be band-limited [20].

11. REPRODUCING KERNEL HILBERT SPACES IN OPTICS

The interest of RKHSs in optics stems, first of all, from the Moore–Aronszajn theorem, according to which every RKHS possesses a non-negative definite reproducing kernel $K(x,y)$ and conversely every non-negative definite kernel specifies a RKHS. Since any cross-spectral density (CSD) has to be non-negative definite [3], we can say that any CSD specifies a RKHS and vice versa. This means that the very large number of CSDs devised throughout several years in coherence theory offers a good pool of examples for the theory of RKHSs. Quite often, the integral equations associated to CSDs were solved explicitly, thus giving the pertinent eigenfunctions and eigenvalues. On the other hand, there is a collection of reproducing kernels thoroughly studied in mathematics that did not penetrate into optics. Szegö’s and Bergman’s kernels are examples of this and could deserve attention from the opticist.

It is to be stressed that the knowledge of the RKHS associated to a certain CSD gives access to the regularity features of the underlying optical fields (Sections 6 and 10). This is of interest whenever the associated optical fields are to be processed with such operations as filtering or sampling.

The integral representation of RKs (Section 8) offered a safe criterion for devising genuine CSDs in hundreds of papers. It is worthwhile to remark that such a representation has a further merit in that it specifies an integral transform. Accordingly, beside well-known transforms (such as Fourier, Fresnel, and Hankel) there is a wealth of new transforms that would deserve to be explored.

A stimulating challenge is to inquire whether, in addition to known cases, it is possible to transform other possibilities offered by RKHSs into laboratory techniques. Let us refer to a concrete example: in Eq. (56) we saw a case in which the inner product of two signals requires one to integrate the product of their first derivatives. It does not seem that great difficulties should arise in converting such an operation in an experimental procedure, possibly combining analogical and numerical techniques. Similar possibilities exist for several other operations involved in RKHS analysis. This affords a rich field of action to the experimentalist in addition to the theoretical subjects treated above.

APPENDIX A: FUNCTIONAL ANALYSIS VIEW

In the present paper, we followed an approach to RKHSs in which we used the type of mathematical tools adopted in Fourier optics and in coherence theory. Current alternative presentations rely more often on concepts of functional analysis. For the sake of completeness as well as a suggestion toward extensions, we give here some hints in this direction.

A functional in a Hilbert space of functions is an operator $F$ that, upon application to a typical function $f(\cdot)$, gives rise to a (generally complex) number ${a_f}$, i.e.,

$$F\{f(\cdot)\} = {a_f}.$$
In particular, we are interested in linear functionals, i.e., functionals such that
$$F\{\alpha f(\cdot) + \beta g(\cdot)\} = \alpha {a_f} + \beta {a_g},$$
with obvious symbols. A simple example is as follows. Given a function $g(\cdot)$, the inner product of a typical function $f(\cdot)$ with $g(\cdot)$ gives a (complex) number. Thus we created the simple functional
$$F\{f(\cdot)\} = {\langle f(\cdot),g(\cdot)\rangle _{\cal H}},$$
where $g$ is fixed. In a sense, the function $g(\cdot)$ represents the functional $F$. An amazing result is that under reasonable regularity hypotheses all the possible functionals can be given the present form. To express this in precise terms we need some more definitions.

A linear functional is said to be continuous if $\forall \varepsilon \gt 0$ there exists a ${\delta _\varepsilon} \gt 0$ such that if $||f|| \lt {\delta _\varepsilon}$ then $|F\{f\} | \lt \varepsilon$.

A linear functional is said to be bounded if $\exists M \gt 0$ such that $\forall f(\cdot)$ we have $|F\{f(\cdot)\} | \le M||f||$ where $M$ does not depend on $f$.

It can be proved that a continuous functional is bounded and vice versa.

Now we can give the fundamental representation theorem of Riesz and Frechét [13]: In a Hilbert space of functions any continuous (or bounded) functional$F$ can be given the form of an inner product$F\{f(\cdot)\} = {\langle f(\cdot),g(\cdot)\rangle _{\cal H}}$ where $g$ depends on $F$ but not on $f$.

A peculiar type of functional is the so-called evaluation functional at $y \in X$. Denoting it by ${V_y}$, we define it as one that when applied to $f(\cdot)$ gives rise to $f(y)$, i.e.,

$${V_y}\{f(\cdot)\} = f(y).$$
Using the representation theorem we can say that there exists a function, say, ${v_y}(\cdot)$, such that ${V_y}\{f(\cdot)\} = \langle f(\cdot),{v_y}(\cdot)\rangle$. Replacing $\cdot$ by $x$, we notice that ${v_y}(x)$ depends both on $x$ and on $y$. In other words, ${v_y}(x)$ is a function of $x$ and $y$. Putting things together we can write Eq. (A4) as
$$\langle f(\cdot),{v_y}(\cdot)\rangle = f(y).$$

Compared with the reproduction property from Eq. (33), we conclude that ${v_y}(\cdot) = K(\cdot ,y)$.

With the above elements it is possible to show that a RKHS can be defined as follows: A Hilbert space of functions defined on $X$ is a RKHS if the evaluation functional at $x$ is continuous$\;\forall x \in X$.

It will be noted that the above definition does not mention the reproduction property, which is instead proved as a consequence of the definition itself.

Funding

Ministerio de Economía y Competitividad (PID2019-104268GB-C21).

Acknowledgment

The authors gratefully thank Gemma Piquero, Juan Carlos Gonzáles de Sande, and Massimo Santarsiero for useful discussions on the subject matter of this paper.

Disclosures

The authors declare no conflicts of interest.

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

REFERENCES

1. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill, 1968).

2. A. Papoulis, Systems and Transforms with Applications in Optics (McGraw-Hill, 1968).

3. L. Mandel and E. Wolf, Optical Coherence and Quantum Optics (Cambridge University, 1995).

4. A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics (Kluwer, Academic, 2004).

5. M. Abramowitz and I. Stegun, Handbook of Mathematical Functions (National Bureau of Standards, 1972).

6. G. Szego, Orthogonal Polynomials (American Mathematical Society, 1975).

7. S. Saitoh and Y. Sawano, Theory of Reproducing Kernels and Applications (Springer, 2016).

8. R. Martínez-Herrero and F. Gori, “Christoffel-Darboux sources,” Opt. Lett. 46, 973–976 (2021). [CrossRef]  

9. N. Aronszajn, “Theory of reproducing kernels,” Trans. Am. Math. Soc. 68, 337–404 (1950). [CrossRef]  

10. S. Saitoh, Integral Transforms, Reproducing Kernels and Their Applications (Longman, 1997).

11. V. I. Paulsen and M. Raghupathi, An Introduction to the Theory of Reproducing Kernel Hilbert Spaces (Cambridge, 2016).

12. R. A. Kennedy and P. Sadeghi, Hilbert Space Methods in Signal Processing (Cambridge University, 2013).

13. F. Riesz and B. Sz. Nagy, Functional Analysis (Blackie and Sons, 1956).

14. F. G. Tricomi, Integral Equations (Dover, 1985).

15. M. Santarsiero, G. Piquero, J. C. G. de Sande, and F. Gori, “Difference of cross-spectral densities,” Opt. Lett. 39, 1713–1716 (2014). [CrossRef]  

16. E. Parzen, “An approach to time series analysis,” Ann. Math. Stat. 32, 951–989 (1961). [CrossRef]  

17. F. Gori and M. Santarsiero, “Devising genuine spatial correlation functions,” Opt. Lett. 32, 3531–3533 (2007). [CrossRef]  

18. R. Martínez-Herrero, P. M. Mejías, and F. Gori, “Genuine cross-spectral densities and pseudo-modal expansions,” Opt. Lett. 34, 1399–1401 (2009). [CrossRef]  

19. R. L. Eubank and T. Hsing, “Canonical correlation for stochastic processes,” Stochastic Process. Appl. 118, 1634–1661 (2008). [CrossRef]  

20. F. Gori, “Fresnel transform and sampling theorem,” Opt. Commun. 39, 293–297 (1981). [CrossRef]  

21. A. G. Garcia, “Orthogonal sampling formulas: a unified approach,” SIAM Rev. 42, 499–512 (2000). [CrossRef]  

22. D. Han, M. Z. Nashed, and Q. Sun, “Sampling expansions in reproducing kernel Hilbert and banach spaces,” Num. Funct. Anal. Optim. 30, 971–987 (2009). [CrossRef]  

Data Availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Equations (169)

Equations on this page are rendered with MathJax. Learn more.

F { f ( ⋅ ) } ( ν ) = f ~ ( ν ) = ∫ − ∞ ∞ f ( x ) exp ⁡ ( − 2 π i ν x ) d x ,
f ~ ( ν ) = 0 f o r | ν | ≥ ν M > 0 ,
f ~ ( ν ) r e c t ( ν 2 ν M ) = f ~ ( ν ) ,
f ( x ) = ∫ − ∞ ∞ f ~ ( ν ) exp ⁡ ( 2 π i ν x ) d ν ,
2 ν M ∫ − ∞ ∞ f ( y ) s i n c [ 2 ν M ( x − y ) ] d y = f ( x ) ,
s i n c ( x ) = sin ⁡ ( π x ) π x .
K ( x , y ) = 2 ν M s i n c [ 2 ν M ( x − y ) ] ,
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∫ − ∞ ∞ f ( y ) g ∗ ( y ) d y ,
⟨ f ( ⋅ ) , K ( ⋅ , x ) ⟩ H = f ( x ) .
f ~ ( ν ) K ~ ( ν ) = f ~ ( ν ) ,
| f ( x ) | ≤ 2 ν M E ,
| f ( x + D ) − f ( x ) | ≤ 4 ν M E [ 1 − s i n c ( 2 ν M D ) ] ,
E = ∫ − ∞ ∞ | f ( x ) | 2 d x .
f ( x ) = ∑ n = − N N f n e 2 π i n x / P ,
f n = 1 P ∫ 0 P f ( x ) e − 2 π i n x / P d x .
1 P ∫ 0 P f ( x ) g ∗ ( x ) d x = ∑ n = − N N f n g n ∗ ,
K ( x − y ) = sin ⁡ [ π ( 2 N + 1 ) ( x − y ) / P ] sin ⁡ [ π ( x − y ) / P ] ,
sin ⁡ [ π ( 2 N + 1 ) t ] sin ⁡ ( π t ) = ∑ n = − N N e 2 π i n t ,
p ( x ) = ∑ n = 0 N c n P n ( x ) ,
c m = ⟨ p ( ⋅ ) , P m ( ⋅ ) ⟩ H = ∫ X p ( x ) P m ( x ) w ( x ) d x , ( m = 0 , 1 , … , N ) ,
∫ X p ( x ) P m ( x ) w ( x ) d x = ∑ n = 0 N c n ∫ X P n ( x ) P m ( x ) w ( x ) d x = c m , ( m = 0 , 1 , … , N ) ,
K N ( x , y ) = ∑ n = 0 N P n ( x ) P n ( y )
∫ X p ( x ) K N ( x , y ) w ( x ) d x = ∑ n = 0 N P n ( y ) ∫ X p ( x ) P n ( x ) w ( x ) d x = ∑ n = 0 N c n P n ( y ) = p ( y ) .
K N ( x , y ) = a N a N + 1 P N + 1 ( x ) P N ( y ) − P N + 1 ( y ) P N ( x ) x − y ,
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H ,
‖ f ‖ = ⟨ f ( ⋅ ) , f ( ⋅ ) ⟩ H .
⟨ f , g ⟩ H = ⟨ g , f ⟩ H ∗ ,
⟨ α f + β g , h ⟩ H = α ⟨ f , h ⟩ H + β ⟨ g , h ⟩ H ,
⟨ f , f ⟩ H ≥ 0 ,
⟨ f , f ⟩ H = 0 i f f f = 0.
⟨ f , α g + β h ⟩ H = ⟨ α g + β h , f ⟩ H ∗ = α ∗ ⟨ f , g ⟩ + β ∗ ⟨ f , h ⟩ .
A ) ∀ y ∈ X ⇒ K ( ⋅ , y ) ∈ H ;
B ) ∀ f ∈ H a n d ∀ y ∈ X ⇒ ⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = f ( y ) .
⟨ K ( ⋅ , x ) , K ( ⋅ , y ) ⟩ H = K ( y , x ) .
⟨ K ( ⋅ , x ) , K ( ⋅ , x ) ⟩ H = ‖ K ( ⋅ , x ) ‖ 2 = K ( x , x ) .
⟨ c K ( ⋅ , x ) , K ( ⋅ , y ) ⟩ H = c K ( y , x )
⟨ K ( ⋅ , x ) , c K ( ⋅ , y ) ⟩ H = c ∗ K ( y , x ) .
⟨ K ( ⋅ , x ) , K ( ⋅ , y ) ⟩ H = [ ⟨ K ( ⋅ , y ) , K ( ⋅ , x ) ⟩ H ] ∗ ,
K ( y , x ) = K ( x , y ) ∗ ,
Q = ∑ i = 1 N ∑ j = 1 N K ( x i , x j ) c i ∗ c j ,
Q ≥ 0 ,
Q = ∑ i = 1 N ∑ j = 1 N c i ∗ c j ⟨ K ( ⋅ , x j ) , K ( ⋅ , x i ) ⟩ H ,
Q = ⟨ ∑ j = 1 N c j K ( ⋅ , x j ) , ∑ i = 1 N c i K ( ⋅ , x i ) ⟩ H = | | ∑ i = 1 N c i K ( ⋅ , x i ) | | 2 ≥ 0.
K ( x , x ) ≥ 0 ,
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∫ f ~ ( ν ) g ~ ∗ ( ν ) K ~ ( ν ) d ν ,
‖ f ‖ 2 = ⟨ f ( ⋅ ) , f ( ⋅ ) ⟩ H = ∫ | f ~ ( ν ) | 2 K ~ ( ν ) d ν < + ∞ ,
F { K ( ⋅ − y ) } ( ν ) = K ~ ( ν ) e − 2 π i ν y .
⟨ f ( ⋅ ) , K ( ⋅ − y ) ⟩ H = ∫ f ~ ( ν ) K ~ ( ν ) e 2 π i ν y K ~ ( ν ) d ν ,
K ( x − y ) = 1 2 e − | x − y | ,
K ~ ( ν ) = 1 1 + 4 π 2 ν 2 .
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∫ − ∞ ∞ f ~ ( ν ) g ~ ∗ ( ν ) ( 1 + 4 π 2 ν 2 ) d ν .
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∫ − ∞ ∞ [ f ( x ) g ∗ ( x ) + f ′ ( x ) g ′ ∗ ( x ) ] d x .
K ( x , y ) = min ( x , y ) ,
⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = ∫ 0 1 U { f ( x ) } U { K ( x , y ) } d x = f ( y ) ,
f ( y ) − f ( 0 ) = ∫ 0 y f ′ ( x ) d x .
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∫ 0 1 f ′ ( x ) g ′ ( x ) d x .
K ( x , y ) = min ( x , y ) − x y .
f ( z ) = ∑ n = 0 ∞ a n z n , g ( z ) = ∑ n = 0 ∞ b n z n .
⟨ f , g ⟩ = 1 2 π ∑ n = 0 ∞ ∑ m = 0 ∞ a n b m ∗ ∫ 0 2 π e i ( n − m ) θ d θ = ∑ n = 0 ∞ a n b n ∗ .
⟨ f ( z 1 ) , K ( z 1 , z 2 ) ⟩ H = f ( z 2 ) = ∑ n = 0 ∞ a n z 2 n .
K ( z 1 , z 2 ) = ∑ n = 0 ∞ b n ( z 2 ) z 1 n .
⟨ f ( z 1 ) , K ( z 1 , z 2 ) ⟩ = ∑ n = 0 ∞ a n b n ∗ ( z 2 ) .
∑ n = 0 ∞ a n z 2 n = ∑ n = 0 ∞ a n b n ∗ ( z 2 ) ,
b n ( z 2 ) = ( z 2 ∗ ) n .
K ( z 1 , z 2 ) = ∑ n = 0 ∞ z 1 n ( z 2 ∗ ) n .
K ( z 1 , z 2 ) = 1 1 − z 1 z 2 ∗ ,
f ( z ) = ∑ n = 0 ∞ a n z n ; g ( z ) = ∑ n = 0 ∞ b n z n .
⟨ f ( z ) , g ( z ) ⟩ = ∫ f ( z ) g ∗ ( z ) d 2 z = ∑ n = 0 ∞ ∑ m = 0 ∞ a n b m ∗ ∫ 0 1 ∫ 0 2 π r n + m e i ( n − m ) ϑ r d r d ϑ = π ∑ n = 0 ∞ a n b n ∗ ∫ 0 1 t n d t = π ∑ n = 0 ∞ a n b n ∗ n + 1 .
K ( z 1 , z 2 ) = ∑ n = 0 ∞ k n ( z 2 ) r 1 n e i n ϑ 1 .
⟨ f ( z 1 ) , K ( z 1 , z 2 ) ⟩ = π ∑ n = 0 ∞ a n k n ∗ ( z 2 ) n + 1 ,
k n ( z 2 ) = n + 1 π r 2 n e − i n ϑ 2 .
K ( z 1 , z 2 ) = 1 π ∑ n = 0 ∞ ( n + 1 ) z 1 n z 2 ∗ n = 1 π 1 ( 1 − z 1 z 2 ∗ ) 2 ,
f ( x ) = ∑ n = 1 N a n K ( x , x n ) .
⟨ f ( x ) , g ( y ) ⟩ H = ∑ n = 1 N ∑ m = 1 N a n b m ∗ ⟨ K ( x , x n ) , K ( x , x m ) ⟩ H = ∑ n = 1 N ∑ m = 1 N a n b m ∗ K ( x m , x n ) ,
⟨ f ( ⋅ ) , K ( ⋅ , x ) ⟩ H = ∑ n = 1 N a n ⟨ K ( ⋅ , x n ) , K ( ⋅ , x ) ⟩ H = ∑ n = 1 N a n K ( x , x n ) = f ( x ) .
| | K ( ⋅ , y ) − K 1 ( ⋅ , y ) | | 2 > 0.
| | K ( ⋅ , y ) − K 1 ( ⋅ , y ) | | 2 = ⟨ K ( ⋅ , y ) − K 1 ( ⋅ , y ) , K ( ⋅ , y ) − K 1 ( ⋅ , y ) ⟩ H ,
| | K ( ⋅ , y ) − K 1 ( ⋅ , y ) | | 2 = ⟨ K − K 1 , K ⟩ H − ⟨ K − K 1 , K 1 ⟩ H .
| ⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H | ≤ | | f | | | | g | | .
| f ( y ) | = | ⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H | ≤ | | f | | K ( y , y ) ,
| K ( x , y ) | = | ⟨ K ( ⋅ , y ) , K ( ⋅ , x ) ⟩ H | ≤ K ( x , x ) K ( y , y ) .
| f n ( x ) − f m ( x ) | = | ⟨ f n ( ⋅ ) − f m ( ⋅ ) , K ( ⋅ , x ) ⟩ H | ≤ | | K ( ⋅ , x ) | | | | f n − f m | | ,
| f n ( x ) − f m ( x ) | ≤ K ( x , x ) | | f n − f m | | .
| f ( x ) − f ( y ) | = | ⟨ f ( ⋅ ) , K ( ⋅ , x ) ⟩ H − ⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H | = | ⟨ f ( ⋅ ) , K ( ⋅ , x ) − K ( ⋅ , y ) ⟩ H | ,
| f ( x ) − f ( y ) | ≤ | | f | | | | K ( ⋅ , x ) − K ( ⋅ , y ) | | .
| | K ( ⋅ , x ) − K ( ⋅ , y ) | | 2 = ⟨ K ( ⋅ , x ) − K ( ⋅ , y ) , K ( ⋅ , x ) − K ( ⋅ , y ) ⟩ H ,
| | K ( ⋅ , x ) − K ( ⋅ , y ) | | 2 = ⟨ K ( ⋅ , x ) , K ( ⋅ , x ) ⟩ H + ⟨ K ( ⋅ , y ) , K ( ⋅ , y ) ⟩ H − 2 ℜ { ⟨ K ( ⋅ , x ) , K ( ⋅ , y ) ⟩ H } ,
| | K ( ⋅ , x ) − K ( ⋅ , y ) | | 2 = K ( x , x ) + K ( y , y ) − 2 ℜ { K ( x , y ) } .
| f ( x ) − f ( y ) | ≤ | | f | | d ( x , y ) ,
d ( x , y ) = K ( x , x ) + K ( y , y ) − 2 ℜ { K ( x , y ) } ,
∫ D Φ ( y ) K ( x , y ) d y = λ Φ ( x ) ,
∬ | K ( x , y ) | 2 d x d y < + ∞ ,
K ( x , y ) = ∑ n = 0 ∞ λ n Φ n ( x ) Φ n ∗ ( y ) ,
Φ j ( y ) = ⟨ Φ j ( ⋅ ) , K ( ⋅ , y ) ⟩ H ,
Φ j ( y ) = ∑ n λ n ⟨ Φ j ( ⋅ ) , Φ n ( ⋅ ) Φ n ∗ ( y ) ⟩ H .
Φ j ( y ) = ∑ n λ n Φ n ( y ) ⟨ Φ j ( ⋅ ) , Φ n ( ⋅ ) ⟩ H .
δ jk = ∑ n λ n δ nk ⟨ Φ j ( ⋅ ) , Φ n ( ⋅ ) ⟩ H ,
δ jk = λ k ⟨ Φ j ( ⋅ ) , Φ k ( ⋅ ) ⟩ H .
⟨ Φ j ( ⋅ ) , Φ k ( ⋅ ) ⟩ H = δ jk / λ k .
f ( x ) = ∑ j f j Φ j ( x ) ,
g ( x ) = ∑ k g k Φ k ( x ) .
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∑ j ∑ k f j g k ∗ ⟨ Φ j ( ⋅ ) , Φ k ( ⋅ ) ⟩ H ,
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ∑ k f k g k ∗ / λ k .
⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = ∑ k f k λ k Φ k ( y ) / λ k ,
⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = f ( y )
| | f | | 2 = ⟨ f ( ⋅ ) , f ( ⋅ ) ⟩ H = ∑ k | f k | 2 / λ k .
∫ − ∞ ∞ K ( x − y ) Φ ( y ) d y = λ Φ ( x )
K ~ ( ν ) Φ ~ ( ν ) = λ Φ ~ ( ν ) .
Φ ~ ν 0 ( ν ) = δ ( ν − ν 0 ) ,
K ~ ( ν ) δ ( ν − ν 0 ) ≡ K ~ ( ν 0 ) δ ( ν − ν 0 ) = λ ( ν 0 ) δ ( ν − ν 0 ) ,
K ( x , y ) = min ( x , y ) ,
∫ 0 x y Φ ( y ) d y + x ∫ x 1 Φ ( y ) d y = λ Φ ( x ) .
∫ x 1 Φ ( y ) d y = λ Φ ′ ( x ) ,
− Φ ( x ) = λ Φ ′ ′ ( x ) .
Φ ( x ) = A sin ⁡ ( α x ) ,
α n = ( 2 n + 1 ) π 2 , λ n = 1 / α n 2 , ( n = 0 , 1 , 2 … )
K ( x , y ) = min ( x , y ) = 4 π 2 ∑ n = 0 ∞ 1 ( 2 n + 1 ) 2 sin ⁡ ( 2 n + 1 2 π x ) sin ⁡ ( 2 n + 1 2 π y ) .
K ( x , y ) = ∫ T A ( t , y ) A ∗ ( t , x ) w ( t ) d t , ( x , y ∈ X ) ,
Q = ∫ T | ∑ i c i A ( t , x i ) | 2 w ( t ) d t ≥ 0 ,
f ( x ) = L { F ( ⋅ ) } ( x ) = ∫ T F ( t ) A ∗ ( t , x ) w ( t ) d t ,
K ( x , y ) = L { A ( ⋅ , y ) } ( x ) .
F ( t ) = L − 1 { f ( ⋅ ) } ( t ) ; A ( t , y ) = L − 1 { K ( ⋅ , y ) } ( t ) .
⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H = ( L − 1 { f } ( ⋅ ) , L − 1 { g } ( ⋅ ) ) L 2 = ( F ( ⋅ ) , G ( ⋅ ) ) L 2 ,
( F ( ⋅ ) , G ( ⋅ ) ) L 2 = ∫ T F ( t ) G ∗ ( t ) w ( t ) d t .
| | f ( ⋅ ) | | 2 = ∫ T | F ( t ) | 2 w ( t ) d t .
⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = ( F ( ⋅ ) , A ( ⋅ , y ) ) L 2 ,
⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H = f ( y ) ,
K ( x , y ) = ∫ − ν M ν M e − 2 π i ( x − y ) t d t = 2 ν M s i n c [ 2 ν M ( x − y ) ] .
T = ( − ∞ , ∞ ) ; A ( t , x ) = e − α ( t − x ) 2 ; w ( t ) = e − γ t 2 ,
K ( x , y ) = ∫ − ∞ ∞ e − α ( t − x ) 2 − α ( t − y ) 2 e − γ t 2 d t
K ( x , y ) = π 2 α + γ exp ⁡ [ − α γ 2 α + γ ( x 2 + y 2 ) − α 2 2 α + γ ( x − y ) 2 ] .
f ( x ) = π 2 α + γ ∫ − ∞ ∞ F ( t ) e − α ( t − x ) 2 − γ t 2 d t .
K ( x , y ) = K 0 e π i α ( x 2 − y 2 ) s i n c [ β ( x − y ) ] ,
K ( x , y ) = K 0 α β ∫ − β / ( 2 α ) β / ( 2 α ) e − π i α ( y − t ) 2 e π i α ( x − t ) 2 d t ,
A ( t , y ) = α K 0 β e − π i α ( y − t ) 2 .
f ( x ) = α K 0 β ∫ − β / ( 2 α ) β / ( 2 α ) F ( t ) e π i α ( x − t ) 2 d t ,
⟨ f n , f m ⟩ H = ( F n , F m ) L 2 = δ nm .
0 = ⟨ f n , f ⟩ H = ( F n , F ) L 2 , ∀ n .
K ( x , y 0 ) = ∑ n = 0 ∞ K n ( y 0 ) f n ( x ) ,
K n ( y 0 ) = ⟨ K ( x , y 0 ) , f n ( x ) ⟩ H = ∫ T F n ∗ ( t ) A ( t , y 0 ) d t = f n ∗ ( y 0 ) .
K ( x , y 0 ) = ∑ n = 0 ∞ f n ∗ ( y 0 ) f n ( x ) ,
K ( x 0 , y ) = ∑ n = 0 ∞ f n ( x 0 ) f n ∗ ( y ) .
K ( x , y ) = ∑ n = 0 ∞ f n ( x ) f n ∗ ( y ) ,
| | f n | | L 2 2 = ∫ X | f n ( x ) | 2 d x
| f n ( x ) | 2 = | ∫ T A ( t , x ) F n ( t ) d t | 2 ≤ ∫ T | A ( t , x ) | 2 d t ∫ T | F n ( t ) | 2 d t ,
| | f n | | L 2 2 ≤ ∫ X K ( x , x ) d x < ∞ ,
K ( x , y ) = ∑ n | | f n | | L 2 2 ψ n ( x ) ψ n ∗ ( y ) ,
ψ n ( x ) = f n ( x ) | | f n | | L 2 .
K ( x , y ) = h ∗ ( x ) h ( y ) J 0 [ b ( x − y ) ] ,
K ( x , y ) = h ∗ ( x ) h ( y ) 2 π ∫ 0 2 π e i b ( x − y ) cos ⁡ φ d φ ,
F n ( φ ) = e i n φ 2 π ,
f n ( x ) = h ( x ) 2 π ∫ 0 2 π e − i b x cos ⁡ φ e i n φ d φ = h ( x ) 2 π i − n J n ( b x ) .
K ( x , y ) = h ∗ ( x ) h ( y ) 2 π ∑ n = 0 ∞ J n ( b x ) J n ( b y ) .
F ( t ) = ∑ n = − ∞ ∞ F n A ( t , t n ) .
f ( x ) = ∑ n = − ∞ ∞ F n ∫ T A ( t , t n ) A ∗ ( t , x ) w ( t ) d t = ∑ n = − ∞ ∞ F n K ( x , t n ) ,
F n = ∫ T F ( t ) A ∗ ( t , t n ) w ( t ) d t = f ( t n ) ,
f ( x ) = ∑ n = − ∞ ∞ f ( t n ) K ( x , t n ) .
f ( x ) = ∫ − 1 / 2 1 / 2 F ( t ) e 2 π i t x d t .
A ( x , t ) = e − 2 π i t x .
f ^ α ( x ) = − i α ∫ − ∞ ∞ f ( t ) e π i α ( t − x ) 2 d t ,
A ( t , x ) = − i α e π i α ( t − x ) 2 ; w ( t ) = 1 ,
f ( t ) = i α ∫ − ∞ ∞ f ^ α ( x ) e − π i α ( t − x ) 2 d x ,
K ( x , y ) = ∫ T A ( t , y ) A ∗ ( t , x ) d t = | α | e − π i α ( x 2 − y 2 ) ∫ − ξ 0 ξ 0 e 2 π i α ( x − y ) t d t = e − π i α ( x 2 − y 2 ) s i n c [ 2 α ξ 0 ( x − y ) ] ,
f ( x ) = e − π i α x 2 ∑ n = − ∞ ∞ f ( n 2 | α | ξ 0 ) × exp ⁡ [ π i α ( n 2 | α | ξ 0 ) 2 ] s i n c ( 2 | α | ξ 0 x − n ) .
F { f ( ⋅ ) } = a f .
F { α f ( ⋅ ) + β g ( ⋅ ) } = α a f + β a g ,
F { f ( ⋅ ) } = ⟨ f ( ⋅ ) , g ( ⋅ ) ⟩ H ,
V y { f ( ⋅ ) } = f ( y ) .
⟨ f ( ⋅ ) , v y ( ⋅ ) ⟩ = f ( y ) .
Select as filters
Select Topics Cancel
Top
       
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.
Privacy | Terms of Use