Kernel functions

Here we provide a description of the different classes used to build and evaluate the radial basis function networks. Two different kernel functions \(\phi(r)\) are defined in the code, the Exponential kernel (RBF_Exponential) and the the Spline kernel (RBF_Spline).

The fit method of these classes solves the system \(A s = F\). In the case of RBF interpolants, the vector \(F\) contains the values of the function where the points have been evaluated and the matrix \(A\) is defined as:

\[\begin{split}A = \begin{bmatrix} \Phi & P \\ P^T & 0. \end{bmatrix}\end{split}\]

The matrix \(\Phi\) is defined as:

\[\Phi_{i,j} = \phi(r_{i,j}),\]

where \(r_{i,j} = \left \| x^i-x^j \right\|\) is the Euclidean distance between the centers \(x^i\) and \(x^j\). \(P\) represents the added polynomial terms and the vector \(s\) contains the weights of the RBF network.

Once the fit method has been used to build the surrogate model (find the vector \(s\)), it is possible to evaluate points using the method evaluate.

If new centers are added, the methods retrain and retrain_schur can be used to inverse the augmented RBF matrix

\[\begin{split}A = \begin{bmatrix} \Phi & \Phi_{1,2} \\ \Phi_{2,1} & \Phi_{2,2}. \end{bmatrix}\end{split}\]

where \(\mathbf{\Phi}_{2,2}\) represents the kernel matrix of the new centers added by the online k-means algorithm, and \(\mathbf{\Phi}_{1,2} = \mathbf{\Phi}_{2,1}^T\) the cross kernel matrix between initial and new centers, respectively.

RONAALP.utilities.schur_complement.schur_inverse(Ainv, B, C, D)

Compute inverse of matrix M = [[A B], [C D]], defined by the block matrix A, B, C and D using the Schur complement.

Parameters

Ainvarray-like, shape (l1,l1), inverse of block matrix A

Barray-like, shape (l1,l2), block matrix B

Carray-like, shape (l2,l1), block matrix C

Darray-like, shape (l2,l2), block matrix D

Returns

Minvarray-like, shape (l1+l2,l1+l2), inverse of matrix M

References

1

https://chrisyeh96.github.io/2021/05/19/schur-complement.html

Exponential Kernel

The Exponential Kernel is defined as follows:

\[\Phi(r) = \exp \left( -\dfrac{r^2}{2 l^2} \right),\]

where \(l>0\) and \(r=\|y-x\|\), where \(\|\cdot\|\) is the Euclidean norm. In this case, the polynomial terms can be omitted.

class RONAALP.utilities.kernels.exponential.RBF_Exponential(l=1.0, epsilon=0.001)

RBF Exponential kernel class.

Parameters

lfloat or ndarray, shape (m,), optional

Internal parameter. Width of the kernel.

epsilonfloat, optional

Smoothing parameter.

Attributes

lfloat or ndarray, shape (m,)

Internal parameter. Width of the kernel.

sndarray, shape(m,)

RBF coefficients.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,p,)

Array of function values at x.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,)

Array of interpolated values at y.

fit(x, f)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,p)

Array of function values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

Andarray, shape(m*(d+1),m*(d+1),)

RBF matrix with linear polynomial terms.

retrain(x_new, f_new, l_new=1.0)

Retrain the surrogate model in a brute force approach.

Parameters

x_newndarray, shape (m2,d,)

New array of points where function values are known. m2 is the number of new sampling points.

f_newndarray, shape (m2,p)

Array of new function values at x_new.

l_newfloat or ndarray, shape (m2,)

Internal parameter. Width of the kernel on this new points.

retrain_schur(x_new, f_new, l_new=1.0)

Efficiently retrain the surrogate model using the Schur complement.

Parameters

x_newndarray, shape (m2,d,)

New array of points where function values are known. m2 is the number of new sampling points.

f_newndarray, shape (m2,p)

Array of new function values at x_new.

l_newfloat or ndarray, shape (m2,)

Internal parameter. Width of the kernel on this new points.

update(l=1.0)

Update internal parameters of the kernel.

Parameters

lfloat or ndarray, shape (d,), optional

Internal parameter. Width of the kernel.

Spline Kernel

The thin-plate spline kernel is defined as follows:

\[\Phi(r) = r^2 \log \left(r\right),\]

where \(r=\|y-x\|\), where \(\|\cdot\|\) is the Euclidean norm.

class RONAALP.utilities.kernels.spline.RBF_Spline(epsilon=0.001, degree=1, phi_min=0.005)

RBF thinplate spline kernel class.

Parameters

degreeint, default = 1

Highest degree of added polynomial terms.

epsilonfloat, default = 1e-3

Smoothing parameter.

phi_minfloat, default = 5e-3

Distance threshold for extrapolation detection.

Attributes

sndarray, shape(m+monomes(degree),)

RBF coefficients: first m coefficients correspond to each kernel, followed by monomes(degree) coeffiecients for polynomial terms.

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,p,)

Array of function values at x.

f0ndarray, shape (m+monomes(degree),p,)

Array of function values at x supplemented with zeros for polyharmonic terms.

Kinvndarray, shape (m,m,)

Inverse of the RBF kernel matrix Phi.

evaluate(y)

Evaluate surrogate model at given points.

Parameters

yndarray, shape (n,d,)

Array of points where we want to evaluate the surrogate model.

Returns

fndarray, shape(n,p,)

Array of interpolated values.

extrp: ndarray, shape(n,)

Array of extrapolation flag:

0 -> interpolation, 1 -> extrapolation.

fit(x, f)

Build surrogate model.

Parameters

xndarray, shape (m,d,)

Array of points where function values are known. m is the number of sampling points and d is the number of dimensions.

fndarray, shape (m,p)

Array of function values at x.

Returns

Phindarray, shape(m,m,)

RBF matrix.

retrain(x_new, f_new)

Retrain the surrogate model in a brute force approach.

Parameters

x_newndarray, shape (m2,d,)

New array of points where function values are known. m2 is the number of new sampling points.

f_newndarray, shape (m2,p)

Array of new function values at x_new.

retrain_schur(x_new, f_new)

Efficiently retrain the surrogate model using the Schur complement.

Parameters

x_newndarray, shape (m2,d,)

New array of points where function values are known. m2 is the number of new sampling points.

f_newndarray, shape (m2,p)

Array of new function values at x_new.

RONAALP.utilities.kernels.spline.monomial_powers(ndim, degree)

Return the powers for each monomial in a polynomial.

Parameters

ndimint

Number of variables in the polynomial.

degreeint

Degree of the polynomial.

Returns

(nmonos, ndim) int ndarray

Array where each row contains the powers for each variable in a monomial.

RONAALP.utilities.kernels.spline.polynomial_matrix(x, powers)

Evaluate monomials, with exponents from powers, at x.