ATHENA: a first step into Active Subspaces

Let’s begin our journey in parameter space reduction using ATHENA

This is the first of a series of stories focused on how to use the ATHENA Python package [1] and how to enhance your scientific machine learning pipeline.

ATHENA: Parameter space reduction made easy in Python

Here we are going to present the first steps into the magic world of Active Subspaces [2] for parameter space reduction. For more details on the code used here and some references please have a look at this notebook.

Active Subspaces

Let’s prepare some data to analyse. We consider a simple 2D example, where we have a sine propagating along the bisector of the first and third quadrant

In general you should always rescale your inputs into the n-dimensional reference domain [-1, 1]. Let’s have a look at our data:

We would like to find a rotation of the domain such that we can project all the inputs and obtain a good lower dimensional representation of our function. Spoiler alert: such direction exists and its the bisector of the first and third quadrant!

Let’s start by randomly sample 17 points, we compute the corresponding outputs values and the gradients with the following code:

inputs = np.random.uniform(-1, 1, (17, 2))
sin_arg = 2 * np.pi * (inputs[:, 0] + inputs[:, 1])
outputs = np.sin(sin_arg)
gradients = np.array([2 * np.pi * np.cos(sin_arg),
                      2 * np.pi * np.cos(sin_arg)]).T

Now we need to assemble the covariance matrix C of the gradients and decompose it, that is

This is done by the following lines of code:

# compute the active subspace
ss = ActiveSubspaces(dim=1, method='exact')
ss.fit(gradients=gradients)

The partition of the eigenpairs has to be chosen in advance through the parameter dim, alternatively we can plot the eigenvalues decay and select the truncation looking at a significative gap. For this particularly simple example we have 2 eigenvalues and only 1 is actually needed. Here is the code and the resulting plot:

ss.plot_eigenvalues(figsize=(10, 6), title='Eigenvalues decay')

We can also plot the first eigenvector components which tells us the weights of the linear combination of all the input parameters to project the data onto the active subspace. Here’s the code and the plot:

ss.plot_eigenvectors(figsize=(10, 6), title='First eigenvector')

Finally we can plot the outputs f(x)against the active variable obtained by projecting the inputs using the first eigenvector. This kind of plot is called sufficient summary plot and we can call this function with different data withe respect to the ones used to train the model.

ss.plot_sufficient_summary(inputs, outputs, figsize=(10, 6))

After finding the active subspace it is interesting to train a regressor with the inputs mapped along the active dimension and the corresponding outputs. Here we used a Gaussian process and we also plotted some test data to prove the quality of the regression.

Here the sufficient summary plot has been integrated with a Gaussian process trained in the active subspace using the original data.

In the following tutorials we are going to show how to circumvent the need of gradients and some possible applications. Stay tuned!

References

[1] ATHENA: Advanced Techniques for High dimensional parameter spaces to Enhance Numerical Analysis. https://github.com/mathLab/ATHENA

[2] P.G. Constantine. Active subspaces: Emerging ideas for dimension reduction in parameter studies. SIAM Spotlights, 2015.