posthoc.WorkbenchOptimizer

class posthoc.WorkbenchOptimizer(model, cov=None, cov_param_x0=None, cov_param_bounds=None, pattern_modifier=None, pattern_param_x0=None, pattern_param_bounds=None, normalizer_modifier=None, normalizer_param_x0=None, normalizer_param_bounds=None, random_search=0, optimizer_options=None, loo_patterns_method='auto', scoring='neg_mean_squared_error', random_state=None, verbose=True)

Work bench for post-hoc alteration of a linear model, with optimization.

Decomposes the .coef_ of a linear model into a covariance matrix, a pattern and a normalizer. These components are then altered by user speficied functions and the linear model is re-assembled.

In this optimizing version of the workbench, the functions speficied to alter the components of the model can have free parameters. These parameters will be optimized using an inner leave-one-out cross validation loop, using a general purpose convex optimization algorithm (L-BFGS-B).

Parameters:
model : instance of sklearn.linear_model.LinearModel

The linear model to alter.

cov : instance of CovEstimator

The method used to estimate the covariance. Can either be one of the predefined CovEstimator objects, or a function. If a function is used, it must have the signature: def cov_modifier(cov, X, y), take the training data as input and return a matrix that will be added to the emperical covariance matrix. Defaults to None, which means the default empirical estimator of the covariance matrix is used.

cov_param_x0 : tuple | None

The initial parameters for the covariance estimator. These parameters will be optimized. Defaults to None, which means the settings defined in the CovEstimator object will be used.

cov_param_bounds : list of tuple | None

For each parameter for the covariance estimator, the (min, max) bounds. You can set a boundary to None to indicate the parameter is unbounded in that direction. By default, the settings defined in the CovEstimator object will be used.

pattern_modifier : function | None

Function that takes a pattern (an ndarray of shape (n_features, n_targets)) and modifies it. Must have the signature: def pattern_modifier(pattern, X, y) and return the modified pattern. Defaults to None, which means no modification of the pattern.

pattern_param_x0 : tuple | None

The initial parameters for the pattern modifier function. These parameters will be optimized. Defaults to None, which means no parameters of the pattern modifier function will be optimized.

pattern_param_bounds : list of tuple | None

For each parameter for the pattern modifier function, the (min, max) bounds. You can set a boundary to None to indicate the parameter is unbounded in that direction. By default, all parameters are considered unbounded.

normalizer_modifier : function | None

Function that takes a normalizer (an ndarray of shape (n_targets, n_targets)) and modifies it. Must have the signature: def normalizer_modifier(normalizer, X, y, pattern, coef) and return the modified normalizer. Defaults to None, which means no modification of the normalizer.

normalizer_param_x0 : tuple | None

The initial parameters for the normalizer modifier function. These parameters will be optimized. Defaults to None, which means no parameters of the normalizer modifier function will be optimized.

normalizer_param_bounds : list of tuple | None

For each parameter for the normalizer modifier function, the (min, max) bounds. You can set a boundary to None to indicate the parameter is unbounded in that direction. By default, all parameters are considered unbounded.

random_search : int

Number of random sets of parameters to try, before picking the best set to use as starting point for the L-BFGS-S algorithm. Set to 0 to disable the random search and use the specified x0 instead. Defaults to 0.

optimizer_options : dict | None

A dictionary with extra options to supply to the L-BFGS-S algorithm. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html.http://scikit-learn.org/stable/modules/model_evaluation.html for a list of parameters.

loo_patterns_method : ‘auto’ | ‘traditional’ | ‘kernel’

Method for computing the pattern matrix for the leave-one-out inner crossvalidation loop. The ‘traditional’ setting will compute it by performing a linear regression between the output of the initial model and the weight matrix, as described in [1]. The ‘kernel’ setting will use the matrix inversion lemma to compute the patterns, which can be much faster when the number of features is larger than the number of training samples, at the possible loss of some precision. The default ‘auto’ setting will switch between the ‘traditional’ and ‘kernel’ approaches based on the ratio between the number of features and training samples.

scoring : str | function

The scoring function to use while optimizating the parameters. Can be any scikit-learn scorer, see: http://scikit-learn.org/stable/modules/model_evaluation.html. Defaults to 'neg_mean_squared_error' which causes the optimization algorithm to attempt to minimize the mean-squared-error (MSE) between the model output and training labels.

random_state : numpy.random.RandomState | int | None

Random state to use. Can either be a NumPy numpy.random.RandomState object, an integer seed or None to use a different seed every time. Defaults to None.

verbose : bool

Whether to be verbose or not. Defaults to True.

Attributes:
coef_ : ndarray, shape (n_targets, n_features)

Matrix containing the filter weights.

intercept_ : ndarray, shape (n_targets)

The intercept of the linear model.

pattern_ : ndarray, shape (n_features, n_targets)

The altered pattern.

normalizer_ : ndarray, shape (n_targets, n_targets)

The altered normalizer.

Methods

fit(X, y) Fit the model to the data and optimize all parameters.
fit_transform(X[, y]) Fit to data, then transform it.
get_params([deep]) Get parameters for this estimator.
inverse_predict(y) Apply the inverse of the linear model to the targets.
predict(X) Predict using the linear model
score(X, y[, sample_weight]) Returns the coefficient of determination R^2 of the prediction.
set_params(**params) Set the parameters of this estimator.
transform(X) Apply the model to the data.
__hash__($self, /)

Return hash(self).