Classes

class ShapGAM (n_estimators=10, feature_fraction=0.7, random_state=None, ebm_kwargs: dict = {})

Base class for all estimators in scikit-learn.

Inheriting from this class provides default implementations of:

• setting and getting parameters used by GridSearchCV and friends;
• textual and HTML representation displayed in terminals and IDEs;
• estimator serialization;
• parameters validation;
• data validation;
• feature names validation.

Read more in the :ref:User Guide <rolling_your_own_estimator>.

Notes

All estimators should specify all the parameters that can be set at the class level in their __init__ as explicit keyword arguments (no *args or **kwargs).

Examples

>>> import numpy as np
>>> from sklearn.base import BaseEstimator
>>> class MyEstimator(BaseEstimator):
...     def __init__(self, *, param=1):
...         self.param = param
...     def fit(self, X, y=None):
...         self.is_fitted_ = True
...         return self
...     def predict(self, X):
...         return np.full(shape=X.shape[0], fill_value=self.param)
>>> estimator = MyEstimator(param=2)
>>> estimator.get_params()
{'param': 2}
>>> X = np.array([[1, 2], [2, 3], [3, 4]])
>>> y = np.array([1, 0, 1])
>>> estimator.fit(X, y).predict(X)
array([2, 2, 2])
>>> estimator.set_params(param=3).fit(X, y).predict(X)
array([3, 3, 3])

Initialize the ensemble EBM classifier.

Parameters

n_estimators : int, default=10

Number of EBM classifiers to create with different random subsets of features.

feature_fraction : float or "uniform", default=0.7

Fraction of features to use for each EBM classifier ("uniform": a random fraction each time).

random_state : int, optional

Seed for the random number generator, for reproducibility.

ebm_kwargs : dict, optional

Keyword arguments passed to each ExplainableBoostingMachine.

Expand source code

class ShapGAM(BaseEstimator):
    def __init__(self, n_estimators=10, feature_fraction=0.7, random_state=None, ebm_kwargs: dict = {}):
        """
        Initialize the ensemble EBM classifier.

        Parameters
        ----------
        n_estimators : int, default=10
            Number of EBM classifiers to create with different random subsets of features.
        feature_fraction : float or "uniform", default=0.7
            Fraction of features to use for each EBM classifier ("uniform": a random fraction each time).
        random_state : int, optional
            Seed for the random number generator, for reproducibility.
        ebm_kwargs : dict, optional
            Keyword arguments passed to each ExplainableBoostingMachine.
        """
        require_optional_dependency(
            'interpret', type(self).__name__,
            purpose='provides the ExplainableBoostingMachine used for each ensemble member')
        self.n_estimators = n_estimators
        self.feature_fraction = feature_fraction
        self.random_state = random_state
        self.models = []
        self.feature_subsets = []
        self.ebm_kwargs = ebm_kwargs

    def fit(self, X, y):
        """
        Fit the ensemble of EBM classifiers on random feature subsets.
        """
        rng = np.random.default_rng(self.random_state)
        n_features = X.shape[1]

        for _ in range(self.n_estimators):
            # Randomly select a subset of features
            if isinstance(self.feature_fraction, float):
                n_features = max(1, int(self.feature_fraction * n_features))
            elif self.feature_fraction == 'uniform':
                n_features = rng.integers(1, n_features + 1)
            feature_subset = rng.choice(
                n_features,
                size=n_features,
                replace=False
            )
            self.feature_subsets.append(feature_subset)

            # Create an EBM with the selected feature subset
            cls = ExplainableBoostingClassifier if isinstance(
                self, ShapGAMClassifier) else ExplainableBoostingRegressor
            ebm = cls(
                random_state=self.random_state, **self.ebm_kwargs)
            X_subset = X[:, feature_subset]
            ebm.fit(X_subset, y)
            self.models.append(ebm)

        return self

    def predict_proba(self, X):
        """
        Predict probabilities by averaging the shaping functions for each feature across models then predicting
        """
        if isinstance(self, ShapGAMRegressor):
            raise NotImplementedError(
                "This method is not implemented for regression tasks.")
        # Aggregate predictions from all models
        probs = np.zeros((X.shape[0], 2))

        for feat_num in range(X.shape[1]):
            X_ = np.zeros_like(X)
            X_[:, feat_num] = X[:, feat_num]
            feat_num_counts = 0
            probs_feat = np.zeros((X.shape[0], 2))
            for ebm, feature_subset in zip(self.models, self.feature_subsets):
                if feat_num in feature_subset:
                    # todo: there is an error here -- aggregating probas is not the same as aggregating logits!
                    raise NotImplementedError(
                        "This is not the same as aggregating logits Look at the predict function for smth more accurate!")
                    probs_ = ebm.predict_proba(X[:, feature_subset])
                    probs_feat += probs_
                    feat_num_counts += 1
            if feat_num_counts > 0:
                probs_feat /= feat_num_counts
                probs += probs_feat

                # for ebm, feature_subset in zip(self.models, self.feature_subsets):
                # probs_ = ebm.predict_proba(X[:, feature_subset])
                # probs += probs_

                # Average the probabilities across all models
                # probs /= self.n_estimators
        return probs

    def predict(self, X):
        """
        Predict class labels by averaging the shape function outputs and taking the argmax.
        """
        if isinstance(self, ShapGAMClassifier):
            return np.argmax(self.predict_proba(X), axis=1)
        else:
            # naively averaging the predictions across all models
            # preds = np.zeros(X.shape[0])
            # for ebm, feature_subset in zip(self.models, self.feature_subsets):
            #     preds += ebm.predict(X[:, feature_subset])
            # preds /= self.n_estimators
            # return preds

            # averaging the predictions across all models for each feature
            preds = np.zeros(len(X))
            for ex_num in range(len(X)):
                for feat_num in range(X.shape[1]):
                    feat_count = 0
                    pred_feat = 0
                    for ebm, feature_subset in zip(self.models, self.feature_subsets):
                        if feat_num in feature_subset:
                            expl_local = ebm.explain_local(
                                X[ex_num, feature_subset])
                            feat_num_in_subset = np.where(
                                feature_subset == feat_num)[0][0]
                            pred_feat += expl_local.data(
                                0)['scores'][feat_num_in_subset]
                            feat_count += 1
                    if feat_count > 0:
                        pred_feat /= feat_count
                    preds[ex_num] += pred_feat

            return preds

    _PARAM_NAMES = ("n_estimators", "feature_fraction", "random_state")

    def get_params(self, deep=True):
        return explicit_get_params(self, self._PARAM_NAMES, deep=deep)

    def set_params(self, **params):
        return explicit_set_params(self, self._PARAM_NAMES, **params)

Ancestors

• sklearn.base.BaseEstimator
• sklearn.utils._estimator_html_repr._HTMLDocumentationLinkMixin
• sklearn.utils._metadata_requests._MetadataRequester

Subclasses

• ShapGAMClassifier
• ShapGAMRegressor

Methods

def fit(self, X, y)

Fit the ensemble of EBM classifiers on random feature subsets.

Expand source code

def fit(self, X, y):
    """
    Fit the ensemble of EBM classifiers on random feature subsets.
    """
    rng = np.random.default_rng(self.random_state)
    n_features = X.shape[1]

    for _ in range(self.n_estimators):
        # Randomly select a subset of features
        if isinstance(self.feature_fraction, float):
            n_features = max(1, int(self.feature_fraction * n_features))
        elif self.feature_fraction == 'uniform':
            n_features = rng.integers(1, n_features + 1)
        feature_subset = rng.choice(
            n_features,
            size=n_features,
            replace=False
        )
        self.feature_subsets.append(feature_subset)

        # Create an EBM with the selected feature subset
        cls = ExplainableBoostingClassifier if isinstance(
            self, ShapGAMClassifier) else ExplainableBoostingRegressor
        ebm = cls(
            random_state=self.random_state, **self.ebm_kwargs)
        X_subset = X[:, feature_subset]
        ebm.fit(X_subset, y)
        self.models.append(ebm)

    return self

def get_params(self, deep=True)

Get parameters for this estimator.

Parameters

deep : bool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns

params : dict

Parameter names mapped to their values.

Expand source code

def get_params(self, deep=True):
    return explicit_get_params(self, self._PARAM_NAMES, deep=deep)

def predict(self, X)

Predict class labels by averaging the shape function outputs and taking the argmax.

Expand source code

def predict(self, X):
    """
    Predict class labels by averaging the shape function outputs and taking the argmax.
    """
    if isinstance(self, ShapGAMClassifier):
        return np.argmax(self.predict_proba(X), axis=1)
    else:
        # naively averaging the predictions across all models
        # preds = np.zeros(X.shape[0])
        # for ebm, feature_subset in zip(self.models, self.feature_subsets):
        #     preds += ebm.predict(X[:, feature_subset])
        # preds /= self.n_estimators
        # return preds

        # averaging the predictions across all models for each feature
        preds = np.zeros(len(X))
        for ex_num in range(len(X)):
            for feat_num in range(X.shape[1]):
                feat_count = 0
                pred_feat = 0
                for ebm, feature_subset in zip(self.models, self.feature_subsets):
                    if feat_num in feature_subset:
                        expl_local = ebm.explain_local(
                            X[ex_num, feature_subset])
                        feat_num_in_subset = np.where(
                            feature_subset == feat_num)[0][0]
                        pred_feat += expl_local.data(
                            0)['scores'][feat_num_in_subset]
                        feat_count += 1
                if feat_count > 0:
                    pred_feat /= feat_count
                preds[ex_num] += pred_feat

        return preds

def predict_proba(self, X)

Predict probabilities by averaging the shaping functions for each feature across models then predicting

Expand source code

def predict_proba(self, X):
    """
    Predict probabilities by averaging the shaping functions for each feature across models then predicting
    """
    if isinstance(self, ShapGAMRegressor):
        raise NotImplementedError(
            "This method is not implemented for regression tasks.")
    # Aggregate predictions from all models
    probs = np.zeros((X.shape[0], 2))

    for feat_num in range(X.shape[1]):
        X_ = np.zeros_like(X)
        X_[:, feat_num] = X[:, feat_num]
        feat_num_counts = 0
        probs_feat = np.zeros((X.shape[0], 2))
        for ebm, feature_subset in zip(self.models, self.feature_subsets):
            if feat_num in feature_subset:
                # todo: there is an error here -- aggregating probas is not the same as aggregating logits!
                raise NotImplementedError(
                    "This is not the same as aggregating logits Look at the predict function for smth more accurate!")
                probs_ = ebm.predict_proba(X[:, feature_subset])
                probs_feat += probs_
                feat_num_counts += 1
        if feat_num_counts > 0:
            probs_feat /= feat_num_counts
            probs += probs_feat

            # for ebm, feature_subset in zip(self.models, self.feature_subsets):
            # probs_ = ebm.predict_proba(X[:, feature_subset])
            # probs += probs_

            # Average the probabilities across all models
            # probs /= self.n_estimators
    return probs

def set_params(self, **params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as :class:~sklearn.pipeline.Pipeline). The latter have parameters of the form <component>__<parameter> so that it's possible to update each component of a nested object.

Parameters

**params : dict

Estimator parameters.

Returns

self : estimator instance

Estimator instance.

Expand source code

def set_params(self, **params):
    return explicit_set_params(self, self._PARAM_NAMES, **params)

class ShapGAMClassifier (n_estimators=10, feature_fraction=0.7, random_state=None, ebm_kwargs: dict = {})

Base class for all estimators in scikit-learn.

Inheriting from this class provides default implementations of:

• setting and getting parameters used by GridSearchCV and friends;
• textual and HTML representation displayed in terminals and IDEs;
• estimator serialization;
• parameters validation;
• data validation;
• feature names validation.

Read more in the :ref:User Guide <rolling_your_own_estimator>.

Notes

All estimators should specify all the parameters that can be set at the class level in their __init__ as explicit keyword arguments (no *args or **kwargs).

Examples

>>> import numpy as np
>>> from sklearn.base import BaseEstimator
>>> class MyEstimator(BaseEstimator):
...     def __init__(self, *, param=1):
...         self.param = param
...     def fit(self, X, y=None):
...         self.is_fitted_ = True
...         return self
...     def predict(self, X):
...         return np.full(shape=X.shape[0], fill_value=self.param)
>>> estimator = MyEstimator(param=2)
>>> estimator.get_params()
{'param': 2}
>>> X = np.array([[1, 2], [2, 3], [3, 4]])
>>> y = np.array([1, 0, 1])
>>> estimator.fit(X, y).predict(X)
array([2, 2, 2])
>>> estimator.set_params(param=3).fit(X, y).predict(X)
array([3, 3, 3])

Initialize the ensemble EBM classifier.

Parameters

n_estimators : int, default=10

Number of EBM classifiers to create with different random subsets of features.

feature_fraction : float or "uniform", default=0.7

Fraction of features to use for each EBM classifier ("uniform": a random fraction each time).

random_state : int, optional

Seed for the random number generator, for reproducibility.

ebm_kwargs : dict, optional

Keyword arguments passed to each ExplainableBoostingMachine.

Expand source code

class ShapGAMClassifier(ShapGAM, ClassifierMixin):
    ...

Ancestors

• ShapGAM
• sklearn.base.BaseEstimator
• sklearn.utils._estimator_html_repr._HTMLDocumentationLinkMixin
• sklearn.utils._metadata_requests._MetadataRequester
• sklearn.base.ClassifierMixin

Inherited members

• ShapGAM:
  • fit
  • get_params
  • predict
  • predict_proba
  • set_params

class ShapGAMRegressor (n_estimators=10, feature_fraction=0.7, random_state=None, ebm_kwargs: dict = {})

Base class for all estimators in scikit-learn.

Inheriting from this class provides default implementations of:

• setting and getting parameters used by GridSearchCV and friends;
• textual and HTML representation displayed in terminals and IDEs;
• estimator serialization;
• parameters validation;
• data validation;
• feature names validation.

Read more in the :ref:User Guide <rolling_your_own_estimator>.

Notes

All estimators should specify all the parameters that can be set at the class level in their __init__ as explicit keyword arguments (no *args or **kwargs).

Examples

>>> import numpy as np
>>> from sklearn.base import BaseEstimator
>>> class MyEstimator(BaseEstimator):
...     def __init__(self, *, param=1):
...         self.param = param
...     def fit(self, X, y=None):
...         self.is_fitted_ = True
...         return self
...     def predict(self, X):
...         return np.full(shape=X.shape[0], fill_value=self.param)
>>> estimator = MyEstimator(param=2)
>>> estimator.get_params()
{'param': 2}
>>> X = np.array([[1, 2], [2, 3], [3, 4]])
>>> y = np.array([1, 0, 1])
>>> estimator.fit(X, y).predict(X)
array([2, 2, 2])
>>> estimator.set_params(param=3).fit(X, y).predict(X)
array([3, 3, 3])

Initialize the ensemble EBM classifier.

Parameters

n_estimators : int, default=10

Number of EBM classifiers to create with different random subsets of features.

feature_fraction : float or "uniform", default=0.7

Fraction of features to use for each EBM classifier ("uniform": a random fraction each time).

random_state : int, optional

Seed for the random number generator, for reproducibility.

ebm_kwargs : dict, optional

Keyword arguments passed to each ExplainableBoostingMachine.

Expand source code

class ShapGAMRegressor(ShapGAM, RegressorMixin):
    ...

Ancestors

• ShapGAM
• sklearn.base.BaseEstimator
• sklearn.utils._estimator_html_repr._HTMLDocumentationLinkMixin
• sklearn.utils._metadata_requests._MetadataRequester
• sklearn.base.RegressorMixin

Inherited members

• ShapGAM:
  • fit
  • get_params
  • predict
  • predict_proba
  • set_params