Skip to content

CMIM

skfeature.function.information_theoretical_based.cmim

Description

CMIM (Conditional Mutual Information Maximization) scores features as J(f) = I(f; y) - max_j(I(fj; f) - I(fj; f|y)), capping the redundancy penalty via the most similar selected feature.

Note

This information-theoretic method requires discrete input features. Discretize continuous data first, for example with sklearn.preprocessing.KBinsDiscretizer.

Usage

import numpy as np
from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest
from sklearn.preprocessing import KBinsDiscretizer

from skfeature.function.information_theoretical_based import cmim

X, y = load_iris(return_X_y=True)

# information-theoretic scores require discrete features
X = KBinsDiscretizer(n_bins=5, encode="ordinal").fit_transform(X).astype(float)

# integrate with scikit-learn pipelines via SelectKBest
selector = SelectKBest(score_func=cmim.cmim, k=5)
X_selected = selector.fit_transform(X, y)

Parameters

  • mode: {{"rank", "index"}}, default "rank" — "rank" returns an array of feature indices ordered by importance and aligned with sklearn.feature_selection.SelectKBest; "index" returns the indices of the selected features with the most important one first
  • X: numpy array, shape (n_samples, n_features) — input data, must be discrete
  • y: numpy array, shape (n_samples,) — class labels
  • **kwargs: additional parameters (see n_selected_features below)

Optional keyword arguments:

  • n_selected_features: int — number of features to select

Returns

  • score: numpy array, shape (n_features,) — ranking score of every feature, aligned with sklearn.feature_selection.SelectKBest

References

  • Fleuret, François. "Fast binary feature selection with conditional mutual information." JMLR 2004.