### mRMR 特征选择方法概述 最小冗余最大相关性(mRMR)是一种用于特征选择的方法，旨在挑选一组具有高类相关性和低互信息冗余度的特征。该算法通过最大化所选特征集合的相关性并最小化这些特征之间的冗余来工作[^1]。 ### 实现mRMR算法的关键要素 为了实现这一目标，在Python中通常会利用`sklearn.feature_selection.mutual_info_classif`函数计算特征间的互信息以及它们与类别标签之间的互信息。接着基于贪心策略迭代地选取最优特征直到满足特定条件为止[^2]。 ```python import numpy as np from sklearn.datasets import load_iris from sklearn.feature_selection import mutual_info_classif from scipy.stats import entropy def calc_redundancy(features, selected_features_indices): """Calculate average redundancy between already chosen features.""" redundancies = [] for i in range(len(selected_features_indices)): for j in range(i + 1, len(selected_features_indices)): f_i_idx = selected_features_indices[i] f_j_idx = selected_features_indices[j] mi_fi_fj = mutual_info_classif( features[:, [f_i_idx]], features[:, f_j_idx], discrete_features=True)[0] redundancies.append(mi_fi_fj) return sum(redundancies)/len(redundancies) def mrmr_select(X, y, num_features=5): """ Select top `num_features` using Minimum Redundancy Maximum Relevance algorithm. Parameters: X : array-like of shape (n_samples, n_features) Training vectors with multiple features. y : array-like of shape (n_samples,) Target values or class labels. num_features : int, default=5 Number of best features to select from the dataset. Returns: List containing indices corresponding to the most relevant and least redundant features. Note that this is a simplified version which may not cover all edge cases found within real-world datasets. """ # Calculate MI scores for each feature w.r.t target variable Y miscores = mutual_info_classif(X, y, discrete_features=False) # Initialize set S with highest scoring single feature F* s_prime_index = np.argmax(miscores) S = {s_prime_index} while len(S) < num_features: max_score_diff = float('-inf') next_feature_to_add = None for idx in range(X.shape[1]): if idx not in S: current_set_union_new_feat = list(S.union({idx})) avg_mi_with_y = mutual_info_classif( X[:, current_set_union_new_feat], y, discrete_features=False).mean() redun = calc_redundancy(X, current_set_union_new_feat) score_difference = avg_mi_with_y - redun if score_difference > max_score_diff: max_score_diff = score_difference next_feature_to_add = idx S.add(next_feature_to_add) return sorted(list(S)) ``` 上述代码定义了一个名为`mrmr_select()`的功能，它接受训练样本矩阵X及其关联的目标向量y作为输入参数，并返回由索引组成的列表表示最合适的前几个特征的位置。此过程遵循了经典的mRMR框架，即先找到单个最有价值的属性加入到初始解集中；之后每轮循环都尝试寻找能够使得平均互信息增量减去新增加成员带来的内部重复程度达到最大的那个候选项加以扩充直至指定数量为止[^3]。