### Adaboost预测算法的Python实现 Adaboost（自适应提升）是一种经典的集成学习方法，其核心思想是通过加权的方式将多个弱分类器组合成一个强分类器。下面是一个基于Python的Adaboost预测算法的具体实现。 #### 使用NumPy手动实现Adaboost 以下代码展示了如何使用`NumPy`库从零开始构建Adaboost分类器： ```python import numpy as np class AdaBoost: def __init__(self, n_estimators=5): self.n_estimators = n_estimators # 弱分类器的数量 self.models = [] # 存储弱分类器 self.alphas = [] # 每个弱分类器的权重 def fit(self, X, y): n_samples, _ = X.shape w = np.full(n_samples, (1 / n_samples)) # 初始化样本权重 for _ in range(self.n_estimators): model = DecisionStump() # 假设有一个决策树桩类 min_error = float('inf') for feature_i in range(X.shape[1]): thresholds = np.unique(X[:, feature_i]) for threshold in thresholds: p = 1 prediction = np.ones(np.shape(y)) prediction[X[:, feature_i] < threshold] = -1 error = sum(w[y != prediction]) if error > 0.5: error = 1 - error p = -1 if error < min_error: min_error = error best_polarity = p best_threshold = threshold best_feature = feature_i model.polarity = best_polarity model.threshold = best_threshold model.feature_index = best_feature epsilon = 1e-10 # 防止除以零 alpha = 0.5 * np.log((1.0 - min_error + epsilon) / (min_error + epsilon)) model.alpha = alpha predictions = np.ones(np.shape(y)) negative_idx = (model.polarity * X[:, model.feature_index] < model.polarity * model.threshold) predictions[negative_idx] = -1 w *= np.exp(-alpha * y * predictions) w /= np.sum(w) self.models.append(model) self.alphas.append(alpha) def predict(self, X): model_predictions = [alpha * model.predict(X) for model, alpha in zip(self.models, self.alphas)] y_pred = np.sign(sum(model_predictions)).astype(int) return y_pred class DecisionStump: def __init__(self): self.polarity = 1 self.feature_index = None self.threshold = None self.alpha = None def predict(self, X): predictions = np.ones(len(X)) negative_idx = (self.polarity * X[:, self.feature_index] < self.polarity * self.threshold) predictions[negative_idx] = -1 return predictions ``` 此代码实现了Adaboost的核心逻辑，并利用了决策树桩作为基分类器[^3]。 --- #### 使用Scikit-Learn实现Adaboost 如果希望快速应用Adaboost而不需自己编写底层代码，则可以借助`scikit-learn`库中的`AdaBoostClassifier`模块。以下是具体示例： ```python from sklearn.ensemble import AdaBoostClassifier from sklearn.tree import DecisionTreeClassifier from sklearn.datasets import make_classification from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score # 创建数据集 X, y = make_classification(n_samples=100, n_features=2, n_informative=2, n_redundant=0, random_state=1) # 划分训练集和测试集 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1) # 定义Adaboost分类器 adaboost = AdaBoostClassifier( base_estimator=DecisionTreeClassifier(max_depth=1), n_estimators=100, learning_rate=1.0, algorithm='SAMME.R' ) # 训练模型 adaboost.fit(X_train, y_train) # 测试模型并计算精度 y_pred = adaboost.predict(X_test) accuracy = accuracy_score(y_test, y_pred) print(f"Accuracy: {accuracy:.4f}") ``` 这段代码演示了如何使用`scikit-learn`轻松配置和运行Adaboost分类器[^2]。 --- ### 总结 无论是手动实现还是依赖于现成工具包，Adaboost都提供了强大的框架来解决复杂的分类问题。上述两种方法各有优劣：手动实现实现有助于深入理解算法机制；而使用`scikit-learn`则更加高效便捷。