? Bootstrap Aggregation (Bagging) in Python

Bootstrap Aggregation (Bagging) is an ensemble learning technique used to improve the stability and accuracy of machine learning models by combining multiple weak learners. It works by training multiple instances of the same algorithm on different random subsets of data and averaging their predictions (for regression) or using majority voting (for classification).

? Steps to Implement Bagging in Python

We will use Scikit-learn to implement Bagging.

1?? Import Required Libraries

import numpy as npimport matplotlib.pyplot as pltfrom sklearn.ensemble import BaggingClassifierfrom sklearn.tree import DecisionTreeClassifierfrom sklearn.datasets import make_classificationfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import accuracy_score

2?? Generate Sample Data

# Create a synthetic datasetX, y = make_classification(n_samples=1000, n_features=10, random_state=42)# Split into training and test setsX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

3?? Train a Decision Tree Without Bagging

# Train a single Decision Treedt = DecisionTreeClassifier(random_state=42)dt.fit(X_train, y_train)# Predict and evaluatey_pred_dt = dt.predict(X_test)accuracy_dt = accuracy_score(y_test, y_pred_dt)print("Accuracy (Decision Tree):", accuracy_dt)

4?? Train a Bagging Classifier

# Use Bagging with Decision Treesbagging = BaggingClassifier(    base_estimator=DecisionTreeClassifier(),  # Weak learner    n_estimators=50,  # Number of models    max_samples=0.8,  # 80% of data sampled for each model    bootstrap=True,   # Sampling with replacement    random_state=42)bagging.fit(X_train, y_train)# Predict and evaluatey_pred_bagging = bagging.predict(X_test)accuracy_bagging = accuracy_score(y_test, y_pred_bagging)print("Accuracy (Bagging with Decision Trees):", accuracy_bagging)

5?? Compare Performance

print(f"Single Decision Tree Accuracy: {accuracy_dt:.2f}")print(f"Bagging Classifier Accuracy: {accuracy_bagging:.2f}")

Bagging typically improves accuracy and reduces overfitting compared to a single Decision Tree.

? When to Use Bagging?

? Works well with high-variance models (like Decision Trees).
? Reduces overfitting.
? Ideal when you have limited data but want better performance.

Do you need an example for regression too? ?