Polynomial Regression in Python

Polynomial Regression is a type of regression that models the relationship between the independent variable $x$ and the dependent variable $y$ as an nth-degree polynomial. It is a useful method when the relationship between variables is not linear but can be approximated by a polynomial function.

In Python, polynomial regression can be implemented using NumPy and Scikit-learn. Scikit-learn provides the PolynomialFeatures class, which is used to generate polynomial features from the input data, and the LinearRegression class to fit the model.

Steps for Polynomial Regression:

1. Create the data: Generate or load the data to be used.

2. Transform the data: Use PolynomialFeatures to create polynomial features.

3. Fit the model: Use LinearRegression to fit a linear regression model to the polynomial features.

4. Visualize the result: Plot the polynomial regression curve.

Example of Polynomial Regression in Python

Let's walk through an example of polynomial regression using some sample data:

1. Import necessary libraries

import numpy as npimport matplotlib.pyplot as pltfrom sklearn.linear_model import LinearRegressionfrom sklearn.preprocessing import PolynomialFeatures

2. Create the data

For this example, let's create some sample data that has a non-linear relationship.

# Sample data (X and Y)X = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).reshape(-1, 1)  # Feature (Independent variable)y = np.array([1, 4, 9, 16, 25, 36, 49, 64, 81, 100])  # Target (Dependent variable)

3. Transform the data into polynomial features

We'll transform the original data into polynomial features. In this case, we'll use a quadratic (degree 2) polynomial.

# Create polynomial features (degree 2 for quadratic regression)poly = PolynomialFeatures(degree=2)X_poly = poly.fit_transform(X)

4. Fit the model

Now, we will use LinearRegression to fit the model to the polynomial features.

# Fit the polynomial regression modelmodel = LinearRegression()model.fit(X_poly, y)

5. Visualize the result

We can plot the original data points and the polynomial regression curve to visualize the fit.

# Predict using the fitted modely_poly_pred = model.predict(X_poly)# Plot the original dataplt.scatter(X, y, color='blue', label='Original Data')# Plot the polynomial regression curveplt.plot(X, y_poly_pred, color='red', label='Polynomial Regression (degree 2)')# Add labels and titleplt.xlabel('X')plt.ylabel('y')plt.title('Polynomial Regression')# Show legend and plotplt.legend()plt.show()

Complete Code Example

import numpy as npimport matplotlib.pyplot as pltfrom sklearn.linear_model import LinearRegressionfrom sklearn.preprocessing import PolynomialFeatures# Sample data (X and Y)X = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).reshape(-1, 1)  # Feature (Independent variable)y = np.array([1, 4, 9, 16, 25, 36, 49, 64, 81, 100])  # Target (Dependent variable)# Create polynomial features (degree 2 for quadratic regression)poly = PolynomialFeatures(degree=2)X_poly = poly.fit_transform(X)# Fit the polynomial regression modelmodel = LinearRegression()model.fit(X_poly, y)# Predict using the fitted modely_poly_pred = model.predict(X_poly)# Plot the original dataplt.scatter(X, y, color='blue', label='Original Data')# Plot the polynomial regression curveplt.plot(X, y_poly_pred, color='red', label='Polynomial Regression (degree 2)')# Add labels and titleplt.xlabel('X')plt.ylabel('y')plt.title('Polynomial Regression')# Show legend and plotplt.legend()plt.show()

Output:

• The scatter plot shows the original data points.

• The red curve represents the polynomial regression curve that fits the data. In this case, it is a quadratic (degree 2) polynomial.

Adjusting the Degree of the Polynomial

The degree of the polynomial determines the flexibility of the regression curve. A higher degree allows the curve to fit the data more closely, but it also increases the risk of overfitting. To try higher-degree polynomials, you can change the degree parameter of PolynomialFeatures.

For example, to fit a cubic polynomial (degree 3), you can modify the code as follows:

poly = PolynomialFeatures(degree=3)

Polynomial Regression vs. Linear Regression

• Linear Regression: Models the relationship between the independent variable $x$ and dependent variable $y$ as a straight line.

• Polynomial Regression: Models the relationship using a polynomial function (e.g., quadratic, cubic), which is useful when the relationship between $x$ and $y$ is not linear.

Polynomial regression can fit data that is non-linear, while linear regression can only fit linear relationships.

Conclusion

Polynomial regression is a useful method when the data has a non-linear relationship. Using Scikit-learn's PolynomialFeatures and LinearRegression, you can easily fit polynomial regression models of different degrees to your data. Always be cautious about choosing the correct degree, as higher-degree polynomials can lead to overfitting.