Linear Regression in Python

Linear Regression is a statistical method used to model the relationship between a dependent variable (target) and one or more independent variables (predictors). The goal is to find the best-fitting straight line (in the case of simple linear regression) or hyperplane (in the case of multiple linear regression) that can predict the target variable based on the input features.

In Python, scikit-learn is the most commonly used library for implementing linear regression.

Steps to Perform Linear Regression

We'll use scikit-learn to perform Linear Regression in Python. Here's how you can do it:

1. Install Required Libraries

First, ensure that you have the necessary libraries installed. You can install them via pip:

pip install numpy pandas matplotlib scikit-learn

2. Import Required Libraries

import numpy as npimport pandas as pdimport matplotlib.pyplot as pltfrom sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LinearRegressionfrom sklearn.metrics import mean_squared_error, r2_score

• numpy: For numerical operations.

• pandas: For handling datasets.

• matplotlib.pyplot: For visualizing the data and regression line.

• train_test_split: To split the data into training and test sets.

• LinearRegression: For performing linear regression.

• mean_squared_error, r2_score: For model evaluation.

3. Prepare the Dataset

Here, we will generate a simple synthetic dataset. However, in a real-world scenario, you'd likely be using data from a CSV file or a database.

# Generate some data for demonstration# X will be the input (e.g., years of experience) and y will be the output (e.g., salary)X = np.random.rand(100, 1) * 10  # 100 random values between 0 and 10y = 2 * X + 3 + np.random.randn(100, 1)  # Linear relationship with some noise (y = 2X + 3)

Here:

• X is the independent variable (e.g., years of experience).

• y is the dependent variable (e.g., salary), with some random noise added.

4. Split the Data into Training and Testing Sets

We will split the data into training (80%) and testing (20%) sets to evaluate the model's performance.

# Split the data into training and testing sets (80% train, 20% test)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

5. Create and Train the Linear Regression Model

Now, let's create a linear regression model and fit it to the training data.

# Create a Linear Regression modelmodel = LinearRegression()# Train the model on the training datamodel.fit(X_train, y_train)

• The fit() method fits the model to the training data.

6. Make Predictions

Now that the model is trained, you can use it to make predictions on the test set.

# Make predictions using the test datay_pred = model.predict(X_test)

7. Evaluate the Model

We can evaluate the performance of the model using metrics like Mean Squared Error (MSE) and R-squared (R²).

# Calculate the Mean Squared Error (MSE)mse = mean_squared_error(y_test, y_pred)print(f"Mean Squared Error: {mse:.2f}")# Calculate the R-squared scorer2 = r2_score(y_test, y_pred)print(f"R-squared: {r2:.2f}")

• Mean Squared Error (MSE): Measures the average squared difference between the predicted and actual values. A lower MSE means a better fit.

• R-squared: Measures how well the model fits the data. A value of 1 indicates perfect prediction, and values closer to 0 indicate poor fit.

8. Visualize the Results

You can visualize the regression line along with the actual data to see how well the model fits.

# Plot the test data and the regression lineplt.scatter(X_test, y_test, color='blue', label='Test data')plt.plot(X_test, y_pred, color='red', label='Regression line')plt.title('Linear Regression')plt.xlabel('X (Input)')plt.ylabel('y (Output)')plt.legend()plt.show()

This will create a scatter plot of the test data and the predicted regression line.

Complete Example Code

Here’s the complete code from start to finish:

import numpy as npimport pandas as pdimport matplotlib.pyplot as pltfrom sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LinearRegressionfrom sklearn.metrics import mean_squared_error, r2_score# Generate some random dataX = np.random.rand(100, 1) * 10  # 100 random values between 0 and 10y = 2 * X + 3 + np.random.randn(100, 1)  # Linear relationship with some noise# Split the data into training and testing setsX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# Create a Linear Regression modelmodel = LinearRegression()# Train the model on the training datamodel.fit(X_train, y_train)# Make predictions on the test datay_pred = model.predict(X_test)# Evaluate the modelmse = mean_squared_error(y_test, y_pred)r2 = r2_score(y_test, y_pred)print(f"Mean Squared Error: {mse:.2f}")print(f"R-squared: {r2:.2f}")# Visualize the resultsplt.scatter(X_test, y_test, color='blue', label='Test data')plt.plot(X_test, y_pred, color='red', label='Regression line')plt.title('Linear Regression')plt.xlabel('X (Input)')plt.ylabel('y (Output)')plt.legend()plt.show()

Multiple Linear Regression

If you have more than one feature (i.e., multiple independent variables), you can still use LinearRegression for Multiple Linear Regression. The process is the same, except that X will be a 2D array with multiple columns.

Example:

# Generate some random data with multiple featuresX = np.random.rand(100, 3) * 10  # 100 samples with 3 featuresy = 2 * X[:, 0] + 3 * X[:, 1] + 4 * X[:, 2] + 5 + np.random.randn(100, 1)  # Linear relationship with noise# Split the data into training and testing setsX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# Create and train the modelmodel = LinearRegression()model.fit(X_train, y_train)# Predictions and Evaluationy_pred = model.predict(X_test)mse = mean_squared_error(y_test, y_pred)r2 = r2_score(y_test, y_pred)print(f"Mean Squared Error: {mse:.2f}")print(f"R-squared: {r2:.2f}")

Conclusion

• Linear Regression is a simple and effective method for predicting a continuous target variable based on one or more features.

• Python’s scikit-learn makes it easy to implement linear regression, evaluate the model, and visualize the results.

• In Multiple Linear Regression, you have more than one input feature to predict the target.

Let me know if you need further clarification or help!