Pandas Tutorial in Python

Pandas is a powerful, open-source library for data analysis and manipulation in Python. It provides data structures like DataFrame and Series that make it easier to work with structured data. Pandas is widely used for tasks like data cleaning, transformation, analysis, and visualization.

Getting Started with Pandas

To begin, you need to install pandas (if it's not already installed) using the following command:

pip install pandas

Once installed, you can import pandas in your Python script:

import pandas as pd

Pandas Data Structures

1. Series

A Series is a one-dimensional array-like object that can hold various types of data (integer, float, string, etc.). It is similar to a list or array in Python but has additional features such as labels (index).

import pandas as pd# Creating a Seriesdata = [10, 20, 30, 40]series = pd.Series(data)print(series)

Output:

0    101    202    303    40dtype: int64

You can also create a Series with custom index labels:

index = ['a', 'b', 'c', 'd']series = pd.Series(data, index=index)print(series)

Output:

a    10b    20c    30d    40dtype: int64

2. DataFrame

A DataFrame is a two-dimensional, size-mutable, and potentially heterogeneous tabular data structure. It is similar to a table, an Excel spreadsheet, or a SQL table.

import pandas as pd# Creating a DataFramedata = {'Name': ['John', 'Anna', 'Peter', 'Linda'],        'Age': [28, 24, 35, 32],        'City': ['New York', 'Paris', 'Berlin', 'London']}df = pd.DataFrame(data)print(df)

Output:

    Name  Age      City0   John   28  New York1   Anna   24     Paris2  Peter   35    Berlin3  Linda   32    London

You can specify row indexes as well:

index = ['a', 'b', 'c', 'd']df = pd.DataFrame(data, index=index)print(df)

Output:

    Name  Age      Citya   John   28  New Yorkb   Anna   24     Parisc  Peter   35    Berlind  Linda   32    London

Basic DataFrame Operations

1. Accessing Columns and Rows

You can access individual columns using the column name:

print(df['Name'])  # Access the 'Name' column

Accessing rows can be done using .loc[] (for labels) or .iloc[] (for integer position):

print(df.loc['a'])  # Access row with index 'a'print(df.iloc[0])   # Access the first row

2. Adding Columns

You can add new columns to a DataFrame:

df['Country'] = ['USA', 'France', 'Germany', 'UK']print(df)

Output:

    Name  Age      City  Countrya   John   28  New York      USAb   Anna   24     Paris   Francec  Peter   35    Berlin  Germanyd  Linda   32    London       UK

3. Deleting Columns

Use the drop() method to delete columns:

df = df.drop('Country', axis=1)  # axis=1 indicates a columnprint(df)

Output:

    Name  Age      Citya   John   28  New Yorkb   Anna   24     Parisc  Peter   35    Berlind  Linda   32    London

4. Renaming Columns

You can rename the columns using the rename() method:

df = df.rename(columns={'Name': 'Full Name', 'Age': 'Age Group'})print(df)

Output:

  Full Name  Age Group      Citya      John         28  New Yorkb      Anna         24     Parisc     Peter         35    Berlind     Linda         32    London

Data Cleaning

1. Handling Missing Data

Pandas provides functions to handle missing data:

• isnull() to check for null values

• dropna() to remove rows or columns with null values

• fillna() to replace null values with a specified value

df = pd.DataFrame({'Name': ['John', 'Anna', None, 'Linda'],                   'Age': [28, None, 35, 32]})print(df.isnull())  # Check for null values# Remove rows with missing datadf_cleaned = df.dropna()print(df_cleaned)# Replace missing data with a specific valuedf_filled = df.fillna({'Name': 'Unknown', 'Age': 0})print(df_filled)

Data Selection and Filtering

1. Conditional Selection

You can filter data based on conditions:

# Select rows where Age is greater than 30filtered_data = df[df['Age'] > 30]print(filtered_data)

2. Multiple Conditions

You can apply multiple conditions using & (AND) or | (OR):

# Select rows where Age > 25 and Name is not nullfiltered_data = df[(df['Age'] > 25) & (df['Name'].notnull())]print(filtered_data)

Sorting Data

1. Sorting by Column

You can sort data by columns using sort_values():

df_sorted = df.sort_values(by='Age', ascending=False)print(df_sorted)

2. Sorting by Index

You can sort data by index using sort_index():

df_sorted_by_index = df.sort_index()print(df_sorted_by_index)

Grouping Data

You can group data based on certain columns using groupby() and perform aggregate functions like sum, mean, etc.

# Group by 'City' and calculate the average 'Age' for each groupgrouped_data = df.groupby('City')['Age'].mean()print(grouped_data)

Merging and Joining DataFrames

You can combine DataFrames using the merge() function, similar to SQL joins.

df1 = pd.DataFrame({'ID': [1, 2, 3], 'Name': ['John', 'Anna', 'Peter']})df2 = pd.DataFrame({'ID': [1, 2, 4], 'Age': [28, 24, 35]})# Merging dataframes on 'ID'merged_df = pd.merge(df1, df2, on='ID', how='inner')print(merged_df)

Output:

   ID   Name  Age0   1   John   281   2   Anna   24

Exporting Data

You can save a DataFrame to various file formats:

1. Save to CSV

df.to_csv('data.csv', index=False)

2. Save to Excel

df.to_excel('data.xlsx', index=False)

Conclusion

Pandas is an essential library for data analysis in Python, providing fast, flexible, and expressive data structures like Series and DataFrame. With functions for reading data, cleaning, filtering, grouping, and visualizing, Pandas helps you work with structured data efficiently. Whether you're analyzing small datasets or performing complex data transformations, Pandas is a powerful tool for data manipulation and analysis.