Pandas Interview Questions

Last Updated : 27 Jul, 2026

Pandas is an open-source Python library used for data manipulation and analysis. In interviews, questions on Pandas are often asked to assess your ability to work with structured data effectively. Below are some of the most frequently asked interview questions and answers covering key Pandas topics.

1. What is Pandas, and why is it widely used in Python for data analysis?

  • Pandas is an open-source Python library, built on top of NumPy, that provides fast, flexible and expressive data structures for working with structured (tabular, multi-dimensional, time-series) data.
  • Pandas is widely used for data cleaning, transformation, exploration and analysis, and it integrates well with other libraries such as NumPy, Matplotlib and Scikit-learn.

2. List Key Features of Pandas.

Pandas are used for efficient data analysis. The key features of Pandas are as follows:

  • Fast and efficient data manipulation and analysis
  • Provides time-series functionality
  • We can easily handle missing data
  • Faster data merging and joining
  • Flexible reshaping of data sets
  • Group by functionality
  • Data from different file objects can be loaded
  • Integrates with NumPy

3. Difference between Pandas and NumPy

Pandas and NumPy are two fundamental Python libraries used for data analysis. NumPy is designed for fast numerical computations using homogeneous arrays, whereas Pandas is built on top of NumPy and provides labeled data structures for data manipulation and analysis.

NumPy

  • Designed for numerical and scientific computing.
  • Uses ndarray for storing homogeneous data.
  • Faster for mathematical and vectorized operations.
  • Provides linear algebra, statistical, and mathematical functions.
  • Goal: Efficient numerical computation.

Pandas

  • Built on top of NumPy.
  • Uses Series (1D) and DataFrame (2D) for labeled data.
  • Handles heterogeneous data and missing values efficiently.
  • Provides functions for filtering, grouping, merging, reshaping, and time-series analysis.
  • Goal: Data cleaning, manipulation, and analysis.

4. What are the Different Types of Data Structures in Pandas?

The two data structures that are supported by Pandas are Series and DataFrames.

  • Series is a one-dimensional labelled array that can hold data of any type. It is mostly used to represent a single column or row of data.
  • DataFrame is a two-dimensional heterogeneous data structure. It stores data in a tabular form. Its three main components are data, rows and columns.

5. What is Series in Pandas?

  • A Series in Pandas is a one-dimensional labelled array. Its columns are like an Excel sheet that can hold any type of data like integer, string, Python objects, etc.
  • Its axis labels are known as the index.
  • Series contains homogeneous data and its values can be changed but the size of the series is immutable.
  • A series can be created from a Python tuple, list and dictionary. The syntax for creating a series is as follows:
Python 
import pandas as pd
series = pd.Series(data)

6. What are the Different Ways to Create a Series?

In Pandas, a series can be created in many ways. They are as follows:

1. Creating a Series from a List

We can create a series using a Python list and pass it to the Series() constructor.

Python 
import pandas as pd
list1 = ['g', 'e', 'e', 'k', 's']

print(pd.Series(list1))

Output:

0 g
1 e
2 e
3 k
4 s
dtype: object

2. Creating a Series from Dictionary

A Series can also be created from a Python dictionary. The keys of the dictionary as used to construct indexes of the series.

Python 
import pandas as pd

dict = {'Geeks': 10,'for': 20, 'geeks': 30}

print(pd.Series(dict))

Output:

Geeks 10
for 20
geeks 30
dtype: int64

3. Creating a Series from Scalar Value

To create a series from a Scalar value, we must provide an index. The Series constructor will take two arguments, one will be the scalar value and the other will be a list of indexes. The value will repeat until all the index values are filled.

Python 
import pandas as pd
import numpy as np

ser = pd.Series(10, index=[0, 1, 2, 3, 4, 5])

print(ser)

Output:

0 10
1 10
2 10
3 10
4 10
5 10
dtype: int64

4. Creating a Series using NumPy Functions

The Numpy module's functions, such as numpy.linspace() and numpy.random.randn() can also be used to create a Pandas series.

Python 
import pandas as pd
import numpy as np

ser1 = pd.Series(np.linspace(3, 33, 3))
print(ser1)

ser2 = pd.Series(np.random.randn(3))
print("\n", ser2)

Output:

0 3.0
1 18.0
2 33.0
dtype: float64


0 -0.341027
1 -1.700664
2 0.364409
dtype: float64

5. Creating a Series using List Comprehension

Here, we will use the Python list comprehension technique to create a series in Pandas. We will use the range function to define the values and a for loop for indexes.

Python 
import pandas as pd
ser = pd.Series(range(1, 20, 3),
index=[x for x in 'abcdefg'])
print(ser)

Output:

a 1
b 4
c 7
d 10
e 13
f 16
g 19
dtype: int64

7. What is a DataFrame in Pandas?

  • A DataFrame in Panda is a data structure used to store the data in tabular form, that is in the form of rows and columns.
  • It is two-dimensional, size-mutable and heterogeneous in nature.
  • The main components of a dataframe are data, rows and columns.
  • A dataframe can be created by loading the dataset from existing storage such as SL database, CSV file, Excel file, etc. The syntax for creating a dataframe is as follows:
Python 
import pandas as pd
dataframe = pd.DataFrame(data)

8. What are the Different ways to Create a DataFrame in Pandas?

In Pandas, a dataframe can be created in many ways. They are as follows:

1. Creating a DataFrame using a List

In order to create a DataFrame from a Python list, just pass the list to the DataFrame() constructor.

Python 
import pandas as pd

lst = ['Geeks', 'For', 'Geeks', 'is',
'portal', 'for', 'Geeks']

print(pd.DataFrame(lst))

Output:

0
0 Geeks
1 For
2 Geeks
3 is
4 portal
5 for
6 Geeks

2. Creating a DataFrame using a Dictionary

A DataFrame can be created from a Python dictionary and passed to the DataFrame() constructor. The Keys of the dictionary will be the column names and the values of the dictionary are the data of the DataFrame.

Python 
import pandas as pd

data = {'Name':['Tom', 'nick', 'krish', 'jack'], 'Age':[20, 21, 19, 18]}

print(pd.DataFrame(data))

Output:

Name Age
0 Tom 20
1 nick 21
2 krish 19
3 jack 18

3. Creating a DataFrame using a List of Dictionaries

Another way to create a DataFrame is by using Python list of dictionaries. The list is passed to the DataFrame() constructor. The Keys of each dictionary element will be the column names.

Python 
import pandas as pd

lst = [{1: 'Geeks', 2: 'For', 3: 'Geeks'},
{1: 'Portal', 2: 'for', 3: 'Geeks'}]

print(pd.DataFrame(lst))

Output:

1 2 3
0 Geeks For Geeks
1 Portal for Geeks

4. Creating a DataFrame from Pandas Series

A DataFrame in Pandas can also be created by using the Pandas series.

Python 
import pandas as pd

lst = pd.Series(['Geeks', 'For', 'Geeks'])

print(pd.DataFrame(lst))

Output:

0
0 Geeks
1 For
2 Geeks

9. Difference between Series and DataFrame

Series and DataFrame are the two primary data structures in Pandas. A Series is a one-dimensional labeled array that stores a single column of data, whereas a DataFrame is a two-dimensional labeled table that stores multiple rows and columns, with each column potentially having a different data type.

Series

  • One-dimensional labeled array.
  • Represents a single column of data.
  • Has an index and values.
  • Can store any data type.
  • Goal: Store a single sequence of labeled data.

DataFrame

  • Two-dimensional labeled table.
  • Represents multiple rows and columns.
  • Each column is a Series.
  • Can store different data types across columns.
  • Goal: Store and analyze tabular data.

10. How to Read Data into a DataFrame from a CSV file?

We can create a data frame from a CSV (Comma Separated Values) file. This can be done by using the read_csv() method which takes the csv file as the parameter.

Python 
pandas.read_csv(file_name)

Another way to do this is by using the read_table() method which takes the CSV file and a delimiter value as the parameter.

Python 
pandas.read_table(file_name, delimiter)

11. How to Read Large CSV Files Efficiently in Pandas?

When a file is too large to fit comfortably in memory, read_csv() supports a chunksize parameter that returns an iterator, so the file can be processed in smaller pieces instead of loading it all at once:

Python 
import pandas as pd

for chunk in pd.read_csv('large_file.csv', chunksize=100000):
    process(chunk)   # process each chunk (e.g. filter, aggregate)

12. How can a DataFrame be Converted to an Excel File?

A Pandas dataframe can be converted to an Excel file by using the to_excel() function which takes the file name as the parameter. We can also specify the sheet name in this function.

Python 
DataFrame.to_excel(file_name)

13. How to Export a DataFrame to Other File Formats (JSON, SQL, Parquet)?

Besides CSV and Excel, Pandas can export a DataFrame to several other common formats:

Python 
DataFrame.to_json('data.json')          # Export to JSON
DataFrame.to_sql('table_name', conn)    # Export to a SQL database table
DataFrame.to_parquet('data.parquet')    # Export to Parquet (columnar, compressed)
DataFrame.to_html('data.html')          # Export to an HTML table

to_parquet() is preferred for large datasets since Parquet is compressed and columnar, making it faster to read/write and more memory-efficient than CSV.

14. How to Convert a DataFrame into a Numpy Array?

Pandas Numpy is an inbuilt Python package that is used to perform large numerical computations. It is used for processing multidimensional array elements to perform complicated mathematical operations.

Pandas dataframe can be converted to a NumPy array by using the to_numpy() method. We can also provide the datatype as an optional argument.

Python 
Dataframe.to_numpy()

We can also use .values to convert dataframe values to NumPy array

Python 
df.values

15. How to access the first few rows of a dataframe?

The first few records of a dataframe can be accessed by using the pandas head() method. It takes one optional argument n, which is the number of rows. By default, it returns the first 5 rows of the dataframe. The head() method has the following syntax:

Python 
df.head(n)

Another way to do it is by using iloc() method. It is similar to the Python list-slicing technique. It has the following syntax:

Python 
df.iloc[:n]

16. How to Select a Single Column of a DataFrame?

There are many ways to Select a single column of a dataframe. They are as follows:

By using the Dot operator, we can access any column of a dataframe.

Python 
Dataframe.column_name

Another way to select a column is by using the square brackets [].

Python 
DataFrame[column_name]

17. How to Filter Rows Based on a Condition (Boolean Indexing)?

Boolean indexing lets you select rows that satisfy one or more conditions by passing a boolean Series/array inside []:

Python 
import pandas as pd

df = pd.DataFrame({'Name': ['Tom', 'Nick', 'Krish'], 'Age': [20, 21, 19]})

# Single condition
print(df[df['Age'] > 19])

# Multiple conditions (use & , | , ~ and wrap each condition in parentheses)
print(df[(df['Age'] > 19) & (df['Name'] != 'Tom')])

18. What is the query() Method in Pandas?

query() provides a more readable, SQL-like way to filter rows using a string expression instead of boolean indexing syntax:

Python 
df.query('Age > 19 and Name != "Tom"')
  • External variables can be referenced using the @ symbol, e.g. df.query('Age > @min_age').
  • It's often faster and more readable than chained boolean indexing on large DataFrames, though boolean indexing remains more flexible for complex logic.

19. Difference between loc, iloc, at and iat in Pandas.

  • loc: Label-based indexing — access rows/columns using their labels (names).
Python 
df.loc[row_labels, column_labels]
  • iloc: Integer-position-based indexing — access rows/columns using numeric positions.
Python 
df.iloc[row_positions, column_positions]
  • at: Label-based access to a single scalar value — faster than loc for scalar lookups.
Python 
df.at[row_label, 'column_name']
  • iat: Integer-position-based access to a single scalar value — faster than iloc for scalar lookups.
Python 
df.iat[row_position, column_position]

20. What is the Difference Between axis=0 and axis=1 in Pandas?

Many Pandas functions (drop(), mean(), apply(), concat(), etc.) accept an axis parameter to specify the direction of the operation:

  • axis=0 (default): Operates down the rows — i.e., column-wise / across the index. Example: df.mean(axis=0) computes the mean of each column.
  • axis=1: Operates across the columns — i.e., row-wise. Example: df.mean(axis=1) computes the mean of each row.

21. How to Rename a Column in a DataFrame?

A column of the dataframe can be renamed by using the rename() function. We can rename a single as well as multiple columns at the same time using this method.

Python 
DataFrame.rename(columns={'column1': 'COLUMN_1', 'column2':'COLUMN_2'}, inplace=True)

Another way is by using the set_axis() function which takes the new column name and axis to be replaced with the new name.

Python 
DataFrame.set_axis(labels=['COLUMN_1','COLUMN_2'], axis=1, inplace=True)

In case we want to add a prefix or suffix to the column names, we can use the add_prefix() or add_suffix() methods.

Python 
DataFrame.add_prefix(prefix='PREFIX_')
DataFrame.add_suffix(suffix='_suffix')

22. How to add Row or Column to an Existing Dataframe?

1. Adding Rows

The df.loc[] is used to access a group of rows or columns and can be used to add a row to a dataframe.

Python 
DataFrame.loc[Row_Index]=new_row

We can also add multiple rows in a dataframe by using pandas.concat() function which takes a list of dataframes to be added together.

Python 
pandas.concat([Dataframe1,Dataframe2])

2. Adding Columns

We can add a column to an existing dataframe by just declaring the column name and the list or dictionary of values.

Python 
DataFrame[data] = list_of_values

Another way to add a column is by using df.insert() method which take a value where the column should be added, column name and the value of the column as parameters.

Python 
DataFrameName.insert(col_index, col_name, value)

We can also add a column to a dataframe by using df.assign() function

Python 
DataFrame.assign(**kwargs)

23. How to Delete an Row or Column from an Existing DataFrame?

We can delete a row or a column from a dataframe by using df.drop() method. and provide the row or column name as the parameter.

1. To delete a column

Python 
DataFrame.drop(['Column_Name'], axis=1)

2. To delete a row

Python 
DataFrame.drop([Row_Index_Number], axis=0)

24. How to Iterate Over Rows in a DataFrame?

Pandas provides a few ways to iterate over rows, though iteration should generally be avoided in favor of vectorized operations (see next question):

  • iterrows(): Returns (index, Series) pairs for each row. Simple to use, but slow — every row is converted into a Series, which loses dtype consistency and adds overhead.
Python 
for index, row in df.iterrows():
      print(index, row['col_name'])
  • itertuples(): Returns each row as a namedtuple. Considerably faster than iterrows() since it avoids Series creation.
Python 
for row in df.itertuples():
    print(row.col_name)
  • items(): Iterates over the DataFrame column by column, returning (column_name, Series) pairs.

25. What is Vectorization in Pandas and Why is it Preferred Over Loops?

  • Vectorization refers to performing operations on entire arrays/Series at once instead of looping through elements one at a time in Python.
  • Vectorized operations are significantly faster because they avoid Python-level loop overhead and use optimized, compiled code under the hood.
  • As a general performance rule when working with Pandas: prefer vectorized operations > apply() > itertuples() > iterrows(), in that order, when performance matters.

26. How to Merge Two DataFrames?

In pandas, we can combine two dataframes using the pandas.merge() method which takes 2 dataframes as the parameters.

Python 
import pandas as pd

df1 = pd.DataFrame({'A': [1, 2, 3],
'B': [4, 5, 6]},
index=[10, 20, 30])

df2 = pd.DataFrame({'C': [7, 8, 9],
'D': [10, 11, 12]},
index=[20, 30, 40])

result = pd.merge(df1, df2, left_index=True, right_index=True)
print(result)

Output:

A B C D
20 2 5 7 10
30 3 6 8 11

27. Difference between join(), merge() and concat()

join(), merge(), and concat() are Pandas functions used to combine DataFrames.

merge()

  • Combines DataFrames using one or more common columns (keys).
  • Similar to SQL JOIN.
  • Supports inner, left, right, and outer joins.
  • Best for combining related datasets.
  • Goal: Merge DataFrames based on matching keys.

join()

  • Combines DataFrames using the index by default.
  • Can also join using a specified column.
  • Simpler syntax than merge().
  • Best when DataFrames share the same index.
  • Goal: Join DataFrames based on index.

concat()

  • Combines DataFrames by stacking them.
  • Supports both row-wise (axis=0) and column-wise (axis=1) concatenation.
  • Does not require common keys.
  • Best for appending DataFrames with similar structures.
  • Goal: Combine multiple DataFrames into one.

28. How to Sort a Dataframe?

A dataframe in pandas can be sorted in ascending or descending order according to a particular column. We can do so by using the sort_values() method and providing the column name according to which we want to sort the dataframe. We can also sort it by multiple columns.

To sort it in descending order, we pass an additional parameter 'ascending' and set it to False.

Python 
DataFrame.sort_values(by='Age',ascending=True)

29. How to Compute Mean, Median, Mode, Variance, Standard Deviation and Various Quantile Ranges in Pandas?

The mean, median, mode, Variance, Standard Deviation and Quantile range can be computed using the following commands in Python.

30. Difference between Shallow Copy and Deep Copy?

In Pandas, there are two ways to create a copy of the Series. They are as follows:

1. Shallow Copy is a copy of the series object where the indices and the data of the original object are not copied. It only copies the references to the indices and data. This means any changes made to a series will be reflected in the other. A shallow copy of the series can be created by writing the following syntax:

Python 
ser.copy(deep=False)

2. Deep Copy is a copy of the series object where it has its own indices and data. This means changes made to a copy of the object will not be reflected to the original series object. A deep copy of the series can be created by writing the following syntax:

Python 
ser.copy(deep=True)

The default value of the deep parameter of the copy() function is set to True.

31. What is SettingWithCopyWarning in Pandas?

SettingWithCopyWarning is raised when Pandas cannot determine whether an operation is modifying a view of the original DataFrame or a copy of it — meaning your intended change might silently not apply to the original data.

Python 
sub_df = df[df['Age'] > 20]
sub_df['Age'] = 0   # may raise SettingWithCopyWarning

This typically happens with chained indexing (df[condition]['col'] = value). To avoid it:

  • Use .loc for the assignment in one step: df.loc[df['Age'] > 20, 'Age'] = 0
  • Or explicitly create a copy first: sub_df = df[df['Age'] > 20].copy()

32. How to Check and Remove Duplicate Values in Pandas.

In pandas, duplicate values can be checked by using the duplicated() method.

Python 
DataFrame.duplicated()

To remove the duplicated values we can use the drop_duplicates() method.

Python 
DataFrame.drop_duplicates()

33. How to Handle Missing Data in Pandas?

Datasets often have missing values due to data collection issues, entry errors or unavailable observations. Pandas provides several functions to handle this:

  • isnull(): It returns True for NaN values or null values and False for present values
  • notnull(): It returns False for NaN values and True for present values
  • dropna(): It analyzes and drops Rows/Columns with Null values
  • fillna(): It let the user replace NaN values with some value of their own
  • replace(): It is used to replace a string, regex, list, dictionary, series, number, etc.
  • interpolate(): It fills NA values in the dataframe or series.

34. Difference between NaN, None and NaT in Pandas

NaN, None, and NaT are used to represent missing values in Pandas, but they apply to different types of data. NaN represents missing numeric values, None is Python's generic null object used mainly for object data, and NaT represents missing date/time values.

NaN

  • Stands for Not a Number.
  • Represents missing values in numeric data.
  • Based on NumPy (numpy.nan).
  • Automatically used by Pandas for missing numerical values.
  • Goal: Represent missing numeric data.

None

  • Python's built-in null object.
  • Commonly used for object or mixed-type data.
  • Pandas often converts None to NaN in numeric columns.
  • Goal: Represent a generic missing value.

NaT

  • Stands for Not a Time.
  • Represents missing values in date and time columns.
  • Equivalent of NaN for datetime data.
  • Goal: Represent missing datetime values.

35. Difference between the interpolate() and fillna()

The interpolate() and fillna() methods in pandas are used to handle missing or NaN (Not a Number) values in a DataFrame or Series.

fillna()

  • Replaces missing values with a specified value or method.
  • Can fill with constants, mean, median, mode, or use forward fill (ffill) and backward fill (bfill).
  • Does not estimate missing values.
  • Best for categorical data or when a replacement value is known.
  • Goal: Replace missing values with predefined values.

interpolate()

  • Estimates missing values using existing data points.
  • Supports methods such as linear, polynomial, and time-based interpolation.
  • Commonly used for numerical and time-series data.
  • Produces smoother and more realistic estimates.
  • Goal: Estimate missing values based on nearby values.

36. Difference between map(), applymap() and apply()

The map(), applymap() and apply() methods are used in pandas for applying functions or transformations to elements in a DataFrame or Series.

map()

  • Works only on a Series.
  • Applies a function, dictionary, or mapping to each element.
  • Commonly used for value replacement or transformation.
  • Goal: Transform values in a single column.

applymap()

  • Works only on a DataFrame.
  • Applies a function to every element in the DataFrame.
  • Used for element-wise transformations.
  • Goal: Transform every value in a DataFrame.

apply()

  • Works on both Series and DataFrame.
  • Applies a function along rows (axis=1) or columns (axis=0).
  • Can perform aggregation, transformation, or custom operations.
  • Goal: Apply functions row-wise, column-wise, or on a Series.

37. How to Set and Reset the Index in a Panda dataFrame?

1. Set Index: We can set the index to a Pandas dataframe by using the set_index() method, which is used to set a list, series or dataframe as the index of a dataframe.

Python 
DataFrame.set_index('Column_Name')

2. Reset Index: The index of Pandas dataframes can be reset by using the reset_index() method. It can be used to simply reset the index to the default integer index beginning at 0.

Python 
DataFrame.reset_index(inplace = True)

38. What is Reindexing in Pandas?

  • Reindexing in Pandas as the name suggests means changing the index of the rows and columns of a dataframe.
  • It can be done by using the Pandas reindex() method.
  • In case of missing values or new values that are not present in the dataframe, the reindex() method assigns it as NaN.
Python 
df.reindex(new_index)

39. What is Multi-Indexing in Pandas?

Multi-indexing refers to selecting two or more rows or columns in the index. It is a multi-level or hierarchical object for pandas object and deals with data analysis and works with higher dimensional data. Multi-indexing in Pandas can be achieved by using a number of functions such as:

  • MultiIndex.from_arrays
  • MultiIndex.from_tuples
  • MultiIndex.from_product
  • MultiIndex.from_frame

40. What are stack() and unstack() in Pandas?

stack() and unstack() reshape data between "wide" and "long" formats using hierarchical indexing:

  • stack(): Pivots the innermost column level into the row index, converting wide-format data into long-format data (increases index levels).
  • unstack(): Pivots the innermost row index level into columns, converting long-format data into wide-format data (decreases index levels).

41. What is the Significance of Pandas Described Command?

Pandas describe() is used to view some basic statistical details of a data frame or a series of numeric values. It can give a different output when it is applied to a series of strings. It can get details like percentile, mean, standard deviation, etc.

Python 
DataFrame.describe()

42. How to Find the Correlation Using Pandas?

Pandas dataframe.corr() method is used to find the correlation of all the columns of a dataframe. It automatically ignores any missing or non-numerical values.

Python 
DataFrame.corr()

43. What is groupby() in Pandas and how is it used?

The groupby() function in Pandas is used to split the data into groups based on one or more columns, then apply an operation (like aggregation, transformation or filtering) on each group separately.

Python 
df.groupby(by_column)

For example:

Python 
import pandas as pd

data = {'Dept': ['IT', 'IT', 'HR', 'HR'],
        'Salary': [50000, 60000, 45000, 55000]}
df = pd.DataFrame(data)

result = df.groupby('Dept')['Salary'].mean()
print(result)

Output:

Dept
HR 50000.0
IT 55000.0
Name: Salary, dtype: float64

44. How can we use Pivot table in Pandas?

  • In Pandas, pivot_table() is used to summarize and reshape data into a tabular format.
  • It allows you to aggregate values like sum, mean, count, etc by specifying which columns become rows (index), which become columns and which contain the values to aggregate.
  • We can pivot the dataframe in Pandas by using the pivot_table() method.
  • To unpivot the dataframe to its original form we can melt the dataframe by using the melt() method.

45. What is the difference between pivot_table() and groupby()

Both pivot_table() and groupby() are useful methods in pandas used for aggregating and summarizing data.

groupby()

  • Groups data based on one or more columns.
  • Performs aggregation using functions like sum(), mean(), count(), etc.
  • Returns grouped data that can be further processed.
  • More flexible for complex data transformations.
  • Goal: Group and aggregate data.

pivot_table()

  • Creates a summary table similar to an Excel Pivot Table.
  • Organizes data into rows, columns, and values.
  • Supports multiple aggregation functions.
  • Can automatically handle missing values using fill_value.
  • Goal: Create cross-tabulated summary reports.

46. What is crosstab() ?

pd.crosstab() computes a simple cross-tabulation (frequency table) of two or more factors, showing the count (by default) of observations for each combination of categories:

Python 
pd.crosstab(df['Gender'], df['Department'])

47. What is Data Aggregation in Pandas?

  • In Pandas, data aggregation refers to the act of summarizing or decreasing data in order to produce a statistical summary of one or more columns in a dataset.
  • In order to calculate statistical measures like sum, mean, minimum, maximum, count, etc aggregation functions must be applied to groups or subsets of data.
  • The agg() function in Pandas is frequently used to aggregate data.
  • Applying one or more aggregation functions to one or more columns in a DataFrame or Series is possible using this approach.
Python 
DataFrame.agg({'Col_name1': ['sum', 'min', 'max'], 'Col_name2': 'count'})

48. What are Categorical Data Types in Pandas and Why Use Them?

The category dtype is used for columns that contain a limited, fixed set of repeated values (e.g., "Yes"/"No", city names, product categories). Converting such columns to category dtype improves performance and reduces memory usage significantly, since Pandas stores the unique values once and uses integer codes internally.

Benefits:

  • Much lower memory usage for columns with many repeated string values
  • Faster groupby(), sorting and comparison operations
  • Supports defining an explicit category order (useful for ordinal data like "Low" < "Medium" < "High"

49. What is Time Series in Pandas?

  • Time series is a collection of data points with timestamps. It depicts the evolution of quantity over time.
  • Pandas provide various functions to handle time series data efficiently.
  • It is used to work with data timestamps, resampling time series for different time periods, working with missing data, slicing the data using timestamps, etc.

We have various time-series function in pandas like:

Pandas Built-in Function

Operation

pandas.to_datetime(DataFrame['Date'])

Convert 'Date' column of DataFrame to datetime dtype

DataFrame.set_index('Date', inplace=True)

Set 'Date' as the index

DataFrame.resample('H').sum()

Resample time series to a different frequency (e.g., Hourly, daily, weekly, monthly etc)

DataFrame.interpolate()

Fill missing values using linear interpolation

DataFrame.loc[start_date:end_date]

Slice the data based on timestamps

50. How to convert a String to Datetime in Pandas?

A Python string can be converted to a DateTime object by using:

1. Pandas.to_datetime()

Python 
import pandas as pd


date_string = '2023-07-17'
dateTime = pd.to_datetime(date_string)
print(dateTime)

Output:

2023-07-17 00:00:00

2. datetime.strptime

Python 
from datetime import datetime

date_string = '2023-07-17'
dateTime = datetime.strptime(date_string, '%Y-%m-%d')
print(dateTime)

Output:

2023-07-17 00:00:00

51. What is Time Delta in Pandas?

  • The time delta is the difference in dates and time. It indicates the duration or difference in time.
  • The time delta object can be created by using the timedelta() method and providing the number of weeks, days, seconds, milliseconds, etc as the parameter.
Python 
Duration = pandas.Timedelta(days=7, hours=4, minutes=30, seconds=23)

52. What are rolling() and expanding() Window Functions in Pandas?

These functions are used to perform windowed computations — very commonly asked for time-series/analytics roles.

  • rolling(window=n): Computes a statistic (mean, sum, std, etc.) over a fixed-size sliding window of the last n rows. Commonly used for moving averages.
Python 
df['moving_avg'] = df['price'].rolling(window=3).mean()
  • expanding(): Computes a statistic over all data up to the current row (the window grows as you move forward) — useful for cumulative metrics.
Python 
df['cumulative_avg'] = df['price'].expanding().mean()

53. How to make Label Encoding using Pandas?

Label encoding is used to convert categorical data into numerical data so that a machine-learning model can fit it. To apply label encoding using pandas we can use:

1. pandas.Categorical().codes: It only gives codes.

  • pd.Categorical() converts the data into a Categorical type.
  • .codes gives the integer code for each category.
Python 
import pandas as pd

data = pd.Series(['Red', 'Blue', 'Green', 'Blue'])
encoded = pd.Categorical(data).codes
print(encoded)

Output:

[2 0 1 0]

2. pandas.factorize(): It gives both codes and unique labels.

  • factorize() returns a tuple: (encoded_array, unique_categories).
  • The first array gives integer codes and the second gives the mapping of categories.
  • It automatically detects unique values and assigns codes in their first-seen order.
Python 
import pandas as pd

data = pd.Series(['Red', 'Blue', 'Green', 'Blue'])
encoded, uniques = pd.factorize(data)
print(encoded)

Output:

[0 1 2 1]

54. How to make Onehot Encoding using Pandas?

  • One-hot encoding is a technique for representing categorical data as numerical values in a machine-learning model.
  • It works by creating a separate binary variable for each category in the data.
  • The value of the binary variable is 1 if the observation belongs to that category and 0 otherwise. It can improve the performance of the model.

To apply one hot encoding, we greater a dummy column for our dataframe by using get_dummies() method.

Python 
pd.get_dummies(data, columns=None)

For example:

Python 
import pandas as pd

df = pd.DataFrame({'Color': ['Red', 'Blue', 'Green']})

encoded = pd.get_dummies(df, columns=['Color'])
print(encoded)

Output:

Color_Blue Color_Green Color_Red
0 0 0 1
1 1 0 0
2 0 1 0

55. What is Method Chaining in Pandas?

Method chaining refers to calling multiple DataFrame methods sequentially in a single statement instead of creating intermediate variables at each step:

Python 
result = (
    df.dropna()
      .query('Age > 18')
      .groupby('Department')['Salary']
      .mean()
      .sort_values(ascending=False)
)

56. What is Broadcasting in Pandas?

Broadcasting refers to how Pandas (via NumPy) performs operations between objects of different shapes without requiring them to match exactly:

  • Scalar operations: A scalar operates on every element of a Series/DataFrame (df * 2).
  • Series-DataFrame operations: When a Series is combined with a DataFrame, Pandas aligns the Series along the matching axis (typically columns) and applies the operation to each row.
  • Index alignment: When combining two Series/DataFrames, Pandas first aligns them by index/column labels before performing the operation — mismatched labels produce NaN.

57. How to Find the Nth Highest Value in a Column (e.g., Second Highest Salary)?

nlargest() is preferred over sort_values() here since it's more efficient and directly expresses intent. Always drop duplicates first if the question asks for the Nth distinct value.

Python 
import pandas as pd

df = pd.DataFrame({'id': [1, 2, 3, 4], 'salary': [100, 300, 300, 200]})

# Using drop_duplicates + nlargest
n = 2
distinct_salaries = df['salary'].drop_duplicates()
second_highest = distinct_salaries.nlargest(n).iloc[-1] if len(distinct_salaries) >= n else None
print(second_highest)

Output:

200

58. How to Find and Display Duplicate Rows in a DataFrame?

keep=False marks all occurrences of a duplicate as True (instead of hiding the first/last occurrence), which is usually what's wanted when you want to see the duplicates rather than just drop them.

Python 
import pandas as pd

df = pd.DataFrame({'Name': ['Tom', 'Nick', 'Tom'], 'Age': [20, 21, 20]})

duplicate_rows = df[df.duplicated(keep=False)]
print(duplicate_rows)

Output:

Name Age

0 Tom 20

2 Tom 20

59. How to Find Employees Earning More Than the Average Salary of Their Department?

A classic "window function"-style query, commonly asked to test groupby().transform().

Python 
import pandas as pd

df = pd.DataFrame({
    'Dept': ['IT', 'IT', 'HR', 'HR'],
    'Name': ['A', 'B', 'C', 'D'],
    'Salary': [60000, 50000, 45000, 55000]
})

df['Dept_Avg'] = df.groupby('Dept')['Salary'].transform('mean')
result = df[df['Salary'] > df['Dept_Avg']]
print(result[['Name', 'Dept', 'Salary']])

Output:

Name Dept Salary

0 A IT 60000

3 D HR 55000

transform() is key here — unlike agg(), it returns a result with the same shape/index as the original DataFrame, so it can be compared row-by-row against the original column.

60. How to Find the Top N Rows per Group?

An alternative is df.groupby('Dept').apply(lambda x: x.nlargest(2, 'Salary')), but the sort_values() + groupby().head() combination is generally faster since it avoids a Python-level apply().

Python 
import pandas as pd

df = pd.DataFrame({
    'Dept': ['IT', 'IT', 'IT', 'HR', 'HR'],
    'Name': ['A', 'B', 'C', 'D', 'E'],
    'Salary': [60000, 90000, 50000, 45000, 70000]
})

top_n = (
    df.sort_values('Salary', ascending=False)
      .groupby('Dept')
      .head(2)
)
print(top_n)

61. How to Find Missing Dates in a Time Series?

This pattern — generate the complete expected range with date_range(), then use .difference() against the actual dates — is a very common way to detect gaps in time-series data.

Python 
import pandas as pd

df = pd.DataFrame({'date': pd.to_datetime(['2024-01-01', '2024-01-03', '2024-01-05'])})

full_range = pd.date_range(start=df['date'].min(), end=df['date'].max(), freq='D')
missing_dates = full_range.difference(df['date'])
print(missing_dates)

Output:

DatetimeIndex(['2024-01-02', '2024-01-04'], dtype='datetime64[ns]', freq=None)

62. How to Calculate a Running/Cumulative Total in Pandas?

Related functions: cummax(), cummin(), cumprod() for running maximum, minimum and product respectively.

Python 
import pandas as pd

df = pd.DataFrame({'Day': [1, 2, 3, 4], 'Sales': [100, 150, 200, 50]})
df['Running_Total'] = df['Sales'].cumsum()
print(df)

Output:

Day Sales Running_Total

0 1 100 100

1 2 150 250

2 3 200 450

3 4 50 500

63. How to Rank Values Within Each Group?

The method parameter controls how ties are handled: 'average' (default), 'min', 'max', 'first', or 'dense' (no gaps between ranks after a tie).

Python 
import pandas as pd

df = pd.DataFrame({
    'Dept': ['IT', 'IT', 'HR', 'HR'],
    'Name': ['A', 'B', 'C', 'D'],
    'Salary': [60000, 90000, 45000, 55000]
})

df['Rank'] = df.groupby('Dept')['Salary'].rank(ascending=False, method='dense')
print(df)

64. How to Detect and Remove Outliers Using the IQR Method?

Python 
import pandas as pd

df = pd.DataFrame({'value': [10, 12, 12, 13, 12, 100, 11, 14]})

Q1 = df['value'].quantile(0.25)
Q3 = df['value'].quantile(0.75)
IQR = Q3 - Q1

lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR

clean_df = df[(df['value'] >= lower_bound) & (df['value'] <= upper_bound)]
print(clean_df)

65. How to Filter Rows Using String Pattern Matching?

The .str accessor is used to apply vectorized string operations to a column.

Python 
import pandas as pd

df = pd.DataFrame({'Email': ['a@gmail.com', 'b@yahoo.com', 'c@gmail.com']})

gmail_users = df[df['Email'].str.contains('gmail', case=False, na=False)]
print(gmail_users)

66. How to Find Records Present in One DataFrame but Not in Another (Anti-Join)?

Pandas has no direct anti_join(), so it's commonly built using merge() with indicator=True.

Python 
import pandas as pd

customers = pd.DataFrame({'id': [1, 2, 3]})
orders = pd.DataFrame({'id': [2, 3]})

merged = customers.merge(orders, on='id', how='left', indicator=True)
customers_without_orders = merged[merged['_merge'] == 'left_only'].drop(columns='_merge')
print(customers_without_orders)

Output:

id

0 1

67. How to Calculate Each Row's Percentage Contribution to Its Group's Total?

Python 
import pandas as pd

df = pd.DataFrame({
    'Dept': ['IT', 'IT', 'HR', 'HR'],
    'Salary': [60000, 40000, 30000, 70000]
})

df['Pct_of_Dept'] = df['Salary'] / df.groupby('Dept')['Salary'].transform('sum') * 100
print(df)
Comment

Explore