How to check if pandas DateTimeIndex dates belong to a list?

To check if Pandas DateTimeIndex dates belong to a list, we can use the isin() method. In this we basically consider a subset of dates and check whether those dates are present in the list or not.

Check whether the set of dates exist in date column or not

isin() is a method in Pandas that is used to check whether the element belongs to that particular series or list. This is a useful function as it helps to filter out the elements. If the element is present in that Series, it returns True else it returns False. This is useful for filtering out the dates from the given Series.

Here we will consider an example where we generate dataframe that comprises of dates using date_range and define a list of dates to check whether those exist in the dataframe or not using isin().

import pandas as pd

# Create a DataFrame with a DateTimeIndex for one week
date_range = pd.date_range(start='2024-12-20', periods=7, freq='D')
df = pd.DataFrame({"temperature": [12, 14, 15, 13, 11, 10, 9]}, index=date_range)

# Define a list of holidays
holidays = ['2024-12-22', '2024-12-25']

# Convert holidays to datetime format
holidays = pd.to_datetime(holidays)

# Check if dates in the index belong to the list of holidays
df['is_holiday'] = df.index.isin(holidays)

print(df)

Output:

From the output we can see that we are filtering those dates that are present and setting the values to True. For those dates that do not exist in the list, we set them to False.

Check if the dates belong to that particular list

Intersection is basically finding the common elements between two lists. In Dataframe we can use intersection to check whether the dates belong to that Series or not. Let us assume that we have a dataframe that comprises of dates and associated values. We specify a list of dates and using intersection we check if that date is member or not. Finally we use isin() to set the status as True or False.

import pandas as pd

# Create a DataFrame with a DateTimeIndex
date_range = pd.date_range(start='2024-12-01', periods=7, freq='D')
df = pd.DataFrame({"value": range(10, 80, 10)}, index=date_range)

# Define a list of dates to check
dates_to_check = pd.to_datetime(['2024-12-02', '2024-12-05', '2024-12-07'])

# Find the intersection of dates
common_dates = df.index.intersection(dates_to_check)

# Add a column to indicate whether each date is in the list
df['is_in_list'] = df.index.isin(common_dates)

print(df)

            value  is_in_list
2024-12-01     10       False
2024-12-02     20        True
2024-12-03     30       False
2024-12-04     40       False
2024-12-05     50        True
2024-12-06     60   ...

Using lambda and map

Lambda is an anonymous function that returns the values when certain condition is satisfied. Now using the map we assign boolean values. Let us consider a sample example in which we have dataframe and some other columns. We have a list of dates which we convert to date time series. Using Lambda function we check that the dates exist in that list or not and using map we assign True to those dates that exists.

import pandas as pd

# Create a DataFrame with a DateTimeIndex
date_range = pd.date_range(start='2024-11-20', periods=7, freq='D')
df = pd.DataFrame({"temperature": [13, 24, 15, 13, 11, 10, 9]}, index=date_range)

# Define a list of holidays and convert to datetime
li = pd.to_datetime(['2024-11-22', '2024-11-25'])

# Use a lambda function to check if each date belongs to the list
df['is_in_list'] = df.index.map(lambda date: date in li)

print(df)

Output:

Using List Comprehension to filter out the dates

List Comprehension is an effective technique that is used to filter out the values based on any particular condition. Here we generate dates for the dataframe. These dates also act as indices for the dataframe which is used for comparison and it returns True or False accordingly.

import pandas as pd

# Create a DataFrame with a DateTimeIndex for a week
date_range = pd.date_range(start='2024-11-15', periods=7, freq='D')
df = pd.DataFrame({"sales": [100, 150, 200, 130, 170, 120, 180]}, index=date_range)

# Define a list of dates to check
dates_to_check = pd.to_datetime(['2024-11-16', '2024-11-19', '2024-11-20'])

# Use list comprehension to create a boolean array
df['is_in_list'] = [date in dates_to_check for date in df.index]

# Print the resulting DataFrame
print(df)

            sales  is_in_list
2024-11-15    100       False
2024-11-16    150        True
2024-11-17    200       False
2024-11-18    130       False
2024-11-19    170        True
2024-11-20    120   ...

Comparison of Techniques for Filtering DateTimeIndex dates