Python – Remove None Nested Records

Last Updated : 27 Apr, 2023

Sometimes, while working with Python Records, can have problem in which we need to perform the removal of data which have all key’s values as None in nested records. This kind of problem can have application in data preprocessing. Lets discuss certain ways in which this task can be performed.

Method #1 : Using all() + dictionary comprehension The combination of above functionalities can be used to solve this problem. In this, the removal of all the values with all None values are not included in dictionary re construction are done using all().

Python3

# Python3 code to demonstrate working of 
# Remove None Nested Records
# Using all() + dictionary comprehension
 
# initializing dictionary
test_dict = {'gfg' : {'a' : 1, 'b' : 2}, 
              'is' : {'d' : None, 'e' : None}, 
              'best' : {'g' : 1}}
 
# printing original dictionary
print("The original dictionary is : " + str(test_dict))
 
# Remove None Nested Records
# Using all() + dictionary comprehension
res = {key: sub1 for key, sub1 in test_dict.items() if not
      all(sub2 is None for sub2 in sub1.values())}
 
# printing result 
print("The dictionary after removal : " + str(res)) 

Output :

The original dictionary is : {‘gfg’: {‘b’: 2, ‘a’: 1}, ‘is’: {‘e’: None, ‘d’: None}, ‘best’: {‘g’: 1}} The dictionary after removal : {‘gfg’: {‘b’: 2, ‘a’: 1}, ‘best’: {‘g’: 1}}

Time Complexity: O(n*n), where n is the length of the list test_dict
Auxiliary Space: O(n) additional space of size n is created where n is the number of elements in the res list

Method #2 : Using any() + dictionary comprehension The combination of above functions can also be used to solve this problem. In this, we keep all the records which have any key as not none using any().

Python3

# Python3 code to demonstrate working of 
# Remove None Nested Records
# Using any() + dictionary comprehension
 
# initializing dictionary
test_dict = {'gfg' : {'a' : 1, 'b' : 2}, 
              'is' : {'d' : None, 'e' : None}, 
              'best' : {'g' : 1}}
 
# printing original dictionary
print("The original dictionary is : " + str(test_dict))
 
# Remove None Nested Records
# Using any() + dictionary comprehension
res = {key: sub1 for key, sub1 in test_dict.items() if
       any(sub2 is not None for sub2 in sub1.values())}
 
# printing result 
print("The dictionary after removal : " + str(res)) 

Output :

The original dictionary is : {‘gfg’: {‘b’: 2, ‘a’: 1}, ‘is’: {‘e’: None, ‘d’: None}, ‘best’: {‘g’: 1}} The dictionary after removal : {‘gfg’: {‘b’: 2, ‘a’: 1}, ‘best’: {‘g’: 1}}

Time complexity: O(n*m), where n is the number of keys in the dictionary and m is the maximum number of key-value pairs in any sub-dictionary.
Auxiliary space: O(n*m), a new temporary dictionary for each sub-dictionary and add only the non-None key-value pairs to it

Method 3: Using a for loop and conditional statements:

This method loops over the keys and values of the dictionary using two for loops. For each sub-dictionary, it creates a new temporary dictionary temp_dict and adds only those key-value pairs that have non-None values. If the temporary dictionary is not empty, it is added to the result dictionary res.

Python3

# initializing dictionary
test_dict = {'gfg' : {'a' : 1, 'b' : 2}, 
              'is' : {'d' : None, 'e' : None}, 
              'best' : {'g' : 1}}
 
# printing original dictionary
print("The original dictionary is : " + str(test_dict))
 
# Remove None Nested Records
# Using for loop and conditional statements
res = {}
for key, sub_dict in test_dict.items():
    temp_dict = {}
    for sub_key, sub_value in sub_dict.items():
        if sub_value is not None:
            temp_dict[sub_key] = sub_value
    if temp_dict:
        res[key] = temp_dict
 
# printing result 
print("The dictionary after removal : " + str(res))

Output

The original dictionary is : {'gfg': {'a': 1, 'b': 2}, 'is': {'d': None, 'e': None}, 'best': {'g': 1}}
The dictionary after removal : {'gfg': {'a': 1, 'b': 2}, 'best': {'g': 1}}

Time complexity: O(nm), where n is the number of keys in the dictionary and m is the maximum number of key-value pairs in any sub-dictionary.
Auxiliary space: O(nm), because we create a new temporary dictionary for each sub-dictionary and add only the non-None key-value pairs to it