Python - Remove duplicate values across Dictionary Values
Last Updated :
27 Jan, 2025
Dictionaries often contain lists or sets as their values and sometimes we want to remove duplicate values across all dictionary keys. For example, consider the dictionary d = {'a': [1, 2, 3], 'b': [3, 4, 5], 'c': [5, 6]}. The number 3 appears under both 'a' and 'b', and 5 appears under both 'b' and 'c'. To clean up such overlaps, we can use various methods to ensure that each value is unique across all dictionary keys. Let's explore different methods to achieve this.
Using set
This approach uses a set to keep track of values that have already been encountered, ensuring they are added only once.
Python
d = {'a': [1, 2, 3], 'b': [3, 4, 5], 'c': [5, 6]}
s = set()
# Remove duplicates across dictionary values
for key in d:
d[key] = [v for v in d[key] if v not in s and not s.add(v)]
print(d)
Output{'a': [1, 2, 3], 'b': [4, 5], 'c': [6]}
Explanation:
- A set named 's' is initialized to track values that have been processed.
- For each key in the dictionary, the comprehension filters out values that are already in 's'.
- The add() method is used to update the seen set during iteration.
Let's explore some more ways and see how we can remove duplicate values across dictionary items.
Using Nested Loops
This method manually iterates over each value in the dictionary and removes duplicates by checking against a set.
Python
d = {'a': [1, 2, 3], 'b': [3, 4, 5], 'c': [5, 6]}
s = set()
# Remove duplicates across dictionary values
for key, values in d.items():
a = []
for v in values:
if v not in s:
a.append(v)
s.add(v)
d[key] = a
print(d)
Output{'a': [1, 2, 3], 'b': [4, 5], 'c': [6]}
Explanation:
- A new list 'a' is created for each dictionary key.
- The method checks if a value is already in the set 's'. If not, it is added to both 'a' and 's'.
- After processing, the dictionary is updated with the filtered list of values.
Using a Flat List and Dictionary Comprehension
This method flattens all dictionary values into a single list, processes it to remove duplicates, and then reconstructs the dictionary.
Python
d = {'a': [1, 2, 3], 'b': [3, 4, 5], 'c': [5, 6]}
# Flatten values and remove duplicates
all_val = sum(d.values(), [])
unique_val = list(dict.fromkeys(all_val))
# Reconstruct the dictionary
d = {k: [v for v in unique_val if v in d[k]] for k in d}
print(d)
Output{'a': [1, 2, 3], 'b': [3, 4, 5], 'c': [5, 6]}
Explanation:
- All dictionary values are flattened into a single list.
- Duplicate values are removed by using dict.fromkeys(), which preserves the order of first appearance.
- The dictionary is reconstructed with filtered values.
Similar Reads
Python - Remove duplicate values in dictionary Sometimes, while working with Python dictionaries, we can have problem in which we need to perform the removal of all the duplicate values of dictionary, and we are not concerned if any key get removed in the process. This kind of application can occur in school programming and day-day programming.
8 min read
Remove Duplicity from a Dictionary - Python We are given a dictionary and our task is to remove duplicate values from it. For example, if the dictionary is {'a': 1, 'b': 2, 'c': 2, 'd': 3, 'e': 1}, the unique values are {1, 2, 3}, so the output should be {'a': 1, 'b': 2, 'd': 3}.Using a loopThis method uses a loop to iterate through dictionar
3 min read
Python Remove Item from Dictionary by Value We are given a dictionary and our task is to remove key-value pairs where the value matches a specified target. This can be done using various approaches, such as dictionary comprehension or iterating through the dictionary. For example: d = {"a": 10, "b": 20, "c": 10, "d": 30} and we have to remove
3 min read
Python - Test if all Values are Same in Dictionary Given a dictionary, test if all its values are the same. Input : test_dict = {"Gfg" : 8, "is" : 8, "Best" : 8} Output : True Explanation : All element values are same, 8. Input : test_dict = {"Gfg" : 8, "is" : 8, "Best" : 9} Output : False Explanation : All element values not same. Method #1: Using
7 min read
Remove Duplicate Dictionaries from Nested Dictionary - Python We are given a nested dictionary we need to remove the duplicate dictionaries from the nested dictionary. For example we are given a nested dictionary d = {'key1': [{'a': 1}, {'b': 2}, {'a': 1}], 'key2': [{'x': 3}, {'y': 4}]} we need to remove the duplicate dictionary from this dictionary so output
4 min read
Python - Remove Dictionary if Given Key's Value is N We are given a dictionary we need to remove key if the given value of key is N. For example, we are given a dictionary d = {'a': 1, 'b': 2, 'c': 3} we need to remove the key if the value is N so that the output becomes {'a': 1, 'c': 3}. We can use methods like del, pop and various other methods like
2 min read