Difficulty: Intermediate | Prerequisites: Python Lists, Loops and Functions; Dictionaries and Enumerate study notes
Higher-order functions (filter, map, reduce) and list comprehensions are Python's tools for transforming and summarising data without writing explicit loops. In data science, these patterns replace the verbose for-loop-and-append approach with concise, readable one-liners. This is where Python starts to feel like a data manipulation language rather than a general-purpose scripting language. You should be comfortable writing functions, using lambda expressions, and working with lists of dictionaries before diving in.
filter() selects elements that pass a test, map() transforms every element, and reduce() collapses a sequence into a single value. Nested list comprehensions flatten multi-level lists, and wrapping the result in set() gives you unique values.
Lambda function
An anonymous, single-expression function defined inline. lambda x: x > 5 is equivalent to writing a named function that returns x > 5. Think of it as a throwaway function you use once and do not need to name.
filter(function, iterable)
Returns an iterator of elements from iterable for which function returns True. Wrap it in list() to get a list back. In simple terms, it keeps the items that pass a test and discards the rest.
map(function, iterable)
Applies function to every element of iterable and returns an iterator of the results. Wrap it in list() for a list. In simple terms, it transforms each item in the same way.
reduce(function, iterable)
From the functools module. Applies a two-argument function cumulatively to the elements, reducing the sequence to a single value. reduce(lambda x, y: x + y, [1, 2, 3]) computes ((1 + 2) + 3) = 6. Think of it as folding a list down into one result.
List comprehension
A compact syntax for building a list: [expression for item in iterable]. Equivalent to a for-loop that appends each result. In simple terms, it is a one-line loop that collects results into a list.
Nested list comprehension
A list comprehension with multiple for clauses. [item for sublist in big_list for item in sublist] flattens a list of lists into a single list. The outer for comes first, then the inner for.
set()
An unordered collection of unique elements. Wrapping a list in set() removes duplicates. In simple terms, it answers "what distinct values exist?"
functools
A standard library module containing higher-order utility functions. You need from functools import reduce because reduce is not a built-in like filter and map.
Given a list of workout dictionaries, filter() with a lambda selects items that meet a condition.
intense_workouts = list(filter(
lambda w: w['duration'] > 50 or w['calories'] >= 300,
workouts
))The lambda receives one dictionary w at a time.
It returns True for workouts where duration exceeds 50 minutes OR calories are at least 300.
filter returns an iterator, so wrap it in list() to materialise the result.
The condition uses or, meaning either criterion is sufficient.
map() applies a transformation to every element. Here, converting durations from minutes to hours:
durations = list(map(lambda w: w['duration'] / 60, workouts))The lambda extracts the 'duration' key and divides by 60.
The output is a new list of floats, one per workout.
The original workouts list is not modified.
reduce() accumulates values. Here, summing all durations:
from functools import reduce
total_duration = reduce(lambda x, y: x + y, durations)x is the running total, y is the next element.
On the first call, x = durations[0] and y = durations[1]. The result becomes the new x for the next call.
The final result is a single number: the total.
You must import reduce from functools.
A nested list of workout types per week can be flattened and deduplicated:
weeks = [['cardio', 'strength'], ['yoga'], ['cardio', 'strength', 'yoga']]
unique = set([
workout for week in weeks for workout in week
])The outer for week in weeks iterates over sub-lists.
The inner for workout in week iterates within each sub-list.
The comprehension produces ['cardio', 'strength', 'yoga', 'cardio', 'strength', 'yoga'].
Wrapping in set() removes duplicates, yielding {'cardio', 'strength', 'yoga'}.
The reading order of nested comprehensions matches nested for-loops: outer loop first, inner loop second.
Forgetting to wrap filter() and map() in list(). Both return iterators, not lists. If you try to index the result or print it directly, you will get an iterator object, not the data.
Getting the nested comprehension order wrong. Students often write for workout in week for week in weeks, which is backwards. The outer loop (for week in weeks) must come first.
Confusing reduce with map. map produces a list of the same length; reduce produces a single value. They serve completely different purposes.
Using reduce without importing it. Unlike filter and map, reduce lives in the functools module. Forgetting the import causes a NameError.
Writing lambda w: w['duration'] > 50 and w['calories'] >= 300 when the question says "or". Read the condition carefully: "either above 50 minutes OR at least 300 calories" means or, not and.
⚠️ The exam asks you to fill in the lambda expression inside filter(), map(), or reduce(). You must know the signature of each.
⚠️ Nested list comprehension syntax is tested. The reading order matters: outer loop first.
⚠️ The or vs and distinction in filter conditions is a deliberate exam trap.
⚠️ You are expected to know that reduce requires an import from functools.
What does list(filter(lambda x: x > 3, [1, 2, 3, 4, 5])) return?
True or False: map() changes the length of the input list.
What module do you import reduce from?
In [x for sub in big for x in sub], which loop runs first?
True or False: set([1, 2, 2, 3]) returns [1, 2, 3].
Answers: 1. [4, 5]. 2. False (same length, different values). 3. functools. 4. The outer loop (for sub in big). 5. False (it returns {1, 2, 3}, a set, not a list).
Q: Write a filter() call that selects workouts with duration > 50 or calories >= 300.
A: list(filter(lambda w: w['duration'] > 50 or w['calories'] >= 300, workouts))
Q: Use map() to convert all durations from minutes to hours.
A: list(map(lambda w: w['duration'] / 60, workouts))
Q: Use reduce() to compute the total of a list of durations already in hours.
A: from functools import reduce then reduce(lambda x, y: x + y, durations)
Q: Given weeks = [['cardio', 'strength'], ['yoga'], ['cardio', 'strength', 'yoga']], write a nested list comprehension wrapped in set() to get unique workout types.
A: set([workout for week in weeks for workout in week])
These higher-order functions are the conceptual precursors to pandas operations: filter maps to boolean indexing, map to .apply(), and reduce to .agg(). List comprehensions also connect to NumPy vectorised operations, which do the same transformations far more efficiently on large datasets.
lambda function, filter, map, reduce, functools, list comprehension, nested list comprehension, set, higher-order function, anonymous function, Python functional programming, ECE 20875, Python for Data Science, Purdue