Core Python·Lesson 11 of 67·intermediate
Comprehensions and generator expressions
Build a list, dict or set in one expression — and learn when swapping the brackets for parentheses turns the whole thing lazy.
In this lesson · 16 min
- →Write list, dict and set comprehensions with filters
- →Flatten and nest comprehensions without losing the plot
- →Swap a list comprehension for a generator expression when the data is large
- →Pair generator expressions with sum, any, all, min and max
- →Recognise when a plain for loop is the more readable choice
You met a list comprehension briefly in module 1. It is worth a lesson of its own, because comprehensions are how most idiomatic Python builds a collection — and because the lazy version is what lets you process a corpus that does not fit in memory.
The shape is always [ what to collect for name in iterable ].
Filtering
Add if at the end to keep only some items:
An if before the for is a different thing entirely — that is a conditional expression
choosing what to collect, and it cannot skip items:
Dictionaries and sets
Change the brackets and you change the container:
Inverting a dictionary is a one-liner, and a useful reminder that keys must stay unique:
Nesting and flattening
Two for clauses read in the same order you would write the nested loop:
A comprehension whose expression is itself a comprehension builds a nested structure:
Generator expressions
Swap the square brackets for parentheses and nothing is built. You get a generator: an object that produces items one at a time, on demand.
That last line is the whole trade-off. A list can be read many times; a generator is a one-pass stream. What you get in return is memory:
Generators inside other functions
When a generator expression is the only argument, you can drop its parentheses. This is the form you will write most often:
any and all stop as soon as the answer is settled, so pairing them with a generator means
you never do more work than the question requires:
When a loop is better
A comprehension should build one collection from one source. The moment it needs to do two things, keep running totals, or handle errors, write the loop:
Try it yourself
Build long_docs: a dictionary of id → tokens holding only the documents with more than 500 tokens. Use a dict comprehension.
Print chars: N — the total number of characters across every chunk — without building an intermediate list.
Check yourself
What is the difference between sum([len(c) for c in chunks]) and sum(len(c) for c in chunks)?
Check yourself
After g = (n for n in range(3)), what does list(g) return the second time you call it?
Recap
[expr for x in items if cond]builds a list;{}builds a dict or a set.- A trailing
iffilters; a leadingif … elsetransforms and keeps every item. - Two
forclauses read in the same order as the nested loop they replace. Stop at two. - Parentheses give a generator expression: lazy, one-pass, constant memory.
sum,any,all,min,maxandjoinall take a generator directly, andany/allstop early.- If the body needs error handling or two outputs, write the loop instead.