Week 03 · Advanced Track

Week 3 — Iterators & Generators

Implement the iterator protocol from scratch, write memory-efficient generators with yield, compose pipelines with yield from, and use generator's send() for two-way communication.

Advanced Track2 min readFree lesson

Level: Advanced | Prerequisite: Week 2 + Intermediate Week 2

Learning Goals

Implement the iterator protocol (__iter__, __next__) directly
Delegate to a generator using yield from
Utilize the powerful functions of itertools
Coroutine basics (send)

1. Iterator Protocol

for ... in x is basically iter(x) → next(it) repeatedly.

class Range:
    def __init__(self, start, stop):
        self.cur = start
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.cur >= self.stop:
            raise StopIteration
        self.cur += 1
        return self.cur - 1

print(list(Range(1, 5)))   # [1, 2, 3, 4]

2. yield from

Yields another iterable as-is. Useful for flattening.

def chain(*iters):
    for it in iters:
        yield from it

print(list(chain([1, 2], (3, 4), "ab")))   # [1, 2, 3, 4, 'a', 'b']

3. itertools

A gem of the standard library.

import itertools as it

#Infinite Sequence
counter = it.count(1, 2)        # 1, 3, 5, 7, ...
print(list(it.islice(counter, 5)))  # [1, 3, 5, 7, 9]

#Combination/Permutation
print(list(it.combinations([1, 2, 3, 4], 2)))   # (1,2),(1,3),(1,4),(2,3)...
print(list(it.permutations([1, 2, 3], 2)))

#Group
data = [("a", 1), ("a", 2), ("b", 3), ("b", 4)]
for key, group in it.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

#Dictionary Product
print(list(it.product([1, 2], "ab")))   # (1,'a'),(1,'b'),(2,'a'),(2,'b')

#Cumulative
print(list(it.accumulate([1, 2, 3, 4])))   # [1, 3, 6, 10]

4. Coroutine Basics

yield allows two-way communication — inject values with send(value).

def echo():
    while True:
        x = yield
        print(f"Received: {x}")

co = echo()
next(co)            #Proceed until the first yield (required)
co.send("hello")    #Received: hello
co.send("world")    #Received: world

(From Python 3.5+, async/await is more common — next week)

5. Generator vs List

	Generator	List
Memory	One item at a time	Entire collection
Length	Unknown	`len()`
Reuse	Only once	Multiple times
Indexing	Not possible	Possible

Key Examples (examples/)

File	Content
`01_iterator.py`	Implement iterator directly
`02_yield_from.py`	yield from
`03_itertools.py`	combinations, groupby, accumulate
`04_coroutine.py`	Send-based coroutine

⚠️ Common Mistakes

Iterate over an iterator twice — second time yields empty results; needs to be recreated.
Omitting StopIteration in __next__ — infinite loop.
Do not sort before groupby — groupby only groups consecutive identical keys.

❓ FAQ

Q1. What is the difference between an iterator and a generator? A. A generator is a type of iterator. Made using yield.

Q2. I heard more-itertools is better than itertools? A. Not standard, but powerful. Often used in practice.

📝 Exercises (exercises/)

exercise1.md — chunked_lines(path, n) generator
exercise2.md — groupby group logs by date
exercise3.md — infinite Fibonacci + islice

Next Week

Week 4. Concurrency (1) - threading

Example files

The runnable files used in this lesson. Copy them into your own project and experiment.

examples/01_iterator.pyPython

class Range:
    def __init__(self, start, stop):
        self.cur = start
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.cur >= self.stop:
            raise StopIteration
        self.cur += 1
        return self.cur - 1

print(list(Range(1, 5)))

#The for loop actually works like this
r = Range(1, 4)
it = iter(r)
while True:
    try:
        print(next(it))
    except StopIteration:
        break

examples/02_yield_from.pyPython

def chain(*iters):
    for it in iters:
        yield from it

print(list(chain([1, 2], (3, 4), "ab")))

def flatten(nested):
    for item in nested:
        if isinstance(item, list):
            yield from flatten(item)   #Recursion
        else:
            yield item

print(list(flatten([1, [2, [3, [4, 5]]], 6])))

examples/03_itertools.pyPython

import itertools as it

# count + islice
counter = it.count(1, 2)
print(list(it.islice(counter, 5)))   # [1, 3, 5, 7, 9]

# combinations
print(list(it.combinations([1, 2, 3, 4], 2)))

# permutations
print(list(it.permutations([1, 2, 3], 2)))

#groupby (sorted data needed)
data = [("a", 1), ("a", 2), ("b", 3), ("b", 4), ("a", 5)]
data.sort(key=lambda x: x[0])
for key, group in it.groupby(data, key=lambda x: x[0]):
    print(key, list(group))

#product (pre-multiplication)
print(list(it.product([1, 2], "ab")))

# accumulate
print(list(it.accumulate([1, 2, 3, 4, 5])))

examples/04_coroutine.pyPython

def echo():
    while True:
        x = yield
        print(f"Received: {x}")

co = echo()
next(co)                #Proceed until the first yield
co.send("hello")
co.send("world")
co.close()

#Accumulator with state
def averager():
    total = 0
    count = 0
    avg = None
    while True:
        x = yield avg
        total += x
        count += 1
        avg = total / count

a = averager()
next(a)
print(a.send(10))   # 10.0
print(a.send(20))   # 15.0
print(a.send(30))   # 20.0

Assignment

Goal

A generator that yields the file in chunks of N lines.

Example of use

for chunk in chunked_lines("big.txt", 100):
    process(chunk)   #Process 100 lines at a time

Task 2. Group logs by date using groupby

Goal

Output log lines grouped by date.

Input

2026-05-09 10:00 INFO Start
2026-05-09 10:05 ERROR Error
2026-05-10 09:00 INFO Start
2026-05-10 11:00 INFO Normal

Output

[2026-05-09]
10:00 INFO Start
10:05 ERROR Error
[2026-05-10]
09:00 INFO Start
11:00 INFO Normal

Hint

itertools.groupby(lines, key=date_extraction_function)

Task 3. Infinite Fibonacci + islice

Goal

A generator that yields Fibonacci numbers infinitely, itertools.islice for only the first 20.

Input/Output Example

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181]

Show the reference solution

Try the assignment yourself first — comparing your attempt with a working solution is where most of the learning happens.

solution1.pyPython

from pathlib import Path

def chunked_lines(path, n):
    chunk = []
    with open(path, encoding="utf-8") as f:
        for line in f:
            chunk.append(line.rstrip())
            if len(chunk) >= n:
                yield chunk
                chunk = []
    if chunk:
        yield chunk

#Demo: Yourself in 3 lines each
self_path = Path(__file__)
for i, chunk in enumerate(chunked_lines(self_path, 3), 1):
    print(f"--- chunk {i} ---")
    for line in chunk:
        print(line)

solution2.pyPython

import itertools as it

LOG = """2026-05-09 10:00 INFO Start
2026-05-09 10:05 ERROR Error
2026-05-10 09:00 INFO Start
2026-05-10 11:00 INFO Normal""".splitlines()

def date_of(line):
    return line.split(" ", 1)[0]

#Sorted assumption. groupby only groups consecutive identical keys
for date, group in it.groupby(LOG, key=date_of):
    print(f"[{date}]")
    for line in group:
        print(f"  {line[len(date)+1:]}")

solution3.pyPython

import itertools as it

def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

print(list(it.islice(fib(), 20)))

Learn this lesson with a mentor

Work through the Python Series track on your own, or join a guided cohort at our Pitampura campus and online.

Enroll now Track overview