Python Series track

Week 10 · Advanced Track

Week 10 — Advanced Capstone: Async REST Client

Combine the full advanced track into a production-grade async REST API client library, complete with type hints, retry logic, caching, pytest test suite, and a pip-installable package.

Advanced Track1 min readFree lesson

Level: Advanced | Prerequisite: All weeks 1–9

Learning Goals

  • A practical project integrating asynchronous operations, external libraries, tests, and deployment
  • Package structure + CI implementation
my_project/
├── pyproject.toml
├── README.md
├── src/my_project/
│   ├── __init__.py
│   ├── models.py        (dataclass)
│   ├── client.py        (httpx asynchronous)
│   ├── analyzer.py      (Pandas)
│   └── cli.py           (argparse entry point)
├── tests/
│   ├── test_models.py
│   ├── test_client.py
│   └── test_analyzer.py
└── .github/workflows/ci.yml   (optional)

2. Skeleton — Asynchronous Crawler + Analysis

Refer to examples/crawler_skeleton.py.

3. Checklist

  • Type hints + mypy --strict
  • pytest tests (coverage 70%+)
  • CLI (argparse or click)
  • Asynchronous or parallel processing
  • External libraries (requests/httpx, pandas, etc.)
  • pyproject.toml packaging
  • README (installation guide, usage)

Key Examples (examples/)

File Content
crawler_skeleton.py Skeleton for asynchronous crawling + Pandas analysis
cli_skeleton.py Skeleton with argparse + module separation

⚠️ Common Mistakes

  1. Monolithic single file — Hard to test and reuse
  2. Missing type hints — No IDE support
  3. Starting without tests — Harder to add later
  4. No README — Even you will forget after a few days

❓ FAQ

Q1. What is CI? A. Continuous Integration. Automatic testing on each push using GitHub Actions, etc.

Q2. Which domain should I choose? A. Something you can actually use yourself. Example: daily morning news summary, study time tracking, cryptocurrency price alerts.

📝 Final Assignment (exercises/)

Select 1 out of the following 4 options.

  1. exercise1.md — Asynchronous crawler + analysis
  2. exercise2.md — CLI tool package (PyPI distribution)
  3. exercise3.md — Data pipeline (CSV → cleaning → visualization)
  4. exercise4.md — Automation bot (Slack/Discord webhook)

Submission

  • GitHub repository link
  • Execution video or screenshot
  • README (installation, usage, examples)
  • pytest tests
  • (Optional) PyPI or TestPyPI link

Example files

The runnable files used in this lesson. Copy them into your own project and experiment.

examples/cli_skeleton.pyPython
"""argparse + module separation CLI skeleton"""
import argparse
from pathlib import Path
import json

def cmd_add(args):
    db = Path(args.db)
    items = json.loads(db.read_text(encoding="utf-8")) if db.exists() else []
    items.append({"id": len(items) + 1, "task": args.task})
    db.write_text(json.dumps(items, ensure_ascii=False, indent=2), encoding="utf-8")
    print(f"Added: {args.task}")

def cmd_list(args):
    db = Path(args.db)
    if not db.exists():
        print("(Empty)"); return
    for it in json.loads(db.read_text(encoding="utf-8")):
        print(f"{it['id']}. {it['task']}")

def main():
    parser = argparse.ArgumentParser(prog="todo")
    parser.add_argument("--db", default="todos.json")
    parser.add_argument("--version", action="version", version="todo 0.1.0")

    sub = parser.add_subparsers(dest="cmd", required=True)

    add = sub.add_parser("add")
    add.add_argument("task")
    add.set_defaults(func=cmd_add)

    lst = sub.add_parser("list")
    lst.set_defaults(func=cmd_list)

    args = parser.parse_args()
    args.func(args)

if __name__ == "__main__":
    main()
examples/crawler_skeleton.pyPython
"""Asynchronous crawler + Pandas analysis framework"""
import asyncio
import httpx
import pandas as pd
from dataclasses import dataclass

@dataclass
class Article:
    title: str
    score: int
    url: str

async def fetch_one(client: httpx.AsyncClient, url: str) -> str:
    r = await client.get(url, timeout=10)
    return r.text

async def fetch_articles() -> list[Article]:
    """Actually parsed with BeautifulSoup.
    Returns dummy data here."""
    async with httpx.AsyncClient() as client:
        urls = [f"https://httpbin.org/uuid" for _ in range(5)]
        results = await asyncio.gather(*[fetch_one(client, u) for u in urls])
    return [
        Article(title=f"Article {i}", score=100 - i * 10, url=u)
        for i, u in enumerate(urls)
    ]

async def main():
    articles = await fetch_articles()
    df = pd.DataFrame([a.__dict__ for a in articles])
    print(df)
    print(f"Average score: {df.score.mean()}")
    df.to_csv("articles.csv", index=False)

if __name__ == "__main__":
    asyncio.run(main())

Assignment

Goal

Collect data from sites of interest asynchronously → Analyze with Pandas → Report.

Requirements

  • Data sources: News/Hacker News/Public APIs, etc.
  • Asynchronous fetch with httpx or aiohttp
  • Store collected results in dataclass
  • Statistics with pandas (describe(), groupby())
  • Save results as CSV + JSON
  • pytest testing (using mock)

Final Assignment 2. CLI Tool Package

Goal

Create a practical CLI tool and deploy to PyPI or TestPyPI.

Requirements

  • Package structure based on pyproject.toml
  • CLI using argparse or click
  • Call external APIs (weather, exchange rates, GitHub, etc.)
  • Type hints + pass mypy
  • pytest tests
  • Validate upload to TestPyPI

Ideas

  • Currency converter (real-time exchange rate API)
  • Weather notifications (enter location → 5-day forecast)
  • GitHub Stars tracker

Final Assignment 3. Data Pipeline

Goal

CSV/JSON input → Clean → Aggregate → Visualization pipeline.

Requirements

  • Input: actual datasets (Kaggle or public data)
  • Clean with pandas (handle NaN, type conversion)
  • Aggregate using groupby/pivot
  • Visualization with matplotlib (save as PNG)
  • Module separation + testing

Final Assignment 4. Automation Bot

Goal

Collect information regularly with a scheduler and send it via message.

Requirements

  • Slack/Discord webhook integration
  • Collect information from external APIs
  • Scheduler: schedule or cron
  • Record log files
  • Retry on failure

Ideas

  • Send Weather/News summary every morning
  • Coin/Stock price alerts
  • GitHub issue notifications
Show the reference solution

Try the assignment yourself first — comparing your attempt with a working solution is where most of the learning happens.

solution1_skeleton.pyPython
"""Final Assignment 1 Answer Skeleton — Asynchronous Crawler + Pandas

Actual operation after installing dependencies (httpx, pandas, beautifulsoup4)."""
import asyncio
import json
from dataclasses import dataclass, asdict

@dataclass
class Article:
    title: str
    score: int
    url: str

async def fetch_articles() -> list[Article]:
    #TODO: Parse the actual site
    return [
        Article(title=f"Article{i}", score=100 - i * 5, url=f"https://example.com/{i}")
        for i in range(20)
    ]

def analyze(articles: list[Article]) -> dict:
    if not articles:
        return {}
    scores = [a.score for a in articles]
    return {
        "count": len(articles),
        "avg": sum(scores) / len(scores),
        "max": max(scores),
        "min": min(scores),
    }

async def main():
    articles = await fetch_articles()
    stats = analyze(articles)
    print(json.dumps(stats, indent=2))

    #Save as CSV
    with open("articles.csv", "w", encoding="utf-8") as f:
        f.write("title,score,url\n")
        for a in articles:
            f.write(f"{a.title},{a.score},{a.url}\n")

if __name__ == "__main__":
    asyncio.run(main())

Learn this lesson with a mentor

Work through the Python Series track on your own, or join a guided cohort at our Pitampura campus and online.

Enroll nowTrack overview
Call NowEnroll Now