Python Development Patterns

Idiomatic Python patterns and best practices for building robust, efficient, and maintainable applications.

When to Activate

• Designing Python package structure and module boundaries
• Choosing between dataclasses, Pydantic models, and TypedDicts for a data structure
• Applying idiomatic Python (list comprehensions, generators, context managers, decorators)
• Setting up type hints and mypy for a new module or existing codebase
• Structuring a Python service with dependency injection or hexagonal architecture
• Deciding when to use ABCs vs protocols for interface design

Core Principles

1. Readability Counts

Python prioritizes readability. Code should be obvious and easy to understand.

# Good: Clear and readable
def get_active_users(users: list[User]) -> list[User]:
    """Return only active users from the provided list."""
    return [user for user in users if user.is_active]


# Bad: Clever but confusing
def get_active_users(u):
    return [x for x in u if x.a]

2. Explicit is Better Than Implicit

Avoid magic; be clear about what your code does.

# Good: Explicit configuration
import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Bad: Hidden side effects
import some_module
some_module.setup()  # What does this do?

3. EAFP - Easier to Ask Forgiveness Than Permission

Python prefers exception handling over checking conditions.

# Good: EAFP style
def get_value(dictionary: dict, key: str) -> Any:
    try:
        return dictionary[key]
    except KeyError:
        return default_value

# Bad: LBYL (Look Before You Leap) style
def get_value(dictionary: dict, key: str) -> Any:
    if key in dictionary:
        return dictionary[key]
    else:
        return default_value

When to use which: Use EAFP for external resources (files, APIs, DB) where failures are expected and informative. Use LBYL for simple attribute checks on known objects where a conditional read is cheaper than exception handling.

Type Hints

Type Annotations (Python 3.10+)

Use built-in generic types directly (list[str], dict[str, Any], X | None) — no typing imports for basic annotations:

from typing import Any

def process_user(user_id: str, data: dict[str, Any], active: bool = True) -> User | None:
    return User(user_id, data) if active else None

def process_items(items: list[str]) -> dict[str, int]:
    return {item: len(item) for item in items}

Type Aliases and Generics (Python 3.12+)

from typing import Any

# Type alias (PEP 695 — Python 3.12+)
type JSON = dict[str, Any] | list[Any] | str | int | float | bool | None

def parse_json(data: str) -> JSON:
    return json.loads(data)

# Generic functions with new syntax (Python 3.12+)
def first[T](items: list[T]) -> T | None:
    """Return the first item or None if list is empty."""
    return items[0] if items else None

# Generic classes (Python 3.12+)
class Stack[T]:
    def __init__(self) -> None:
        self._items: list[T] = []

    def push(self, item: T) -> None:
        self._items.append(item)

    def pop(self) -> T:
        return self._items.pop()

Protocol-Based Duck Typing

from typing import Protocol

class Renderable(Protocol):
    def render(self) -> str:
        """Render the object to a string."""

def render_all(items: list[Renderable]) -> str:
    """Render all items that implement the Renderable protocol."""
    return "\n".join(item.render() for item in items)

Error Handling Patterns

Specific Exception Handling

# Good: Catch specific exceptions
def load_config(path: str) -> Config:
    try:
        with open(path) as f:
            return Config.from_json(f.read())
    except FileNotFoundError as e:
        raise ConfigError(f"Config file not found: {path}") from e
    except json.JSONDecodeError as e:
        raise ConfigError(f"Invalid JSON in config: {path}") from e

# Bad: Bare except
def load_config(path: str) -> Config:
    try:
        with open(path) as f:
            return Config.from_json(f.read())
    except:
        return None  # Silent failure!

Custom Exception Hierarchy

class AppError(Exception):
    """Base exception for all application errors."""
    pass

class ValidationError(AppError):
    """Raised when input validation fails."""
    pass

class NotFoundError(AppError):
    """Raised when a requested resource is not found."""
    pass

# Usage
def get_user(user_id: str) -> User:
    user = db.find_user(user_id)
    if not user:
        raise NotFoundError(f"User not found: {user_id}")
    return user

Context Managers

Always use with for resource management (files, DB connections, locks). Prefer @contextmanager for simple cases; use __enter__/__exit__ classes for stateful resources.

Custom Context Managers

from contextlib import contextmanager

@contextmanager
def timer(name: str):
    """Context manager to time a block of code."""
    start = time.perf_counter()
    yield
    elapsed = time.perf_counter() - start
    print(f"{name} took {elapsed:.4f} seconds")

# Usage
with timer("data processing"):
    process_large_dataset()

Context Manager Classes

class DatabaseTransaction:
    def __init__(self, connection):
        self.connection = connection

    def __enter__(self):
        self.connection.begin_transaction()
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is None:
            self.connection.commit()
        else:
            self.connection.rollback()
        return False  # Don't suppress exceptions

# Usage
with DatabaseTransaction(conn):
    user = conn.create_user(user_data)
    conn.create_profile(user.id, profile_data)

Comprehensions and Generators

# List comprehension — prefer over manual loops
names = [user.name for user in users if user.is_active]

# Generator expression — lazy, no intermediate list (O(1) memory)
total = sum(x * x for x in range(1_000_000))

# Generator function — stream large data without loading all into memory
def read_large_file(path: str) -> Iterator[str]:
    with open(path) as f:
        for line in f:
            yield line.strip()

Keep comprehensions simple — if a comprehension needs more than one if or a nested loop, use a regular function instead.

Data Classes

from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class User:
    """Automatic __init__, __repr__, __eq__. Use __post_init__ for validation."""
    id: str
    name: str
    email: str
    created_at: datetime = field(default_factory=datetime.now)
    is_active: bool = True

    def __post_init__(self):
        if "@" not in self.email:
            raise ValueError(f"Invalid email: {self.email}")

For advanced patterns — NamedTuple with methods, function decorators, parameterized decorators, class-based decorators, concurrency (threading, multiprocessing, async/await), hexagonal architecture with FastAPI, and memory optimization — see python-patterns-advanced.

Anti-Patterns

Using a Mutable Default Argument

Wrong:

def append_item(item, collection=[]):  # list is created once at definition time
    collection.append(item)
    return collection

append_item("a")  # ["a"]
append_item("b")  # ["a", "b"]  — unexpected: shares the same list

Correct:

def append_item(item, collection=None):
    if collection is None:
        collection = []
    collection.append(item)
    return collection

Why: Default argument values are evaluated once when the function is defined; mutable defaults accumulate state across calls.

---

Catching Exception (or bare except) and Swallowing It

Wrong:

def load_user(user_id: str) -> User | None:
    try:
        return db.find(user_id)
    except Exception:
        return None  # hides programming errors, network failures, everything

Correct:

def load_user(user_id: str) -> User | None:
    try:
        return db.find(user_id)
    except UserNotFoundError:
        return None
    except DatabaseError as e:
        raise ServiceUnavailableError("Database unreachable") from e

Why: Catching Exception silently swallows bugs and infrastructure failures, making them impossible to observe or recover from correctly.

---

Building Strings with + in a Loop

Wrong:

def build_csv(rows: list[dict]) -> str:
    result = ""
    for row in rows:
        result += ",".join(str(v) for v in row.values()) + "\n"
    return result

Correct:

def build_csv(rows: list[dict]) -> str:
    lines = [",".join(str(v) for v in row.values()) for row in rows]
    return "\n".join(lines) + "\n"

Why: String concatenation in a loop is O(n²) because each + creates a new string; str.join is O(n).

---

Using Optional[X] Instead of X | None in Modern Python

Wrong:

from typing import Optional

def find_user(user_id: str) -> Optional[User]:
    ...

Correct:

def find_user(user_id: str) -> User | None:
    ...

Why: X | None (PEP 604, Python 3.10+) is the idiomatic modern syntax; Optional requires a typing import and is now considered legacy style.

---

Using type(x) == SomeClass Instead of isinstance

Wrong:

def process(value):
    if type(value) == int:
        return value * 2

Correct:

def process(value):
    if isinstance(value, int):
        return value * 2

Why: type(x) == SomeClass breaks for subclasses and does not respect the Python type hierarchy; isinstance handles inheritance correctly.

For advanced patterns — concurrency (threading, multiprocessing, async/await), hexagonal architecture with FastAPI (full working code, Protocol ports, DI wiring, RFC 7807 error handling, tests), memory optimization, tooling (pyproject.toml, ruff, mypy), and anti-patterns — see skill: python-patterns-advanced.