python-app-conventions

Python Application Conventions

Detection — understanding the project type

Before writing code, read pyproject.toml (or setup.py / requirements.txt) to understand what kind of project this is:

# pyproject.toml signals to look for:

[project.scripts]           # → CLI tool; entry points are registered here
myapp = "myapp.cli:main"

[project]
dependencies = [            # → check for framework deps (fastapi, django, flask)
  "pydantic-settings",      # → configuration via env vars
  "click",                  # → CLI framework in use
]

[build-system]
requires = ["poetry-core"]  # → Poetry project
requires = ["hatchling"]    # → Hatch project
requires = ["setuptools"]   # → setuptools / pip project

No fastapi, django, or flask in dependencies → plain Python project; this skill applies.

---

Source layout

Preferred: src/ layout

myproject/
├── pyproject.toml
├── README.md
├── src/
│   └── mypackage/
│       ├── __init__.py        # expose public API only — not everything
│       ├── __main__.py        # enables: python -m mypackage
│       ├── py.typed           # PEP 561 marker — enables mypy type checking by consumers
│       ├── cli.py             # CLI entry point (argparse / click / typer)
│       ├── config.py          # pydantic-settings Settings class
│       ├── core.py            # core business logic
│       └── exporters/
│           ├── __init__.py
│           └── csv_exporter.py
└── tests/
    ├── conftest.py
    ├── test_core.py
    └── exporters/
        └── test_csv_exporter.py

__init__.py exposes the public API explicitly:

# src/mypackage/__init__.py
from mypackage.core import Pipeline
from mypackage.exporters.csv_exporter import CsvExporter

__all__ = ["Pipeline", "CsvExporter"]

Acceptable: flat layout (small projects / scripts)

myproject/
├── pyproject.toml
├── mypackage.py      # single-module library
└── tests/
    └── test_mypackage.py

Or a package without src/:

myproject/
├── pyproject.toml
├── mypackage/
│   ├── __init__.py
│   └── core.py
└── tests/
    └── conftest.py

Match whichever layout the project already uses. Never restructure an existing project unless the BA spec explicitly requires it.

---

CLI with argparse (stdlib, no additional deps)

Use argparse when the project has no CLI framework in its dependencies and adding one is out of scope.

# src/mypackage/cli.py
from __future__ import annotations

import argparse
import sys
from pathlib import Path

from mypackage.config import Settings
from mypackage.core import Pipeline


def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        prog="mypackage",
        description="Process data files and export results.",
    )
    sub = parser.add_subparsers(dest="command", required=True)

    run_cmd = sub.add_parser("run", help="Run the pipeline.")
    run_cmd.add_argument("input", type=Path, help="Input file path.")
    run_cmd.add_argument(
        "--output",
        type=Path,
        default=None,
        help="Output directory (default: from MYAPP_OUTPUT_DIR env var).",
    )
    run_cmd.add_argument(
        "--format",
        choices=["csv", "json"],
        default="csv",
        help="Output format.",
    )

    return parser


def main(argv: list[str] | None = None) -> int:
    parser = build_parser()
    args = parser.parse_args(argv)
    settings = Settings()

    output_dir = args.output or settings.output_dir

    try:
        pipeline = Pipeline(settings=settings)
        pipeline.run(input_path=args.input, output_dir=output_dir, fmt=args.format)
    except FileNotFoundError as exc:
        print(f"Error: {exc}", file=sys.stderr)
        return 1

    return 0


if __name__ == "__main__":
    sys.exit(main())

---

CLI with click (feature-rich, composable commands)

Use click when it is already in the project's dependencies, or when the CLI has many subcommands, option validation, or prompt interactions.

# src/mypackage/cli.py
from __future__ import annotations

from pathlib import Path

import click

from mypackage.config import Settings
from mypackage.core import Pipeline


@click.group()
def cli() -> None:
    """Process data files and export results."""


@cli.command()
@click.argument("input", type=click.Path(exists=True, path_type=Path))
@click.option(
    "--output",
    type=click.Path(path_type=Path),
    default=None,
    help="Output directory. Defaults to MYAPP_OUTPUT_DIR env var.",
)
@click.option(
    "--format",
    "fmt",
    type=click.Choice(["csv", "json"]),
    default="csv",
    show_default=True,
)
def run(input: Path, output: Path | None, fmt: str) -> None:
    """Run the pipeline on INPUT file."""
    settings = Settings()
    output_dir = output or settings.output_dir
    Pipeline(settings=settings).run(input_path=input, output_dir=output_dir, fmt=fmt)


def main() -> None:
    cli()

When to prefer each CLI framework:

Framework	Choose when
argparse	No CLI deps allowed; stdlib only; simple, stable CLI
click	Feature-rich CLI (prompts, colors, progress bars); composable command groups; already in the project
typer	Type-annotated, FastAPI-style API; rapid prototyping; team already uses FastAPI/Pydantic

Match what the project already uses. Do not introduce a new CLI framework without BA approval.

---

CLI with typer

Use typer when it is already in the project's dependencies, or when the team prefers type-annotated CLI definitions.

# src/mypackage/cli.py
from __future__ import annotations

from pathlib import Path
from typing import Annotated

import typer

from mypackage.config import Settings
from mypackage.core import Pipeline

app = typer.Typer(help="Process data files and export results.")


@app.command()
def run(
    input: Annotated[Path, typer.Argument(help="Input file path.", exists=True)],
    output: Annotated[
        Path | None,
        typer.Option(help="Output directory. Defaults to MYAPP_OUTPUT_DIR env var."),
    ] = None,
    fmt: Annotated[str, typer.Option("--format", help="Output format.")] = "csv",
) -> None:
    """Run the pipeline on INPUT."""
    settings = Settings()
    output_dir = output or settings.output_dir
    Pipeline(settings=settings).run(input_path=input, output_dir=output_dir, fmt=fmt)


def main() -> None:
    app()

---

Configuration with pydantic-settings

Read all configuration from environment variables (and optionally a .env file). Never call os.environ.get() inline — consolidate all env var reads into a single Settings class.

# src/mypackage/config.py
from __future__ import annotations

from pathlib import Path

from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_prefix="MYAPP_",      # MYAPP_OUTPUT_DIR, MYAPP_LOG_LEVEL, etc.
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,
    )

    output_dir: Path = Field(default=Path("/tmp/myapp-output"), description="Output directory for exported files.")
    log_level: str = Field(default="INFO", description="Logging level (DEBUG, INFO, WARNING, ERROR).")
    api_key: str = Field(description="External API key. Required. Set via MYAPP_API_KEY env var.")
    max_workers: int = Field(default=4, ge=1, le=32, description="Thread pool size for parallel processing.")


# Singleton — import this throughout the codebase instead of creating new instances
settings = Settings()

Nested settings for complex configuration:

from pydantic import BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict


class DatabaseSettings(BaseModel):
    host: str = "localhost"
    port: int = 5432
    name: str = "myapp"


class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_prefix="MYAPP_", env_nested_delimiter="__")

    database: DatabaseSettings = DatabaseSettings()
    # Set via: MYAPP_DATABASE__HOST=db.prod.example.com

Rules:

• Always use env_prefix to namespace your application's env vars.
• Mark required fields (no default) — pydantic-settings raises ValidationError at startup if they are missing, giving a clear error message.
• Never read os.environ directly in business logic — always go through Settings.

---

Structured logging

Use Python's standard logging module. Configure it once at the application entry point. Never use print() for diagnostics.

# src/mypackage/logging_config.py
from __future__ import annotations

import logging
import sys


def configure_logging(level: str = "INFO") -> None:
    """Configure root logger for the application. Call once at startup."""
    logging.basicConfig(
        level=level.upper(),
        format="%(asctime)s %(levelname)-8s %(name)s  %(message)s",
        datefmt="%Y-%m-%dT%H:%M:%S",
        stream=sys.stderr,
    )

In every module, get a module-scoped logger:

# src/mypackage/core.py
from __future__ import annotations

import logging

logger = logging.getLogger(__name__)


class Pipeline:
    def run(self, input_path: Path, ...) -> None:
        logger.info("Starting pipeline run", extra={"input": str(input_path)})
        try:
            result = self._process(input_path)
            logger.debug("Processing complete, %d records produced", len(result))
        except OSError as exc:
            logger.error("Failed to read input file: %s", exc)
            raise

JSON logging for production (use python-json-logger or structlog when already in the project):

# with python-json-logger
import logging
from pythonjsonlogger.json import JsonFormatter

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter("%(asctime)s %(levelname)s %(name)s %(message)s"))
logging.root.addHandler(handler)
logging.root.setLevel("INFO")

Rule: never use print() for diagnostics outside of __main__.py / cli.py (where printing to stdout is intentional CLI output, not debug noise).

---

Entry points and pyproject.toml packaging

Register CLI commands as console scripts so they are available after pip install / poetry install:

# pyproject.toml

[project]
name = "mypackage"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
    "pydantic-settings>=2.0",
    "click>=8.0",
]

[project.scripts]
myapp = "mypackage.cli:main"          # installs `myapp` command
myapp-admin = "mypackage.admin_cli:main"  # second entry point if needed

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-cov",
    "ruff",
    "mypy",
]

__main__.py allows python -m mypackage without installing the package:

# src/mypackage/__main__.py
import sys

from mypackage.cli import main

sys.exit(main())

---

Module design anti-patterns

Do NOT	Do instead
Import everything in __init__.py	Expose only the public API (__all__) — lazy imports or explicit imports of public symbols only
Use mutable default arguments (def f(items=[]))	Use None as default and initialise inside the function (if items is None: items = [])
Use global mutable state (_cache = {} at module level)	Inject dependencies via constructor or function argument; use functools.lru_cache for pure memoisation
Use print() for diagnostics in library code	Use logging.getLogger(__name__) — callers control the log level and destination
Catch bare except:	Catch specific exceptions (except ValueError:, except OSError as e:)
Inline os.environ.get("API_KEY") throughout codebase	Centralise all env-var reads in a Settings class (pydantic-settings or python-decouple)