litestar-ai-serving

Litestar AI Serving

Use this skill for HTTP-facing AI agent endpoints, Google ADK integration, session-backed conversations, and Litestar service boundaries around model workflows.

Code Style Rules

• Keep agent orchestration behind service functions or providers.
• Use typed request and response DTOs at the HTTP boundary.
• Store multi-turn state through the project's database stack.
• Stream only when the client contract needs incremental output.

Quick Reference

• AI serving patterns: ai-serving.md
• Pair with sqlspec for ADK session stores.
• Pair with litestar-realtime for streaming or event fan-out.

Workflow

1. Define the HTTP contract before agent internals.
2. Wire agent runners through DI.
3. Persist session state through the chosen data stack.
4. Test deterministic failure, timeout, and cancellation paths.

Guardrails

• Do not expose raw agent internals as the API contract.
• Do not block request workers with unbounded model calls.
• Do not store prompts, tool outputs, or memory without a retention decision.
• Do not skip authorization on agent endpoints.

Validation Checkpoint

• Request and response DTOs are explicit.
• Session persistence is wired.
• Timeouts and model failures are handled.
• Auth policy matches the sensitivity of tools and data.

Example

@get("/chat/{session_id:str}")
async def chat(session_id: str, runner: Runner, body: ChatRequest) -> ChatResponse:
    result = await runner.run_async(session_id=session_id, new_message=body.message)
    return ChatResponse(message=result.final_response)

References Index

• ai-serving.md

Official References

• https://docs.litestar.dev/ - Litestar documentation
• https://docs.litestar.dev/latest/reference/ - Litestar API reference

Shared Styleguide Baseline

• General
• Python
• Litestar