Define success metrics for an AI feature — model quality, operational, product, and business KPIs

Evaluate and compare AI models or vendors for a specific product use case

Create an AI product roadmap that accounts for model uncertainty, data requirements, and iterative improvement cycles

Create an AI feature specification with behaviour requirements, evaluation criteria, and fallback design

Review an AI feature or product for ethical risks, bias, safety issues, and regulatory compliance

Evaluate AI capability sourcing options across build, buy, fine-tune, and partner archetypes using a structured decision matrix. Use when deciding whether to build a custom model, use an LLM API, fine-tune a base model, or partner with an AI vendor.

Define a data strategy for AI products covering training data requirements, data quality, labeling strategy, feedback loops, and continuous retraining. Use when assessing data readiness for an AI feature, designing a data flywheel, or planning an ML data pipeline.

Write a complete AI feature specification that defines desired model behaviour, input/output examples, confidence thresholds, fallback logic, and non-functional requirements. Use when defining requirements for an AI-powered feature, writing an AI PRD, or bridging the gap between PM intent and ML team implementation.

Handle AI model failures, quality regressions, bias incidents, and safety issues with a structured response runbook. Use when an AI feature produces harmful outputs, when model quality degrades in production, or when preparing an incident response plan for an AI system.

Evaluate and compare LLMs, ML APIs, and fine-tuned models for product fit across quality, latency, cost, compliance, and vendor risk dimensions. Use when selecting an AI model or vendor, comparing foundation model options, or making build-vs-API decisions for a product use case.

Research user expectations, mental models, trust calibration, and interaction patterns for AI-powered features. Use when planning user research for an AI feature, evaluating user trust and satisfaction with AI outputs, or designing feedback collection for a machine learning system.

Craft, evaluate, and manage production-quality prompts including system prompts, few-shot examples, chain-of-thought instructions, and guardrails. Use when designing prompts for a product feature, improving output quality, reducing hallucinations, or building a prompt versioning strategy.

Assess an AI feature or product for ethical risks, bias, safety issues, fairness gaps, and regulatory compliance. Use when reviewing an AI feature before launch, conducting a responsible AI audit, or responding to a bias or safety concern.