Python Performance Optimization

Profiling and optimization patterns for Python code.

Table of Contents

1. Quick Start

Quick Start

# Basic timing
import timeit
time = timeit.timeit("sum(range(1000000))", number=100)
print(f"Average: {time/100:.6f}s")

Verification: Run the command with --help flag to verify availability.

When To Use

• Identifying performance bottlenecks
• Reducing application latency
• Optimizing CPU-intensive operations
• Reducing memory consumption
• Profiling production applications
• Improving database query performance

When NOT To Use

• Async concurrency - use python-async instead
• CPU/GPU system monitoring - use conservation:cpu-gpu-performance
• Async concurrency - use python-async instead
• CPU/GPU system monitoring - use conservation:cpu-gpu-performance

Modules

This skill is organized into focused modules for progressive loading:

profiling-tools

CPU profiling with cProfile, line profiling, memory profiling, and production profiling with py-spy. Essential for identifying where your code spends time and memory.

optimization-patterns

Ten proven optimization patterns including list comprehensions, generators, caching, string concatenation, data structures, NumPy, multiprocessing, and database operations.

memory-management

Memory optimization techniques including leak tracking with tracemalloc and weak references for caches. Depends on profiling-tools.

benchmarking-tools

Benchmarking tools including custom decorators and pytest-benchmark for verifying performance improvements.

best-practices

Best practices, common pitfalls, and exit criteria for performance optimization work. Synthesizes guidance from profiling-tools and optimization-patterns.

Exit Criteria

• Profiled code to identify bottlenecks
• Applied appropriate optimization patterns
• Verified improvements with benchmarks
• Memory usage acceptable
• No performance regressions

Troubleshooting

Common Issues

Command not found Ensure all dependencies are installed and in PATH

Permission errors Check file permissions and run with appropriate privileges

Unexpected behavior Enable verbose logging with --verbose flag