Optimize Python Code Performance

Covers Python profiling and optimization across four profiling tools - cProfile for CPU profiling with pstats sorting and top-N reporting, line_profiler for line-by-line timing via a @profile decorator and kernprof, memory_profiler for per-line memory usage, and py-spy for attaching to a running production process to generate flamegraphs or dump the current call stack without restarting it - and twenty concrete optimization patterns with before and after benchmarks: list comprehensions and map versus traditional loops, generator expressions for constant memory regardless of input size, efficient string concatenation, dictionary lookups versus linear list searches, local versus global variable access speed, and inlining to avoid function-call overhead. Advanced patterns cover NumPy vectorized operations for numerical work, functools.lru_cache for memoizing recursive algorithms, slots to cut per-instance memory on classes with many instances, multiprocessing.Pool for parallelizing CPU-bound work across processes with a sequential-versus-parallel speedup benchmark, and asyncio with aiohttp for concurrent I/O-bound requests with a synchronous-versus-asynchronous speedup benchmark. Database optimization covers batching database operations instead of row-by-row calls, using indexes and query planning, and selecting only needed columns instead of SELECT *. Memory optimization covers detecting leaks via memory monitoring and forced garbage collection, iterators versus lists for constant-memory file processing, and weakref-based caches that allow garbage collection once no strong references remain. Closes with a custom benchmark decorator, pytest-benchmark for regression-tracked performance tests, a best-practices list, common pitfalls, and a performance checklist.

Use it when profiling slow Python code, choosing between optimization strategies, reducing memory usage, parallelizing CPU-bound work, speeding up I/O-bound operations, or optimizing database access patterns. Skip it for anything unrelated to Python performance or outside this scope.

Input is a performance symptom - a slow function, high memory usage, slow I/O, or a CPU-bound bottleneck - or a profiling need; output is the matching profiling command or optimization pattern with a working before and after benchmark to confirm the improvement.

Built on Python's standard library (cProfile, pstats, functools.lru_cache, multiprocessing, asyncio, gc, weakref) plus three external profilers (line_profiler, memory_profiler, py-spy), NumPy for vectorized numerical work, aiohttp for async HTTP, and pytest-benchmark for regression-tracked performance testing.

Python engineers diagnosing or fixing a performance problem who need concrete profiling commands and benchmarked optimization patterns rather than general performance advice. Also covers identifying the performance bottleneck in the first place, reducing application latency and response times, database query performance, and reducing memory leaks in long-running processes.