Benchmark System Performance Accurately

Provides expert guidance for performance benchmarking, load testing, and system performance analysis - designing statistically valid benchmark strategies, implementing tests across databases/APIs, and producing actionable optimization recommendations.

Use this skill when designing a load test strategy, benchmarking database or API latency and throughput, monitoring system resources during a benchmark run, or establishing a baseline-to-optimization workflow with regression detection. Not a fit for functional/correctness testing unrelated to performance, or for production benchmarking without proper safeguards.

Core principles cover statistical validity (multiple iterations, warm-up periods to account for JIT/caching, reporting P50/P95/P99 alongside averages), realistic test conditions (mirroring production, realistic data volumes, synthetic and real-world workloads), and comprehensive metrics (latency, throughput, resource utilization, error rates, system-level CPU/memory/I/O/network, and user-experience metrics like TTFB).

Provides an Artillery YAML load test configuration with warm-up/sustained/peak load phases, a weighted scenario chaining a list request (capturing a user ID) into a detail request with status code and property assertions. Provides a Python DatabaseBenchmark class executing a query with a warm-up phase followed by timed iterations, computing average/P50/P95/P99 latency and error rate, plus a concurrent load test using ThreadPoolExecutor to drive requests for a fixed duration.

Provides a JavaScript APIBenchmark class using axios and perf_hooks, running warm-up iterations then concurrency-controlled benchmark iterations, tracking per-request duration/status/response size/memory delta, and computing aggregate statistics (error rate, average/P50/P95/P99 latency, throughput in requests/sec, average response size). Provides a bash system monitoring script sampling CPU usage, memory usage/percentage, disk I/O, network I/O, and load average at a configurable interval into a timestamped CSV file.

Best practices cover test environment setup (isolating the benchmark environment, dedicated hardware/containers with resource limits, disabling background processes, documenting environment specs), data collection/analysis (baseline measurements before optimization, correlation analysis for bottlenecks, automated regression detection, historical trend storage), and pitfalls to avoid (relying solely on averages instead of percentiles, including cold-start measurements in final results, ignoring resource constraints, benchmarking production without safeguards). Defines a six-step optimization workflow: establish baseline, identify bottlenecks via profiling, make targeted optimizations, re-run benchmarks to validate, monitor for regressions, and document changes with their performance impact.

Uses Artillery for HTTP load testing, psycopg2 with ThreadPoolExecutor for PostgreSQL benchmarking, axios/perf_hooks for API benchmarking in Node.js, and standard Linux tools (top, free, iostat, /proc/net/dev, uptime) for system resource monitoring.

Performance and platform engineers designing or running benchmarks for APIs, databases, or systems who need statistically valid test implementations and concrete metrics collection rather than ad hoc single-run timing measurements.

def execute_query(self, query, params=None, iterations=100):
    latencies = []