Picchio is a single-file, stdlib-only Python tool that tells you whether a local LLM is actually running on the GPU, or has silently fallen back to the CPU. Run with no arguments, it discovers Ollama tags, local GGUF files, and models cached by Hugging Face or LM Studio, picks one, and runs three measurement passes; you can also point it directly at a model file, an Ollama tag, or a running llama-server URL. In the project's own demo, it catches a 66-layer model with 28 layers stuck on the CPU and reports the resulting 14x decode slowdown.

Use it when a local model feels slower than it should and you need to know whether that's CPU fallback, partial GPU offload, or just a genuinely slow setup - or before downloading a model at all, to check whether it fits in your machine's memory at your intended context length and KV cache type. picchio guard watches something already running - your own command, a loaded model, or a server - and tells you the moment it leaves the GPU; picchio compare diffs two runs and names the first setting that changed between them. It is not a benchmarking suite for comparing model quality or accuracy - it only measures GPU placement, throughput, and power or energy per token.

./picchio MODEL                              # catch CPU fallback and measure the run
./picchio plan MODEL --ctx 262144 --kv q8_0  # check whether it fits at this context and KV type

Output includes the exact number of model layers placed on the GPU, separate prefill, decode, and wall-clock speeds, GPU activity, memory, power, and energy per generated token, and a HEALTHY, CPU FALLBACK, or PARTIAL OFFLOAD verdict. --share bug-report turns one run into a paste-ready GitHub Issue for the Ollama or llama.cpp trackers, and --json gives machine-readable output for scripting.

It reads per-layer placement and timing from llama.cpp, the CPU/GPU weight split from Ollama, Apple GPU activity and power on macOS, NVIDIA GPU stats on Linux and Windows through NVML, and AMD GPU activity through amdgpu sysfs on Linux. Install is one line, with no package manager involved:

curl -fsSL https://raw.githubusercontent.com/logxio/picchio/main/public/picchio.pyz -o picchio

Results from any machine can be submitted to a public comparison page (--share row), which already lists measured prefill, decode, and wall-clock numbers across Apple M5, RTX 4090/5090/4070 SUPER, and CPU-forced runs. It requires Python 3.9+, runs on macOS, Linux, and Windows, and is MIT-licensed.

Anyone running local LLMs through llama.cpp, Ollama, or LM Studio who wants a fast, dependency-free way to confirm the model is actually using the GPU, size a model before downloading it, or get an accurate bug report when it isn't behaving.