Write and review ZeroGPU code for Hugging Face Spaces

Hugging Face ZeroGPU is a development guide for Gradio SDK Spaces running on ZeroGPU hardware. It explains GPU scheduling through @spaces.GPU, model loading, request duration, process boundaries, concurrency, example caching, and dependency compatibility. Docker and Static Spaces are outside its scheduling scope; the source also places Streamlit applications in the Docker category. General Gradio component and layout guidance belongs to a separate skill.

Models should load eagerly at module scope, while actual CUDA computation belongs inside a decorated function. On ZeroGPU, importing spaces patches torch so CUDA availability appears true and module-scope device transfers register tensors for managed migration. That does not mean a real GPU is attached during startup. Weights are packed to disk, then streamed through pinned memory into a GPU worker when a request runs. Warm workers may retain them, but module-scope inference or warmup does not establish reusable GPU computation. A device-selection idiom still supports local CPU environments.

Use this skill when writing or reviewing ZeroGPU handlers, diagnosing quota or serialization failures, choosing duration estimates, or configuring Python and CUDA-dependent packages. It is especially relevant when a demo works for one user but fails under parallel requests.

Do not apply its runtime assumptions to other Space types. Do not infer the current backing GPU, VRAM, supported runtime versions, or tier limits from older examples; the source directs those checks to current official documentation. The torch.compile interface is unsupported, with PyTorch ahead-of-time compilation as the alternative. Local development should install and import the actual spaces package rather than recreating its decorator in a fallback: outside ZeroGPU, its GPU decoration is already a passthrough.

Inputs include Gradio handlers, model-loading code, workload sizes, dependency manifests, and Space configuration. The intended output is code and configuration that respect the scheduler and process boundary. The source supplies this local command for reproducing its documented lazy example-caching settings:

GRADIO_CACHE_EXAMPLES=true GRADIO_CACHE_MODE=lazy python app.py

Duration is a requested reservation, not measured execution time. A visitor's per-call cap can reject an excessive request as illegal duration; insufficient remaining quota or a run-count cap produces quota exceeded. Waiting does not fix an excessive per-call duration. The documented default request is 60 seconds, so a short task can be rejected despite having enough quota for its actual runtime. Use the smallest realistic duration or a callable estimator. Shorter declarations rank higher in the node-level queue. The large size uses half the backing card; xlarge uses the full card and doubles the quota request. Quota accounting uses a fixed 24-hour window from first use with tier-dependent overflow credits; verify current account terms before planning capacity.

Decorated functions execute in separate worker processes. Arguments and returns must be picklable; connections, open handles, locks, and unpicklable closures cannot cross that boundary. Return CPU tensors, arrays, images, or lightweight objects rather than CUDA tensors. The gr.State object is copied through pickle on each yield: in-place changes are invisible until explicitly returned, while gr.update() skips a state update. Minimize repeated serialization of large state and move embedded CUDA data to CPU.

Handlers run concurrently by default. Keep request data out of mutable globals, use unique temporary output paths, and retain only read-only shared models, tokenizers, and configuration. Decorate the outer operation that owns a loop to avoid repeated queue entry, warmup, and serialization; separate substantial CPU preprocessing when charging it as GPU time would be wasteful. Workers can be reused or reclaimed by the scheduler, so persistence is not guaranteed.

The guide covers Gradio, spaces, PyTorch, Transformers, and CUDA-dependent packages. ZeroGPU builds lack nvcc, so CUDA extensions need compatible prebuilt wheels, externally built wheels, or a kernels-community alternative. Runtime CUDA compilation is a distinct capability used by ahead-of-time workflows. Check wheel CUDA, torch, C++ ABI, and Python tags; pin matching torch and Python versions.

Include spaces in a local dependency model while omitting a conflicting pin from the exported requirements.txt because the platform manages that package. For uv, declare it in pyproject.toml to resolve transitive constraints, then exclude it during export. Check torchvision, torchaudio, torchcodec, and other side-cars against the pinned torch release after upgrades. Missing PyPI dependency metadata is not proof of compatibility; consult the package's compatibility table and pin the matching version.

Example caching defaults to lazy on ZeroGPU because startup lacks a GPU. When caching is enabled, click-only controls may be ignored; disable caching explicitly when that interaction is required. The accompanying references expand concurrency, worker lifecycle, quota behavior, and CUDA dependency checks.