Hugging Face Spaces cold starts, explained

What 'sleeping' actually means

A free-tier Hugging Face Space is a container that gets suspended after 48 hours of no incoming traffic. The image and any persistent storage stick around; only the running process and its RAM are torn down. The next request wakes the container back up, which involves rescheduling onto a host, restoring the filesystem, and re-running the entrypoint. This is the classic serverless cold-start pattern applied to long-running ML apps that were never really designed to boot on demand. Understanding what happens in each phase is the fastest path to making Hugging Face Spaces cold starts tolerable — or avoiding them entirely.

Cold start phases and where the seconds go

Scheduling: 1–3 seconds to place the container on a host with capacity
Filesystem restore: 2–10 seconds to hydrate persistent volumes
Container start + Python interpreter: 3–8 seconds
Model loading from disk into RAM/VRAM: 10–90 seconds, dominant for LLM demos
First-token warmup (CUDA kernels, torch.compile, TGI prefill): 2–20 seconds

Why LLM Spaces feel especially slow

For a Gradio app that loads a 500MB image classifier into RAM, the whole cold start might be 10–30 seconds and users tolerate it. For a Docker Space with a big CUDA base image and a 7B model, cold start creeps to 60–120 seconds because weight loading and CUDA kernel compilation dominate. Every megabyte of PyTorch import time and every module resolved at first token adds to the wait. Quantization helps (int4/int8 halves the disk-to-VRAM time), safetensors helps (mmap is faster than pickle), and pre-compiling with `torch.compile` inside the entrypoint hides some CUDA warmup — but none of these eliminates the fundamental problem, which is that big models are big.

How to eliminate cold starts entirely

Upgrade the Space hardware — any paid CPU/GPU tier (CPU upgrade, T4, A10G, A100) runs always-on until you pause or downgrade
Use ZeroGPU (Pro subscription) — shared H100 pool with fast attach, cold start is measured in seconds, not minutes
Move to an Inference Endpoint — dedicated infra, no sleep, per-second billing but zero surprise
Ship a static Space — HTML/JS only, no Python runtime, no cold start at all

ZeroGPU: the middle ground

ZeroGPU is a shared H100 pool for Hugging Face Pro users. You decorate a function with `@spaces.GPU`, and the GPU is attached only for the duration of the call. Between calls the Space runs on CPU and the GPU is released to other users. This makes ZeroGPU dramatically cheaper than a dedicated H100 (which would be $8–10/hr always-on) for demos and low-traffic apps, at the cost of a small per-call attach latency (typically 1–3 seconds). It is the correct answer for community demos, model spotlight pages, and internal review apps where dedicated hardware is overkill.

The warm-ping approach, and why it's fragile

A common hack: run a cron job that hits the Space every 30 minutes to prevent the 48-hour idle from tripping. This works, but it undermines the shared-fairness model the free tier is built on and Hugging Face may throttle or block traffic that looks synthetic. If your Space needs to stay warm continuously it needs to be a paid Space, or a ZeroGPU Space, or an Endpoint — not a cron-warmed free tier. Treat warm-pings as a bridge, not a strategy.

Loader-side optimizations that actually help

Convert PyTorch .bin weights to safetensors — mmap makes load 2–4x faster
Enable torch_dtype=torch.float16 (or bfloat16) to halve weight size in VRAM
Prefer AWQ/GPTQ quantized variants when quality allows — 4x smaller weights, 2x faster load
Lazy-import heavy libraries inside handler functions, not at module top
Warm CUDA kernels with a dummy 1-token generation in the entrypoint

When to give up on Spaces and use an Endpoint

Spaces are optimized for demos, model spotlights, and internal tools. If any of the following is true, cold starts on Spaces will fight you forever and an Inference Endpoint is the better answer: users are outside your org and expect sub-second first-response, the model is above 13B parameters, you need horizontal autoscaling, you need private networking or org-level ACLs, or you have an SLO. Endpoints cost more but eliminate the class of cold-start problems that Spaces will never fully solve.

FAQ

Does upgrading Space hardware actually stop sleep?

Yes. Any paid hardware tier (CPU upgrade, T4, A10G, A100) runs always-on until you downgrade or explicitly pause the Space. You are billed per-second while it runs, which for a T4 works out to roughly $430/month if left on 24/7. Downgrading back to the free CPU tier at any time restores the 48-hour sleep behavior.

Can I pre-warm my Space before a launch?

Send a real HTTP request 30–60 seconds before your launch. There is no explicit warmup API on the free tier, but any inbound request kicks off the cold start. If your Space depends on model loading during the entrypoint, the warmup request is your only signal that boot has completed. For a paid Space, restart it manually via the Settings UI a few minutes before go-live.

How is ZeroGPU different from a dedicated GPU Space?

ZeroGPU shares an H100 pool across many Pro users and attaches the GPU only during a @spaces.GPU call. A dedicated GPU Space rents an entire GPU 24/7 whether or not you use it. ZeroGPU is dramatically cheaper for low-to-medium traffic but adds a small attach latency per call and is unavailable to non-Pro users.

Why is my first request slower than my second, even on paid hardware?

CUDA kernel compilation, torch.compile graph capture, and TGI/vLLM prefill warmup all happen on the first inference call. Even a paid always-on Space that never sleeps will have a slower first request after a redeploy. You can hide this by running a dummy forward pass in your entrypoint before serving user traffic.

Do static Spaces ever cold-start?

Not meaningfully. A static Space is HTML/CSS/JS served from a CDN, so the only latency is CDN edge caching, which is measured in tens of milliseconds. If you can express your demo as a client-side app that calls an inference API separately, a static Space is the fastest and cheapest option.

What's the maximum idle window before free-tier sleep?

48 hours of no incoming traffic. The clock is reset by any HTTP request that reaches the container, including HEAD requests and health probes. If a Space embed on your site is receiving any casual traffic, sleep is rare in practice; if the Space is only linked from a rarely-visited README, expect frequent cold starts.

Does upgrading to Pro help with cold starts on free-tier Spaces?

Only indirectly. Pro unlocks ZeroGPU, which sidesteps the sleep problem for GPU workloads. Pro does not change the sleep behavior of a free-tier CPU Space you own, and it does not accelerate cold-start times on someone else's public Space.

How do I monitor cold-start latency in production?

Log the timestamp at the top of your entrypoint and again after the model finishes loading; the delta is your cold-start budget. For paid Spaces, HF exposes basic metrics in the Space settings UI. For anything beyond that, forward request timings to an external observability tool via a lightweight middleware in your Gradio or FastAPI app.