NVIDIA DGX Spark (GB10, 128 GB unified)

Estimate

Likely fits

Under the current assumptions, the high estimate stays below 90% of the available budget.

Estimated memory range: 4.92 GiB – 5.42 GiB

Weights use selected file metadata. Runtime extras are still estimated.

Available budget: 112.00 GiB · Remaining after estimate: 106.58 GiB – 107.08 GiB

Software compatibility

Documented support. Enough memory does not mean this engine, quant, and OS will run. Support does not mean it will be fast.

Assumptions that affect this result

• Full-resident weight estimates use total parameters, including inactive MoE experts.
• Weight bytes come from the selected weight files only, not from every file in the repository.
• Download file size is not peak GPU memory. Runtime layout, allocator padding, and KV are extra.
• Counted files: qwen2.5-7b-instruct-q4_k_m-00001-of-00002.gguf, qwen2.5-7b-instruct-q4_k_m-00002-of-00002.gguf.
• Context budget is the number of cached tokens per sequence (prompt plus reserved generation).
• Concurrent sequences share one model replica; weights are not multiplied by sequence count.
• KV dtype is independent of weight quantization unless you change it.
• Using 4 KV heads, not 28 query heads.
• Prefill peak and steady decode occupancy are not the same. The high scenario is closer to prefill/workspace pressure.
• Engine reserve is not the same as bytes the model actually uses for weights and KV.
• llama.cpp compute buffers grow with context and batch. Values here are user-adjustable scenarios, not measured traces — unless a matching calibration is shown.
• Unified memory is a shared CPU/GPU pool. Host RAM is not added on top of this capacity.

Calculation method