LLM VRAM calculator
Estimate the weight-only memory for a local language model, then compare it with one GPU’s VRAM. The result excludes KV cache, working buffers and runtime overhead. It helps rule out an undersized setup; it cannot approve a complete inference workload.
Start with the weights.
Choose a model-size scenario, precision and one GPU’s memory capacity. The result is the space needed for weights alone, before running the model.
There is room above the weight floor. Verify the full run.
The remaining number is not free memory measured on your PC. Metadata, unquantized tensors, the KV cache, compute buffers, the runtime and other applications still need space.
8B × 4 bits · 8 GiB nominal capacity · No inference speed or model-fit guarantee.
LLM weight memory at 4, 8 and 16 bits
Compare the five model-size scenarios before changing the calculator inputs. Each value is a theoretical weight-storage floor for a rounded total parameter count at a uniform precision. It is not the VRAM required to run a model.
| Total parameters | 4-bit weights | 8-bit weights | 16-bit weights |
|---|---|---|---|
| 3B | 1.40 GiB | 2.79 GiB | 5.59 GiB |
| 8B | 3.73 GiB | 7.45 GiB | 14.90 GiB |
| 14B | 6.52 GiB | 13.04 GiB | 26.08 GiB |
| 32B | 14.90 GiB | 29.80 GiB | 59.60 GiB |
| 70B | 32.60 GiB | 65.19 GiB | 130.39 GiB |
Weights only. B means one billion parameters; GiB means 1,073,741,824 bytes. Actual formats may add quantization metadata or keep some tensors in higher precision. KV cache, working buffers, runtime and other VRAM use are excluded. A table value below your GPU’s capacity is not a guarantee that the full workload fits.
Compare one scenario with your GPU, then check the full workload separately. See the formula and assumptions.
Similar numbers. Different next moves.
These examples are calculated from the same formula as the tool. They are not measurements, recommendations for a named model, or GPU performance tests.
8 GiB GPU capacity
- Weights alone
- 3.73 GiB
- Before-overhead remainder
- 4.27 GiB
There is a margin above the weight floor. Check the actual model file and your intended context length before interpreting that margin as usable capacity.
12 GiB GPU capacity
- Weights alone
- 13.04 GiB
- Shortfall
- 1.04 GiB
The weights alone exceed the selected GPU capacity. Adding system RAM does not turn it into more dedicated VRAM. Investigate a smaller scenario or an explicitly supported offload path.
16 GiB GPU capacity
- Weights alone
- 14.90 GiB
- Before-overhead remainder
- 1.10 GiB
Only a small margin remains before overhead. A figure below 16 GiB is not evidence that a 16 GiB GPU can run this scenario at your chosen context length.
To see how memory capacity changes a hardware decision, compare the exact MSI RTX 5060 Ti 8GB and 16GB boards. Keep rendering speed, physical fit and runtime support in the same decision.
Count the weights. Keep the rest visible.
Weight GiB = billions of parameters × 1,000,000,000 × bits ÷ 8 ÷ 1,073,741,824
“B” means one billion parameters in decimal notation. A GiB is 2³⁰ bytes. For an 8B, 4-bit scenario, 8 billion × 4 ÷ 8 gives 4 billion bytes, or about 3.73 GiB.
What the inputs mean
- Count total parameters, including inactive experts in a mixture-of-experts model. The active parameter count per token is not the total stored weight count.
- The 3B, 8B, 14B, 32B and 70B choices are rounded planning scenarios. A specific model may have a different exact count.
- The bit setting assumes every parameter uses that precision. Actual quantization formats may retain higher-precision tensors and add scales, metadata or padding.
- Capacity is for one GPU in GiB. This tool does not combine multiple GPUs or add system RAM to dedicated VRAM. Installed capacity is not currently free memory.
Four checks the weight number cannot do.
Choose the exact model file and runtime.
Record the model version, quantization format and application. For supported Transformers or timm models, Hugging Face Accelerate’s model memory estimator can inspect a specific model. Its own caveats distinguish loading memory from inference memory.
Set the context you will actually use.
Longer context can increase runtime memory use. The KV cache depends on the model architecture, cache precision and concurrent sequences. This calculator leaves it unestimated rather than applying one universal allowance. Ollama documents context configuration and checking the allocated context with
ollama ps.Check the exact GPU, OS and driver.
Enough memory does not establish backend support. Verify the card and operating system against the application’s current requirements. Ollama’s hardware support page separates GPU families and operating-system paths; support may differ between them.
Observe a representative run.
Use the intended prompt size, context and concurrency. Check actual memory use, whether the runtime offloads work to the CPU, and response speed. Also leave system RAM and storage for the OS, runtime and model files. Training, fine-tuning and image generation need separate assessments.
Bring the question to your build.
Open the PC builder and add a CPU and memory, then select a graphics card if you plan to use one. In The build review, choose Local language models and expand Adjust model size, precision and context. The AI & YOUR BUILD section compares the scenario with your recorded components.
The standalone calculator does not edit your parts list. Re-enter your chosen scenario in the build review; your existing draft is restored when browser storage is available.
Check my parts list