Models
Specific checkpoints with sourced architecture and weight files.
| Model | Parameters | Attention | Support | Q4_K_M or BF16 @ 8k |
|---|---|---|---|---|
| Qwen2.5-7B-Instruct | 7.61B | gqa | full | 4.92 GiB |
| Qwen2.5-0.5B-Instruct | 0.49B | gqa | full | 1.08 GiB |
| Qwen2.5-1.5B-Instruct | 1.54B | gqa | full | 3.16 GiB |
| Qwen2.5-3B-Instruct | 3.09B | gqa | full | 2.37 GiB |
| Qwen2.5-14B-Instruct | 14.70B | gqa | full | 10.00 GiB |
| Qwen2.5-32B-Instruct | 32.50B | gqa | full | 20.61 GiB |
| Qwen2.5-72B-Instruct | 72.70B | gqa | full | 43.61 GiB |
| Qwen3-1.7B | 1.70B | gqa | full | 4.72 GiB |
| Qwen3-4B | 4.00B | gqa | full | 3.58 GiB |
| Qwen3-8B | 8.20B | gqa | full | 5.93 GiB |
| Qwen3-14B | 14.80B | gqa | full | 9.76 GiB |
| Qwen3-32B | 32.80B | gqa | full | 20.53 GiB |
| Qwen3-30B-A3B | 30.50B | gqa | full | 18.16 GiB |
| Phi-3.5-mini-instruct | 3.82B | mha | full | 10.18 GiB |
| Mistral-7B-Instruct-v0.3 | 7.25B | gqa | full | 14.56 GiB |
| Mixtral-8x7B-Instruct-v0.1 | 46.70B | gqa | full | 88.05 GiB |
| DeepSeek-R1-Distill-Llama-8B | 8.03B | gqa | full | 16.02 GiB |
| DeepSeek-R1-Distill-Qwen-32B | 32.50B | gqa | full | 63.09 GiB |
| DeepSeek-V2-Lite-Chat | 15.70B | mla | unsupported | 29.26 GiB |