Skip to content
NEWSee what changed →

unsloth · llama · based on meta-llama/Llama-3.2-1B-Instruct

Llama-3.2-1B-Instruct

Llama-3.2-1B-Instruct by unsloth: a 1.2B-parameter dense open-weight model under the llama3.2 license with tool calling.

llama3.2Dense131K contextTool callingAuto-imported · not yet reviewed

1,045,136 downloads · 112 likes · Model card · synced 2026-11-19

Specifications

Parameters
1.2B
Architecture
dense · llama
Layers
16
Hidden size
2,048
KV heads · head dim
8 · 64
Vocabulary
128,256
Native dtype
bfloat16
Context window
131,072 tokens
Released
2024-09-25
Last modified on Hub
2025-05-09

Features & licensing

License
llama3.2
Commercial use
Yes
Modalities
text
Tool / function calling
Yes
Reasoning mode
No
Languages
EN
Quantised variants on Hub
AWQ, EXL2, FP8, GGUF, GPTQ, bitsandbytes
Pipeline
text-generation
transformerssafetensorsllamatext-generationllama-3metafacebookunslothconversationalentext-generation-inferenceendpoints_compatible

Task fit

Editorial scores (0–100) used by the recommendation engine.

  • Chat70
  • RAG / Q&A70
  • Code70
  • Summarisation70
  • Extraction70
  • Agents70

Hardware to run Llama-3.2-1B-Instruct

Weights need about 3 GB at FP16, 1 GB at INT8 and 1 GB at INT4. The KV cache adds roughly 33 MB per 1,000 tokens per request. For a reference workload of 5 requests per second with 1,500 input and 300 output tokens, total GPU memory is around 1 GB, which fits on 1 × NVIDIA A100 (80 GB).

GPU configurations at the reference workload
ConfigurationVRAMUtilisationEst. first tokenCloud / month
1 × NVIDIA L4 (24 GB)24 GB4 %~0.5 s$584
1 × NVIDIA L40S (48 GB)48 GB2 %~0.2 s$1,387
1 × NVIDIA A100 (80 GB)recommended80 GB1 %~0.1 s$2,336
1 × NVIDIA H100 (80 GB)80 GB1 %~0.0 s$3,285

Estimates only. Run an assessment for your own traffic.

Auto-imported from Hugging Face. Architecture, license and feature data are synced from the model repository; the task-fit scores below are provisional defaults until a curator reviews them.