dolphin-2.9.1-yi-1.5-34b

dolphin-2.9.1-yi-1.5-34b by dphn: a 34.4B-parameter dense open-weight model under the apache-2.0 license with tool calling.

apache-2.0Dense8K contextTool callingAuto-imported · not yet reviewed

4,820,992 downloads · 67 likes · Model card · synced 2026-09-19

Specifications

Parameters: 34.4B
Architecture: dense · llama
Layers: 60
Hidden size: 7,168
KV heads · head dim: 8 · 128
Vocabulary: 64,000
Native dtype: bfloat16
Context window: 8,192 tokens
Released: 2024-05-18
Last modified on Hub: 2025-09-08

Features & licensing

License: apache-2.0
Commercial use: Yes
Modalities: text
Tool / function calling: Yes
Reasoning mode: No
Languages: EN
Quantised variants on Hub: AWQ, EXL2, GGUF, bitsandbytes
Pipeline: text-generation

transformerssafetensorsllamatext-generationgenerated_from_traineraxolotlconversationaltext-generation-inferenceendpoints_compatible

Task fit

Editorial scores (0–100) used by the recommendation engine.

Chat70
RAG / Q&A70
Code70
Summarisation70
Extraction70
Agents70

Hardware to run dolphin-2.9.1-yi-1.5-34b

Weights need about 76 GB at FP16, 40 GB at INT8 and 23 GB at INT4. The KV cache adds roughly 246 MB per 1,000 tokens per request. For a reference workload of 5 requests per second with 1,500 input and 300 output tokens, total GPU memory is around 36.8 GB, which fits on 1 × NVIDIA A100 (80 GB).

GPU configurations at the reference workload
Configuration	VRAM	Utilisation	Est. first token	Cloud / month
2 × NVIDIA L4 (24 GB)	48 GB	77 %	~0.5 s	$1,168
1 × NVIDIA L40S (48 GB)	48 GB	77 %	~0.2 s	$1,387
1 × NVIDIA A100 (80 GB)recommended	80 GB	46 %	~0.1 s	$2,336
1 × NVIDIA H100 (80 GB)	80 GB	46 %	~0.0 s	$3,285

Estimates only. Run an assessment for your own traffic.

Auto-imported from Hugging Face. Architecture, license and feature data are synced from the model repository; the task-fit scores below are provisional defaults until a curator reviews them.