Blog
Practical, vendor-neutral guides on choosing, sizing and running open-weight language models.
📐 How to size GPUs for an open-source LLM: the three numbers that matter
Weights are the easy part. KV cache multiplied by concurrency is what actually decides how many GPUs you need — and it's the number most first estimates leave out.
⚖️ Open LLM licenses explained: Apache-2.0, MIT, Llama, Gemma and the research-only crowd
Not every open-weight model is open for commercial use in the same way. Here is what the common licenses allow, what they restrict, and what procurement should check.
🧩 Dense vs mixture-of-experts: what it means for your hardware bill
MoE models activate a fraction of their parameters per token. That changes compute and latency — but not memory, which is where naive sizing goes wrong.
⚙️ vLLM, SGLang, TensorRT-LLM or llama.cpp: choosing an inference server
The model is half the decision. The server decides your throughput, your latency profile and how much of your GPU you actually use.
🏛️ On-premises vs private cloud for regulated LLM deployments
Banks, hospitals and governments all ask the same question. The answer depends on three things: data classification, who holds the keys, and how fast you need to scale.
🗜️ Quantization in practice: AWQ, GPTQ, GGUF and FP8
Quantized weights are how most open models reach production. Here is what each format is for, what it costs in quality, and how to verify it on your own data.