Nimbus 7B — parameters, VRAM requirements and local deployment

Nimbus 7B is a 7.2B instruction-tuned text model released under Apache 2.0, with an 8K context window. What makes it worth a page is not the benchmark row — it is that the Q4 build stays coherent on structured output, which is where most 7B models quietly fall apart.

Everything below was run on our own hardware. Numbers carried over from the model card or from other people’s posts are labelled as such.

What it is good at

Structured extraction, rewriting, classification with a fixed label set, and short tool-calling loops. In our tests it held a JSON schema for 40 consecutive requests without a formatting failure, which is unusual at this size.

It is not good at arithmetic beyond two steps, and it does not have the world knowledge to answer factual questions you would normally ask a large hosted model. Treat it as a text transformer, not an assistant.

Running it locally

The one-line path is Ollama. It downloads a 4.4GB Q4_K_M build and starts an interactive session; the first token arrives in about 240ms on a 12GB card.

For anything serving more than one client, use vLLM instead. Same weights, roughly three times the throughput, and it respects the 8K context without you having to fight the template.

The two errors you will hit first

The model ignores your JSON schema. This is almost always Ollama’s default chat template, which drops the tool block. Switch to --chat-template with the template shipped in the repo, or move to vLLM.

Output gets cut mid-sentence. Context was silently capped at 2K by the client. Set num_ctx explicitly; a 7B model has no way to tell you it never saw the end of your prompt.

Is it worth the download

If you want a local model that reliably turns messy text into clean structure, yes — this is the smallest one we have tested that does not need a retry loop. If you want a general assistant, the 6.4GB of VRAM buys you less capability than a hosted model on the same budget.

For the images half of a local pipeline, our Aurora Canvas review covers the browser route and where it stops being enough.

ollama run nimbus:7b

vllm serve northwind/nimbus-7b --max-model-len 8192 --gpu-memory-utilization 0.85

Search "Nimbus 7B" in the model browser, pick Q4_K_M, set context to 8192