Qwen hosting — the open-weight family that runs where the data lives
Qwen hosting means serving Alibaba's open-weight Qwen models on your own infrastructure instead of an API. Qwen3 families range from 0.5B to 235B parameters, served with vLLM or Ollama on GPUs sized to the workload. Pexon deploys Qwen behind a gateway with EU residency and audit logging for teams that want a frontier-class open-weight model without a vendor API.
The strongest open-weight family you can run entirely on your own hardware
Qwen is Alibaba's family of open-weight large language models, from 0.5B up to 235B parameters, with specialised variants for coding (Qwen-Coder), vision (Qwen-VL) and reasoning. It has become one of the most practical open-weight choices for enterprise workloads because the capability gap to frontier APIs has narrowed while the license stays permissive enough to self-host.
Hosting Qwen yourself means the model, the prompts and the outputs never leave your estate. That single property is why teams choose it: the data classification forbids the vendor API, the contract requires on-premise processing, or the token bill has crossed the line where owned hardware is cheaper.
Qwen is one of the strongest self-hostable open-weight families, from 0.5B to 235B parameters, with dedicated coding and vision variants. Hosting it yourself means prompts and outputs never leave your estate.
Choosing the Qwen model size
| Model | Parameters | Hardware | Best for |
|---|---|---|---|
| Qwen3-0.6B | 0.6B | CPU or small GPU | Classification, extraction, high volume |
| Qwen3-7B | 7B | 1× 24 GB GPU | General assistant, RAG, prototyping |
| Qwen3-32B | 32B | 1-2× 48-80 GB GPU | Strong reasoning, production assistant |
| Qwen3-72B | 72B | 2× 80 GB GPU (4-bit) | Frontier-class quality, high-value workloads |
| Qwen3-235B (MoE) | 235B | Multi-GPU cluster | Frontier-class, large-scale serving |
Hardware figures assume 4-bit quantisation for the larger models and are starting points, not quotes. The sizing blueprint measures your prompt lengths, concurrency and context window before the card count is fixed.
The model size decision is a workload decision, not a leaderboard decision. A 7B model that answers your eval set is cheaper to run and easier to operate than a 72B model that answers it slightly better.
The hosting stack, layer by layer
Serving Qwen in production is the same stack as any open-weight deployment, and the layers are not optional. The serving layer — vLLM for throughput, Ollama for simplicity — exposes the model as an OpenAI-compatible API. The gateway in front routes traffic, enforces budgets and logs every request. The access control binds to your existing directory.
The order matters: sizing before procurement, serving before gateway, gateway before users. Skipping a layer is how a deployment becomes a liability with a nice dashboard.
Qwen behind vLLM, behind LiteLLM
The production shape: vLLM serves Qwen, LiteLLM routes and logs.
# vLLM serving Qwen3-32B on your own GPU
vllm serve Qwen/Qwen3-32B-Instruct \
--quantization awq \
--max-model-len 32768 \
--gpu-memory-utilization 0.9
# LiteLLM in front: one API for the estate
model_list:
- model_name: chat-default
litellm_params:
model: vllm/Qwen/Qwen3-32B-Instruct
api_base: http://vllm-qwen:8000/v1EU residency is the reason most teams self-host Qwen — and it is a deployment decision, not a vendor claim. The blueprint verifies the processing location from the configuration, not from a marketing page.
Three questions that decide the Qwen deployment
Each is answered by the two-week measurement, not by a model card.
- Which model size passes your eval set — the 7B-to-72B decision is measured against your own tasks, not against a benchmark leaderboard.
- What the concurrency profile looks like — the number of simultaneous users decides whether vLLM's batching pays for its complexity over Ollama.
- Where the processing happens and who operates it — the runbook, the alerting and the upgrade path are part of the deliverable, not an option.
The model is free to download. The deployment is the product — and the runbook is the part nobody quotes you.
Go deeper on open-weight serving
- open weight model hosting on hardware you control — the money page for hosting Qwen, Nemotron, GLM or DeepSeek
- nemotron — NVIDIA's open-weight family, the usual comparison against Qwen
- Ollama vs. vLLM — Which serving layer fits your Qwen workload.
- GPU inference sizing — The arithmetic behind how many cards a Qwen model actually needs.
Hosting questions
What is Qwen?
Qwen is Alibaba's family of open-weight large language models, covering sizes from 0.5B up to 235B parameters, with specialised variants like Qwen-Coder and Qwen-VL. It is one of the strongest open-weight families, competitive with frontier models on many coding and reasoning tasks, and fully self-hostable.
How do we host Qwen ourselves?
The standard stack is vLLM or Ollama on GPU hardware sized to the model and workload: a 7B model fits on a single 24 GB card, a 70B model needs an 80 GB card at 4-bit or a multi-GPU setup. Behind the serving layer sits a gateway (LiteLLM) for routing, budgets and audit logging. Pexon's two-week blueprint sizes the hardware before you buy.
Is Qwen suitable for EU data residency?
Yes — that is the main reason teams self-host Qwen. Running the model on your own hardware means prompts and outputs never leave your estate, which satisfies data classification and contract requirements that a vendor API cannot. The trade-off is that you operate the infrastructure yourself or with a managed partner.
Next step
Size the Qwen deployment before you buy the hardware
Two-week readiness blueprint at €4,900: we instrument your workload, benchmark the right Qwen model size, size the GPUs and hand back a costed deployment plan with the serving layer wired. The plan is yours to keep whoever builds it.