Sizing for your workload
The hardware tiers describe a team using models interactively. Your workload may be much smaller, and this page says how to size for it.
What the tiers assume
Tier 1 to Tier 3 are sized for people in conversation with a model: coding assistants, agents, question answering. Many conversations are open at once, each with a long context, so memory for the conversations (the context cache) is what runs out first. That is the right shape for a team of 50 to 100 people, and the wrong one for a single automated process.
Workload profiles
| Workload | What drives the hardware | Usual shape | Where to start |
|---|---|---|---|
| Team assistants and coding agents | Many open conversations, long context | 50 to 100 or more people, 32,000 tokens or more of context each | Tier 1 and up. Size against measured session length. |
| One pipeline or batch job: matching, classification, extraction | The smallest model that passes your task. One or a few requests at a time, short prompts | One process, no waiting person | An entry configuration (below) or Tier 0. Run Compare first. |
| Questions over your own documents | A mid-size model, moderate context, plus a small embedding model and a reranker | A few to dozens of people | A single 32GB to 96GB card |
| Embedding and reranking only | Throughput, not memory | Batch indexing, search | A card with a few gigabytes free, often shared with another workload |
The tier table on the hardware page is the right place to start for the first row. For a single pipeline, the model matters more than the machine.
Three numbers decide the hardware
- Memory for the weights. The model's parameters times the bytes each one takes. A 27 billion parameter model takes about 27GB at 8 bits and about 14GB at 4 bits. For a mixture model, memory follows the total parameters, not the active ones.
- Memory for the conversations. Each open conversation needs a context cache that grows with its length. One conversation of 32,000 tokens needs a few gigabytes on a large model. One short request at a time needs almost none.
- Speed. Generating each token means reading the active weights from memory, so the rate is bounded by memory bandwidth divided by the bytes read per token. A mixture model reads only its active experts, so it can be fast even when it is large.
These are estimates, not measurements. The upper bounds below ignore everything that slows a real server, and real rates are lower.
| Model shape | Memory for weights | Upper bound, one stream, Tier 1 card (about 1,792GB/s) | Upper bound, Tier 0 unit (about 273GB/s) |
|---|---|---|---|
| 8B dense, 8 bits | about 8GB | about 220 tokens per second | about 34 |
| 27B dense, 8 bits | about 27GB | about 66 | about 10 |
| 35B mixture with 3B active, 8 bits | about 35GB resident | about 600 | about 90 |
| Very large mixture, native 8 bits, 328GB of weights | about 328GB resident | Tier 3 only | Does not fit |
The last row is why two models with similar names can need very different machines: the memory a model needs follows its size, and a name such as "flash" describes its speed, not its size.
Some communities serve very large mixture models on small hardware by keeping rarely used experts in system memory or on disk, at the cost of speed. Heliast does not yet support those serving engines, so do not plan around them.
A single pipeline, in practice
When one process asks a model a short question over and over, the hardware question is almost entirely the model question.
- Run your task in Compare across a few small models. Pick the smallest one that passes.
- Size for one stream: the weights, a few gigabytes for context, and some headroom.
- If the weights fit in about 24GB to 32GB, a single professional card in an ordinary workstation or server is enough. A Tier 0 unit also works, with one stream and slower speed.
Indicative figures for entry cards, from public listings in October 2026, each to be re-quoted before you buy: the RTX PRO 4000 Blackwell (24GB) at about US$2,300 to 3,100 on price trackers, with patchy stock, and the RTX PRO 4500 Blackwell (32GB) listed at NZ$5,999 plus GST by one New Zealand retailer. Neither price includes the rest of the machine. An entry configuration has not been measured on Heliast, so treat its capability as an estimate until a run exists.
Cards Heliast does not accept
Heliast refuses GeForce cards at hardware acceptance, whichever generation. NVIDIA's licence for the GeForce driver bars deploying the driver in a data centre, and Heliast does not ask you to take that exposure. Use RTX PRO or data-centre cards. Previous-generation professional cards bought used, such as the RTX A-series, are in the same product line and are not affected by that licence term. Their age, support status and driver compatibility are still yours to confirm.
What the console shows, and what it does not
A model's verdict page shows how much memory its weights need and which hardware tiers can hold them. That is a first filter. It does not show a mixture model's active parameters, the context memory per conversation, or any speed. Use it to rule hardware out, and use Compare and a sizing conversation to rule it in.
Limits
- No sizing figure on this page has been measured on Heliast hardware.
- Prices are public listings, not quotes, and change quickly.
- Concurrency for your workload is something to measure, not something to read off a tier.