Self-hosting an LLM vs. calling an API: the numbers

The equation, in one line

Every comparison between renting a GPU and calling an API reduces to a single division:

Break-even volume (tokens per month) = (hourly price of the machine × 730) ÷ (price per token you would otherwise pay)

730 is the number of hours in an average month. Everything else in this article is either one side of that division, or a cost that the division quietly hides. The reason the answer surprises people is not that the arithmetic is hard — it is that the two sides are billed in fundamentally different units. A server bills time. An API bills consumption. You only come out ahead on the time-billed side if you fill the time.

The price side is public and exact

Start with the hardware. A public-cloud instance built around a single NVIDIA H100 sits at roughly €2.80 per hour on a European provider at list price — we re-checked that order of magnitude in August 2026, and it is worth re-checking yourself, because GPU pricing moves and varies by region, by commitment, and by whether storage and egress are bundled. Treat it as a starting figure, not a quote.

Run that continuously and you get ≈ €2,044 per month (€2.80 × 730). That number does not move when your traffic moves. It is the same on a launch week and on a quiet August.

Now the other side. Take Llama 3.3 70B, a model we serve on sovereign EU infrastructure: €1.26 per million tokens, input and output alike — the full price you would actually be billed, not an infrastructure cost. Every price in our model catalog is published the same way, per million tokens in and out.

Divide: €2,044 ÷ €1.26 ≈ 1,622 million tokens. Roughly 1.6 billion tokens a month before one rented H100 starts winning on price alone. If you need two cards — and for a 70B model you probably do, see the next section — the threshold doubles to about 3.2 billion tokens a month.

Scenario	Machine cost / month	Break-even vs €1.26/M tokens
1 × H100, continuous	≈ €2,044	≈ 1.6 billion tokens
2 × H100, continuous	≈ €4,088	≈ 3.2 billion tokens
2 × H100 + a second box for redundancy	≈ €8,176	≈ 6.5 billion tokens

For scale: a prepaid €20 top-up buys about 16 million tokens on that same 70B model, or roughly 357 million on the cheapest model in the catalog. Most teams discover their real monthly volume is two or three orders of magnitude below the break-even.

Why “one H100” is usually the wrong unit

Back-of-envelope estimates go wrong at the same place almost every time: they price one card and then deploy on two. The constraint is memory, and it binds before throughput does.

A 70-billion-parameter model in FP16 needs about 140 GB just for weights — two bytes per parameter. An H100 has 80 GB of HBM. Quantise to FP8 and the weights fall to roughly 70 GB, which technically fits on one card and leaves almost nothing for the KV cache — the per-request memory that grows with context length and batch size, and that is precisely what you need in order to serve many users at once. In practice, a 70B model served with usable context and batching wants two cards.

So before comparing anything, size the machine by memory first and throughput second. Then price that machine. The same arithmetic is why a 158B-parameter model needs roughly 160 GB in FP8 and lands on two H200s or four H100s, while a 2.8-trillion-parameter model is not a single-node conversation at all.

The throughput side is yours to measure

Turn the break-even from tokens per month into tokens per second and the picture sharpens. There are 2,628,000 seconds in a 730-hour month, so:

• 1.6 billion tokens a month ≈ 617 tokens per second, sustained, day and night
• 3.2 billion tokens a month ≈ 1,234 tokens per second, sustained, day and night

We are deliberately not going to hand you a tokens-per-second figure for your model on your hardware. It depends on quantisation, batch size, context length, the mix of prompt and completion, and the serving stack — and a number copied from someone else's blog post is not a number you can plan a budget on. Measure your own, under your own load.

What the arithmetic does tell you, whatever your measurement turns out to be, is that the break-even sits close to the machine's saturation point. That reframes the question entirely. It is not “is a GPU cheaper per token than an API” — at full saturation, it usually is. It is “can I keep this GPU busy every hour I pay for it”. For most teams, the honest answer is no, and that answer is what actually decides the cost.

Utilisation is what actually decides it

Here is the asymmetry in one comparison. Run an internal tool used during business hours — eight hours a day, five days a week — and you have bought 730 hours and used 176 of them. That is 24% utilisation, which means your real cost per delivered token is roughly four times the number you computed at full saturation. The API side of the equation does not move at all: you consumed what you consumed.

Peaks make it worse rather than better. A server has to be sized for your busiest hour, and you pay that size during every quiet one. An elastic per-token API inverts that: the peak costs more precisely when it happens, and nothing when it does not.

What the hourly rate doesn't include

The €2.80 buys you a machine, not a service. Before you compare it to a price that includes an SLA, add:

• Storage and egress, usually billed separately, and model weights are tens to hundreds of gigabytes to move around.
• Redundancy. One box is a single point of failure. Two boxes is twice the bill for the same tokens.
• Engineering time. Deployment, the serving stack, driver and CUDA upgrades, monitoring, and someone reachable when it stops at 3am. This is the cost that never appears in the comparison and is frequently the largest one.
• Model diversity. A box serves the model it has loaded. If your product routes between a cheap model for classification, a coder model for diffs and a large one for reasoning, that is several deployments — or constant reloading, which destroys the utilisation you were counting on. An API catalog gives you all of them behind one endpoint and one key.
• Version churn. Open weights move fast. Every upgrade is a redeployment and a fresh round of validation on infrastructure you own.

When self-hosting genuinely wins

None of the above says never. It says: for specific reasons, at specific volumes. Self-hosting is the right call when

• you have sustained, saturating volume on a single model — the one case where the division comes out clearly in your favour;
• you run custom or fine-tuned weights that no provider serves;
• the model you need is not in any managed catalog at all — increasingly common for brand-new frontier weights;
• you have a requirement no provider meets, whether that is a hard jurisdictional constraint or a latency and locality target.

What we do ourselves, and why

We run this same arithmetic, and it does not always come out on the side you would expect a gateway to argue for.

For mature models, we do not own hardware: they run on managed sovereign clouds — providers with no non-EU capital control, which is what makes them usable for regulated work. For brand-new frontier models that no sovereign managed catalog serves yet, we do rent our own dedicated EU servers, because there is no alternative that keeps the jurisdiction guarantee. And before committing to a machine that bills 24/7, we sometimes serve a model through a per-token US endpoint as a demand filter — labeled “fast access”, never sovereign, because the company operating it is American and therefore reachable under the CLOUD Act regardless of the region it runs in. That distinction is a legal one, not a geographic one.

The point is that we buy the machine when the volume justifies it, not because self-hosting is virtuous. That is the same test we are handing you here.

How to decide in fifteen minutes

1. Pull last month's real token volume, input and output separately. Not your forecast — your logs.
2. Multiply by the published price of the model you would use. That is your API bill, exactly.
3. Size the machine you would actually need — memory first, throughput second — and multiply its hourly rate by 730.
4. If the API bill is below the machine, stop. You have your answer and it took four steps.
5. If it is above, divide the break-even by 2,628,000 to get the sustained tokens per second you would have to hold, then ask honestly whether your traffic pattern holds it at 3am on a Sunday.
6. Add the hidden costs to the machine side — redundancy, engineering time, model diversity — before you conclude.

If you land on the API side, the practical next step is small: our model-by-model comparison covers what each one is good for, the catalog lists every price and hosting jurisdiction, and the API reference shows the two lines you actually have to change. There is no subscription — a prepaid balance from €20, which stops at zero.

FAQ