Our per-token API bill grew faster than our user base.
A dedicated GPU costs the same on a quiet day and a busy one. You know the monthly number before the month starts.
GPU servers for inference
AI inference hosting on dedicated GPUs
AI inference hosting in India on GPUs reserved for your model. From ₹32,000 a card a month, host included.
Why teams switch
Dedicated GPUs vs per-token APIs
Latency is fine in testing and terrible at peak.
We size the fleet for your peak, tune batching and KV cache on a serving stack such as vLLM, and load-test it with your prompts before go-live.
The model stops responding at 2am and nobody notices until morning.
With AI Production Care we monitor the endpoint, the GPU and the queue around the clock and respond to a P1 in under 15 minutes.
We don’t know which GPU we actually need.
The table below sizes common models by memory. We confirm the fit with a benchmark on your prompts, not a guess.
Model sizing
Which GPU fits your model?
Weights only, ideal: parameters times bytes. Pick the smallest card that holds your model, then leave room for context and concurrent users.
| Model | FP16 | FP8 | 4-bit | Smallest GPU at FP8 | Smallest GPU at 4-bit |
|---|---|---|---|---|---|
| Llama 3.1 8B | 16 GB | 8 GB | 4 GB | L4 24GB ₹32,000 a month | L4 24GB ₹32,000 a month |
| Qwen3.8 27B dense | 54 GB | 27 GB | 13.5 GB | L40S 48GB ₹55,000 a month | L4 24GB ₹32,000 a month |
| Qwen 32B class Qwen 2.5 or Qwen3 32B | 64 GB | 32 GB | 16 GB | L40S 48GB ₹55,000 a month | L4 24GB ₹32,000 a month |
| Llama 3.3 70B | 140 GB | 70 GB | 35 GB | H100 80GB ₹1,80,000 a month · tight | L40S 48GB ₹55,000 a month |
| gpt-oss-120b 117B total, 5.1B active, mixture of experts | Native MXFP4: about 65 GB of weights Fits one 80GB card (H100 80GB, ₹1,80,000 a month, tight) or the RTX PRO 6000 96GB (₹1,10,000 a month). | ||||
Weights only, ideal (params × bytes: FP16 2, FP8 1, 4-bit 0.5), counts rounded. Real files are larger; add KV-cache and runtime headroom. Tight means weights fill over 80% of the card. gpt-oss-120b ships in MXFP4; OpenAI’s model card says it runs on one 80GB GPU. Prices are per card per month before taxes, availability confirmed on your quote. Oct 2026.
More cards: RTX PRO 6000 96GB, ₹1,10,000 · B200 180GB
Serving stack
Private inference, tested on your prompts
We install the server, load-test it on your prompts, and hand you an OpenAI-compatible URL.
vLLM, TGI or Triton
We install and tune the engine that suits your model.
OpenAI-compatible endpoint
Use the OpenAI-compatible endpoint with your existing client. We check model names, parameters and tool calls before you switch.
Data stays in India
Weights and prompts stay on cards reserved for you, in India.
Load-tested before go-live
We replay your prompts and agree the latency target before traffic arrives.
Ways to start
Inference GPU plans and prices
GPU prices are per card per month with a one-month minimum. Prices exclude applicable taxes.
NVIDIA L4 24GB
₹32,000per month
For models up to about 8B parameters, embeddings, speech and computer vision at modest concurrency.
- On request: we confirm availability before you commit
- 24GB memory per card
- Low power, good for steady request rates
- Add cards as traffic grows
AI Production Care
₹75,000per workload a month
We deploy the serving stack, benchmark it on your prompts and own the endpoint in production. The GPU card is a separate line.
- Serving stack set up and load-tested
- Endpoint, GPU and queue monitored 24/7; P1 response under 15 minutes
- Scaling and model updates handled with you
NVIDIA H100 80GB
₹1,80,000per month
For quantized 70B models at a validated context and concurrency. Longer contexts or more users move to H200 or more cards.
- 80GB HBM memory per card
- Batching raises throughput when many requests are in flight
- H200 141GB available for larger contexts
AI inference questions, answered
What is an AI inference server?
A server with one or more GPUs that runs a trained model to answer requests behind an API. We rent you the GPU server by the month and can deploy and run the serving stack on it.
Which GPU should I use for AI inference?
Match memory to the model. An L4 (₹32,000) suits small models and an L40S (₹55,000) mid-sized ones. An H100 (₹1,80,000) runs a quantized 70B at a tested context. Longer contexts or more users move to an H200 (₹2,20,000). The sizing table has the arithmetic.
Can I host gpt-oss-120b on one GPU?
Yes. gpt-oss-120b ships in MXFP4, about 65GB of weights, and OpenAI’s model card says it runs on a single 80GB GPU. An H100 80GB is ₹1,80,000 a month; an RTX PRO 6000 96GB is ₹1,10,000. Long contexts and many users need extra headroom, so we benchmark on your traffic first.
Is dedicated cheaper than a per-token API?
It depends on volume. An API bills every token, so cost rises with traffic. A dedicated GPU is a fixed monthly price per card, with no per-token or hourly billing. It wins when requests keep the card busy. Compare your token bill with the card price.
Do you run vLLM or Triton for us?
Yes. Single-node deployment starts at ₹50,000 one-time: drivers, vLLM, TGI or Triton, an OpenAI-compatible endpoint and a load test with your prompts. AI Production Care then runs it 24/7, from ₹75,000 per workload a month. The GPU card is a separate line.
Does inference data leave India?
No. The GPUs are in India and reserved for you, so model weights and prompts stay on your cards, not a shared API. We don’t log prompts unless you ask. This is data residency, not a compliance certification.