# AI inference hosting on dedicated GPUs

> AI inference hosting in India on GPUs reserved for your model. From ₹32,000 a card a month, host included.

- Page: https://zenocloud.io/ai/inference/
- Prices checked 14 Oct 2026. Prices exclude applicable taxes.

## Which GPU fits your model?

Weights only, ideal: parameters times bytes. Pick the smallest card that holds your model, then leave room for context and concurrent users.

| Model | FP16 | FP8 | 4-bit | Smallest GPU at FP8 | Smallest GPU at 4-bit |
| --- | --- | --- | --- | --- | --- |
| Llama 3.1 8B | 16 GB | 8 GB | 4 GB | L4 24GB (₹32,000 a month) | L4 24GB (₹32,000 a month) |
| Qwen3.8 27B (dense) | 54 GB | 27 GB | 13.5 GB | L40S 48GB (₹55,000 a month) | L4 24GB (₹32,000 a month) |
| Qwen 32B class (Qwen 2.5 or Qwen3 32B) | 64 GB | 32 GB | 16 GB | L40S 48GB (₹55,000 a month) | L4 24GB (₹32,000 a month) |
| Llama 3.3 70B | 140 GB | 70 GB | 35 GB | H100 80GB (₹1,80,000 a month · tight) | L40S 48GB (₹55,000 a month) |
| gpt-oss-120b (117B total, 5.1B active, mixture of experts) | Native MXFP4: about 65 GB of weights. Fits one 80GB card (H100 80GB, ₹1,80,000 a month, tight) or the RTX PRO 6000 96GB (₹1,10,000 a month). | — | — | — | — |

Weights only, ideal (params × bytes: FP16 2, FP8 1, 4-bit 0.5), counts rounded. Real files are larger; add KV-cache and runtime headroom. Tight means weights fill over 80% of the card. gpt-oss-120b ships in MXFP4; OpenAI’s model card says it runs on one 80GB GPU. Prices are per card per month before taxes, availability confirmed on your quote. Oct 2026.

## Private inference, tested on your prompts

We install the server, load-test it on your prompts, and hand you an OpenAI-compatible URL.

- vLLM, TGI or Triton: We install and tune the engine that suits your model.
- OpenAI-compatible endpoint: Use the OpenAI-compatible endpoint with your existing client. We check model names, parameters and tool calls before you switch.
- Data stays in India: Weights and prompts stay on cards reserved for you, in India.
- Load-tested before go-live: We replay your prompts and agree the latency target before traffic arrives.

## Inference GPU plans and prices

GPU prices are per card per month with a one-month minimum. Prices exclude applicable taxes.

| Plan | Price | What it is | Includes |
| --- | --- | --- | --- |
| NVIDIA L4 24GB | ₹32,000 per month | For models up to about 8B parameters, embeddings, speech and computer vision at modest concurrency. | On request: we confirm availability before you commit; 24GB memory per card; Low power, good for steady request rates; Add cards as traffic grows |
| AI Production Care | ₹75,000 per workload a month | We deploy the serving stack, benchmark it on your prompts and own the endpoint in production. The GPU card is a separate line. | Serving stack set up and load-tested; Endpoint, GPU and queue monitored 24/7; P1 response under 15 minutes; Scaling and model updates handled with you |
| NVIDIA H100 80GB | ₹1,80,000 per month | For quantized 70B models at a validated context and concurrency. Longer contexts or more users move to H200 or more cards. | 80GB HBM memory per card; Batching raises throughput when many requests are in flight; H200 141GB available for larger contexts |

## FAQ

### What is an AI inference server?

A server with one or more GPUs that runs a trained model to answer requests behind an API. We rent you the GPU server by the month and can deploy and run the serving stack on it.

### Which GPU should I use for AI inference?

Match memory to the model. An L4 (₹32,000) suits small models and an L40S (₹55,000) mid-sized ones. An H100 (₹1,80,000) runs a quantized 70B at a tested context. Longer contexts or more users move to an H200 (₹2,20,000). The sizing table has the arithmetic.

### Can I host gpt-oss-120b on one GPU?

Yes. gpt-oss-120b ships in MXFP4, about 65GB of weights, and OpenAI’s model card says it runs on a single 80GB GPU. An H100 80GB is ₹1,80,000 a month; an RTX PRO 6000 96GB is ₹1,10,000. Long contexts and many users need extra headroom, so we benchmark on your traffic first.

### Is dedicated cheaper than a per-token API?

It depends on volume. An API bills every token, so cost rises with traffic. A dedicated GPU is a fixed monthly price per card, with no per-token or hourly billing. It wins when requests keep the card busy. Compare your token bill with the card price.

### Do you run vLLM or Triton for us?

Yes. Single-node deployment starts at ₹50,000 one-time: drivers, vLLM, TGI or Triton, an OpenAI-compatible endpoint and a load test with your prompts. AI Production Care then runs it 24/7, from ₹75,000 per workload a month. The GPU card is a separate line.

### Does inference data leave India?

No. The GPUs are in India and reserved for you, so model weights and prompts stay on your cards, not a shared API. We don’t log prompts unless you ask. This is data residency, not a compliance certification.