Yes, for the right jobs. An A100 80GB still fits LoRA fine-tuning, serving models up to 70B at 4-bit, batch inference on sparse mixture-of-experts models and classic ML. Pick an H100 for FP8 and training speed, an H200 when 80GB is the limit. The price is on the A100 page.
Last verified: 24 October 2026. Figures marked "our maths" are calculated from the sources, not quoted by them. Benchmarks are labelled measured (conditions published), reported (no conditions) or vendor-run (published by a seller of GPU time or hardware).
Is the A100 still worth it? The short answer by job
| Job | Still worth an A100 80GB? | Why | If not, pick |
|---|---|---|---|
| LoRA or QLoRA fine-tuning up to about 34B | Yes | 80GB holds the 4-bit base, adapters and activations; QLoRA fine-tunes a 65B model on 48GB | none needed |
| Serving a model up to 70B at 4-bit, or a 34B at 8-bit | Yes | 70B at 4-bit is 35GB of weights, 42GB with headroom, leaving about 38GB for KV cache | RTX PRO 6000 for FP8 and FP4 hardware |
| Batch or latency-tolerant serving of sparse mixture-of-experts models | Yes | Sparse models such as gpt-oss-120b are built to run on one 80GB GPU (OpenAI), and latency-tolerant jobs don't need Hopper's speed | none needed |
| Classic ML, vision, recommenders, HPC that needs FP64 | Yes | 19.5 TFLOPS FP64 Tensor, mature CUDA stack, MIG for sharing | none needed |
| Many small models on one card | Yes | MIG splits the card into up to 7 instances of 10GB | none needed |
| Multi-GPU LLM training against a deadline | No | Hopper has FP8 and 900 GB/s NVLink against Ampere's 600 GB/s | H100 |
| FP8 serving at high concurrency | No | Ampere has no FP8 Tensor Cores | H100 or H200 |
| A model or KV cache that needs more than 80GB on one GPU | No | the card is the limit, not the code | H200 |
The rest of this guide is the evidence behind each row. The current monthly price, PCIe or SXM and the quote form are on our A100 80GB page; this guide does not repeat them, and compares the A100 with the H100 and H200 by ratio.
A100 vs H100 vs H200: Ampere against Hopper
| A100 80GB | H100 80GB | H200 141GB | |
|---|---|---|---|
| Architecture | Ampere (2020) | Hopper | Hopper |
| Memory | 80GB HBM2e | 80GB HBM3 (SXM) | 141GB HBM3e |
| Bandwidth | 1.94 TB/s PCIe, 2.04 TB/s SXM | 3.35 TB/s SXM | 4.8 TB/s |
| FP16 Tensor, dense | 312 TFLOPS | 989 TFLOPS SXM | 989 TFLOPS SXM |
| FP8 | None | Yes, with Transformer Engine | Yes, with Transformer Engine |
| NVLink | 600 GB/s | 900 GB/s (SXM) | 900 GB/s |
| MIG | up to 7 at 10GB | up to 7 at 10GB | up to 7 at 18GB (SXM) |
| ZenoCloud, per card a month | the A100 page | 1.8 times the A100 | 2.2 times the A100 |
Sources: NVIDIA A100 datasheet, A100, H100 and H200 pages; dense figures are half of NVIDIA's "with sparsity" headline numbers. The H100 and H200 monthly prices are ₹1,80,000 and ₹2,20,000; the ratios are our maths from our price book.
Three things separate them in practice.
Memory bandwidth sets decode speed. Single-user token generation tracks bandwidth. In one 11-card test of Qwen3.8 27B at 4-bit, the A100 ran 45.9 tokens a second and the H100 80.5, a 1.75 times gap against a bandwidth gap of 1.6 to 1.7 times (GPU Battle, 2 Oct 2026, measured, affiliate link disclosed in the video description).
FP8 is a hardware feature. A poster setting up a server around an A100 80GB measured 43 tokens a second on an FP8 build of a 27B model against 130 on an RTX PRO 6000 with the same vLLM configuration, and 177 against 509 at eight concurrent requests. Replies pointed to INT8 or AWQ builds, and one found that dropping the FP8 KV cache lifted the A100 to 75 to 100 tokens a second with speculative decoding: FP8 weights run through a software kernel on Ampere, but an FP8 KV cache is slow (r/LocalLLaMA, 21 Jun 2026, reported). The A100 runs modern models. The build matters: INT8, AWQ or 4-bit, not an FP8 KV cache.
Interconnect decides multi-GPU training. A team that benchmarked RTX 6000-class cards against A100s for training found the newer cards within about 6% of the A100's wall-clock time, with more memory; a reply warned that without NVLink, four such cards spent 62% of GPU time on all-reduce (r/LocalLLaMA, 29 Mar 2026, reported). That is the A100's remaining edge over a PCIe card: NVLink, at 600 GB/s on SXM and by bridge for two PCIe cards.
A buyer's view from 2024 still describes the trade-off. A researcher choosing among A100, H100 and RTX 6000 Ada wrote that the A100 does not support FP8 and has slower inference than the newer GPUs, and he planned mostly inference (Rosmine ML blog, 13 May 2026, reported). If your job is inference you can quantise to INT8 or 4-bit, that matters less. If it is FP8 serving or large-scale training, it matters a lot.
What the A100 is still good at, measured
| Result | Conditions | Label | Source |
|---|---|---|---|
| Qwen3.8 27B at 4-bit, one user: A100 45.9 tokens a second, L40S 37.0, H100 80.5, RTX PRO 6000 70.6 | Q4_K_M, 11 cards tested | measured, affiliate disclosed | GPU Battle, 2 Oct 2026 |
| 70B models run on 80GB across two NVLinked A100s but "tight": a 120B model needs heavy quantisation and runs out of room for KV cache near 4k context | a homelab with two A100s, 80GB combined | reported | Hacker News, 21 Mar 2026 |
A100 40GB or 80GB?
| A100 40GB | A100 80GB | |
|---|---|---|
| Memory | 40GB HBM2 | 80GB HBM2e |
| Bandwidth | 1,555 GB/s | 1,935 GB/s PCIe, 2,039 GB/s SXM |
| Max power | 250W PCIe, 400W SXM | 300W PCIe, 400W SXM |
| MIG | up to 7 at 5GB | up to 7 at 10GB |
| Fits at 4-bit (weights plus 20%) | up to 34B (20GB); 70B needs 2 cards (42GB) | up to 70B (42GB) |
| Fits at 8-bit | 13B (16GB) | 34B (41GB) |
Source: NVIDIA's A100 datasheet and 40GB PCIe product brief; the fit rows are our maths (parameters times bytes, plus 20% headroom, KV cache on top).
Pick the 80GB card for anything with an LLM in it. 40GB cannot hold a 70B at 4-bit, and it is the version platforms are dropping: one developer platform told its customers it was retiring its A100 40GB PCIe machines and would destroy workloads left on them after 31 May 2026 (Hacker News, 17 May 2026, reported). AWS Mumbai sells the 40GB card only: p4d.24xlarge has eight A100 40GB GPUs, and we found no 80GB p4de size in the Mumbai price list. Take the 40GB card only for small models or classic ML where its lower power and price matter more than headroom. ZenoCloud's A100 is the 80GB card.
Which workloads still fit an A100
Fine-tuning. Parameter-efficient methods fit. The QLoRA paper reports a 65B model fine-tuned on a single 48GB GPU at full 16-bit task quality (Dettmers et al., arXiv 2305.14314), so an 80GB card has room for a 70B base and a long sequence. Full fine-tuning is the limit: mixed-precision Adam needs about 16 bytes a parameter (ZeRO paper, arXiv 1910.02054), so a 7B model wants about 112GB (our maths). For full fine-tuning of 7B and up, use several cards with NVLink or an H100 or H200.
Inference. A 70B at 4-bit fits with about 38GB for KV cache; a 34B at 8-bit fits with room; a 13B at FP16 fits with room for many users. Long context multiplies KV cache: one tester reported fitting 9 concurrent 128K-context users on one A100 with a compressed KV cache, against 2 at FP16 (r/Qwen_AI, 9 Sep 2026, reported; their own method). If your plan needs long context for many users, size for the H200.
Classic ML and HPC. The A100's FP64 and mature CUDA libraries are why it still shows up in recommenders, vision, genomics and simulation. CUDA 13 removed support for Maxwell, Pascal and Volta; Ampere is still supported (NVIDIA CUDA toolkit release notes, 13.x).
Sharing one card. MIG splits an A100 80GB into up to seven isolated 10GB instances. One engineer measured noisy-neighbour effects on shared vLLM against MIG slices on an A100 (r/nvidia, 17 Apr 2026, reported); if several teams share a card, MIG gives each a fixed slice.
When to pick an H100 or H200 instead
The price ratio is the rule. The H100 costs 1.8 times the A100 per month and the H200 2.2 times (our maths from our price book). So an H100 wins per run when it finishes the job in under 55% of the A100's time, and an H200 when it finishes in under 45%. Measure that on your job, not on a spec sheet.
Rent the H100 when the job is FP8 serving, large training runs where time matters, or anything multi-GPU on NVLink: see our H100 price guide. Rent the H200 when 80GB is the measured limit: a 70B at 8-bit on one card, long context, many concurrent users; our H200 price guide has the numbers, and the H100 vs A100 vs H200 comparison puts all three side by side. If the job is single-card inference and you are free to choose, test the RTX PRO 6000 too: it costs about 11% more a month than the A100, with 96GB and FP8 and FP4 hardware, but PCIe only and no NVLink; the RTX PRO 6000 guide has owner benchmarks. For mixed AI and graphics at 48GB, see the L40S guide.
One note from the other side: a commenter wrote that Hopper is a big upgrade over Ampere (r/LocalLLaMA, 29 Mar 2026, reported). Both statements can hold. For FP8 and multi-GPU training, Hopper is the upgrade. For quantised inference on one card and for LoRA, the A100 is often enough.
Rent or buy an A100
We do not print A100 purchase prices. NVIDIA publishes no list price for it, resellers and used listings change weekly, and a dated listing says little about what your quote will be. Two facts matter more than a price.
A second-hand A100 comes with no usage history. A researcher proposing an open health score for retired data-centre cards pointed out in October 2026 that cards from DGX-1 and DGX A100 systems are being resold with nothing to say how much life they have left (NVIDIA forum, 17 Oct 2026, reported). A buyer weighing four used A100 40GB cards wrote that the fear was getting one "on its last leg" (r/LocalLLaMA, 15 Aug 2026, reported).
And buying is not obviously cheaper than renting. Hacker News commenters in 2026 said A100 rental prices had climbed since 2024 and the 80GB card had held its resale value (reported; one cited a public GPU pricing index). A rented card sidesteps both the resale question and the unknown wear. Rent first, test on your own job, and buy only if the card will run most of every month for years and you have the hall and the hands. Our H200 guide has the break-even method and an owner's full count.
Need an A100 for a few hours? Use AWS in Mumbai
For a benchmark or a weekend run, rent by the hour. AWS Mumbai lists p4d.24xlarge: eight A100 40GB GPUs, 96 vCPU, 1,152 GiB, at $26.344 an hour on demand, about $3.293 a GPU (our maths; AWS's price list of 23 October 2026, Linux, Asia Pacific (Mumbai)). At ₹94.94 to the dollar (European Central Bank reference rate, 23 October 2026) that is about ₹2,501 an hour for the node. There is no single-GPU A100 size, and the smallest bill is the whole node. A startup engineer wrote in June 2026 that AWS had no capacity for a single-GPU A100 and you had to rent eight (Hacker News, 25 Jun 2026, reported).
For a few hours that is fine. For a card you keep, the monthly A100 on our page is the cheaper shape. We manage AWS: our Managed AWS desk watches the account 24/7, patches what you approve and explains the bill.
When ZenoCloud is the right choice
ZenoCloud is the right choice when the A100 will run most of the month and your stack is validated on Ampere. The A100 80GB is sourced on request: tell us the card count, PCIe or SXM and the date you need it, we reply within one business day, and we confirm the variant, the host and the availability in writing before you commit. Terms are a one-month minimum, in rupees, on one invoice.
We have run servers since 2009, more than 1,000 of them, on a 99.9% uptime SLA, from partner facilities. The quote states the card, the variant, the host, the boot disk, the network and the price before tax, with the NVIDIA driver installed. You run your own software. See the A100 80GB page for the current price and the quote form, and the GPU cloud page for the whole range.
Not a fit: light or bursty use, where AWS by the hour costs less (we manage AWS if you would rather not run the account); FP8 or FP4 serving at scale, where the H100, H200 or Blackwell cards are the better rent; or a model that needs more than 80GB on one card.
Take a trial node before committing: a short paid trial on the card you plan to use, terms agreed before setup. If you keep it, the trial node becomes your production node. For model sizes, see our GPU inference hosting page.
How to choose in five questions
- Does the model and its KV cache fit in 80GB, after quantising to INT8 or 4-bit? If no, the H200.
- Do you need FP8, or will you train across several GPUs against a deadline? If yes, the H100.
- Is it one card of inference or LoRA? Then the A100 is enough, and the RTX PRO 6000 is worth a test.
- Will several teams share the card? Use MIG on the A100.
- Will it run most of the month? If not, use AWS by the hour.
Frequently Asked Questions
Is the NVIDIA A100 still worth it in 2026?
Yes for LoRA fine-tuning, serving models up to 70B at 4-bit, batch serving of sparse mixture-of-experts models, classic ML and MIG sharing. No for FP8 serving, multi-GPU training against a deadline, or models that need more than 80GB on one GPU.
A100 vs H100: which should I rent?
The H100 for FP8, large training runs and multi-GPU work on NVLink. The A100 for quantised inference and LoRA on one card. The H100 costs 1.8 times the A100 a month, so it must finish the job in under 55% of the time to win per run.
A100 40GB or 80GB: which do I need?
80GB for anything with an LLM: a 70B at 4-bit needs about 42GB with headroom, which 40GB cannot hold. The 40GB card suits small models and classic ML. Some platforms are retiring 40GB cards, and ZenoCloud's A100 is the 80GB.
Can an A100 run a 70B model?
At 4-bit, yes: about 35GB of weights, 42GB with headroom, leaving about 38GB of an 80GB card for KV cache. At 8-bit it is about 84GB with headroom, which does not fit. FP16 needs two or more cards or an H200.
Does the A100 support FP8?
No. Ampere has no FP8 Tensor Cores. Use INT8, AWQ or 4-bit builds. One owner measured 43 tokens a second on a 27B FP8 build and got 75 to 100 after dropping the FP8 KV cache; Hopper cards run FP8 natively.
Is the A100 good for fine-tuning an LLM?
For LoRA and QLoRA, yes: QLoRA fine-tunes a 65B model on 48GB. Full fine-tuning needs about 16 bytes a parameter, so a 7B model wants around 112GB. For that, use several NVLinked cards or an H100 or H200.
Is the A100 obsolete or end of life?
Not by software: CUDA 13 dropped Maxwell, Pascal and Volta but still supports Ampere, and NVIDIA still lists the A100. By job, some work has moved to Hopper and Blackwell, while quantised inference and LoRA still run well.
A100 vs RTX PRO 6000: which is better?
For one card of inference, the RTX PRO 6000: 96GB, FP8 and FP4 hardware, about 11% more a month. For NVLink, MIG and a mature Ampere stack, the A100. One team's benchmark found them within 6% on training time.
Where do I find current A100 rental terms?
On our A100 80GB page: the current monthly price, the PCIe and SXM variants and the quote form. This guide compares the A100 with the H100 and H200 by ratio so it does not go stale.
Should I buy a used A100 or rent one?
Rent first. Used A100s come without usage history, resale and rental markets have been firm in 2026, and a rented card removes both risks. Buy only if the card will run most of every month for years and you have the room and the hands.
Is the A100 PCIe or SXM?
Tell us which you need. PCIe suits single-card inference servers; SXM with NVLink suits multi-GPU training. NVIDIA lists 600 GB/s NVLink for SXM and for a two-card bridge on PCIe. We confirm the variant, the card count and the interconnect before setup.
Sources
Checked 24 October 2026 unless dated.
- ZenoCloud A100 80GB GPU cloud, H100, H200, RTX PRO 6000, H100 vs A100 vs H200, GPU cloud pricing, GPU inference hosting and Managed AWS
- NVIDIA: A100 datasheet, A100 40GB PCIe product brief, A100, H100 and H200 pages, and the CUDA toolkit release notes
- AWS, EC2 On-Demand price list, Linux, Asia Pacific (Mumbai), published 23 October 2026
- European Central Bank euro reference rates, 23 October 2026: USD/INR cross-rate ₹94.94 (via frankfurter.dev)
- Dettmers et al., QLoRA and Rajbhandari et al., ZeRO
- Reddit threads: r/LocalLLaMA, A100 slow with a 27B FP8 model (21 Jun 2026); r/LocalLLaMA, training at home on A100 or H100 (29 Mar 2026); r/LocalLLaMA, RTX PRO 4500 vs four A100s (15 Aug 2026); r/Qwen_AI, compressed KV cache on one A100 (9 Sep 2026); r/nvidia, MIG slices on an A100 (17 Apr 2026)
- Hacker News: homelab with two A100s (21 Mar 2026); platform retiring A100 40GB (17 May 2026); on-demand GPU capacity (25 Jun 2026)
- Rosmine ML blog, Was my $48K GPU server worth it?, 13 May 2026
- GPU Battle, Qwen3.8 27B on 11 GPUs (YouTube, 2 October 2026)
- NVIDIA developer forum: health grading for used data-centre GPUs (17 Oct 2026)