PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Beirit Harvey

In a recent talk, Nikhil Gupta, ML Engineer at TensorWave, walked through two company case studies that show just how far you can push training and inference when you pair modern AMD hardware with the right software stack.

Real AI Workloads on AMD GPUs: Inference, Training, and Scaling

In a recent talk, Nikhil Gupta, ML Engineer at TensorWave, walked through two company case studies that show just how far you can push training and inference when you pair modern AMD hardware with the right software stack.

We’re truly pushing the frontier of what’s possible with machine learning and AI. TensorWave is committed to commoditizing compute in order to drive further progress and innovation.

— Nikhil Gupta, ML Engineer at TensorWave


Why AMD-Only and Why Now

TensorWave was founded a little over two years ago on a contrarian decision: build an AMD-only AI cloud.

Running on state-of-the-art AMD Instinct GPUs inside Supermicro servers, TensorWave’s platform was designed to break the CUDA moat and give teams a viable alternative to the NVIDIA-dominated ecosystem.

That decision is paying off:

 * $100M Series A – the largest in Nevada startup history.
 * Two data centers live – Tucson and Miami, with more on the roadmap.
 * Air-cooled MI300s → direct liquid-cooled MI325/MI355 as the platform has evolved.

TensorWave has:

 * Deployed the largest direct liquid-cooled MI325 cluster, fully networked for large-scale inference and training.
 * Become one of the first to deploy MI355 GPUs with Supermicro’s new Polara AI NICs.

Built an environment optimized for scale: direct liquid cooling means no deafening fans, comfortable data hall temps, and the ability to grow faster and denser than with legacy air-cooled racks.


Case Study 1: Modular 

Company: Modular
Workload type: High-throughput inference on AMD Instinct GPUs.

Modular is building a unified compute layer for AI, made up of three core components:

 * Mojo – GPU programming language.
 * Max – optimized inference engine.
 * Mammoth – cloud orchestration layer.

TensorWave and Modular share the same philosophy: increase optionality for end users and make it easy to move beyond a single-vendor stack.

By combining:

 * Modular’s software optimizations.
 * The higher GPU memory capacity of AMD Instinct MI355 GPUs.
 * TensorWave’s tightly integrated Supermicro infrastructure.

Modular was able to demonstrate:

 * ~2× greater inference throughput on MI355-based clusters when using Max.
 * ~40–60% reduction in inference costs, depending on the workload.

These gains are not theoretical. They come from real enterprise scenarios across multiple workloads, backed by benchmarks that TensorWave and Modular have published together. For organizations running large-scale inference, this combination means:

 * More tokens per second per dollar.
 * The ability to run larger models or more concurrent requests on the same footprint.
 * A clear path away from single-vendor lock-in, without giving up performance.


Case Study 2: A Telecom Giant and ScalarLM

Company: Established telecom company.
Workload type: Training + fine-tuning for internal and customer-facing LLMs.

TensorWave also works closely with ScalarLM, an open-source project developed by Greg Diamos and his team and supported by TensorWave. ScalarLM unifies the inference and training pipeline so that:

 * You can serve models.
 * Fine-tune them.
 * Push updated weights back into the inference stack.

All of this can be done without leaving a single framework.

It’s built completely on open-source components like vLLM and Megatron, aligning strongly with TensorWave and AMD’s commitment to open ecosystems.


The Scale: Billions of Tokens and Nearly a Billion API Calls

The telecom company runs an internal LLM tool that is already operating at serious scale:

 * >9 billion tokens per day.
 * Nearly 1 billion API calls per day.

Now, they’re preparing to make this tool customer-facing, which will dramatically increase traffic and performance requirements.

Sending all that traffic to external APIs (e.g., general-purpose LLM services) would be:

 * Expensive.
 * Harder to control from a latency and reliability perspective.
 * Risky from a data residency and privacy standpoint.

Instead, they wanted a self-hosted, fine-tuned model that would:

 * Keep data fully internal and secure.
 * Optimize for their domain (telecom).
 * Run efficiently on AMD Instinct GPUs at large scale.


The Approach: Smaller Model, Bigger Results

Rather than relying on a massive frontier model, the team fine-tuned a 4B-parameter model based on Gemma on their own telecom-specific data.

With TensorWave infrastructure, ScalarLM, and AMD Instinct GPUs, they were able to:

 * Treat this 4B model as a “tiny specialist”.
 * Iterate on training configs and architectures in just a couple of weeks.
 * Benchmark it head-to-head against much larger, general-purpose models.

The results:

 * The fine-tuned 4B model topped internal telecom benchmarks, outperforming leading general-purpose models from major providers.
 * It also reached the top of the Spider 2.0 SQL benchmark leaderboard within this evaluation context, showcasing strong structured-query reasoning.


Why Tiny Models Matter

Because the model is small and tightly tuned to telecom workloads, it can be dramatically more efficient:

 * Nikhil describes a scenario where a model with the performance of a large Llama-class model (e.g., 405B) can be matched by a tiny, highly specialized model with roughly 1/100th of the footprint.
 * On a single MI355 node, that can translate from running one large model instance per node to running up to 64 model instances using GPU partitioning.
 * Smaller models also run faster, which significantly boosts throughput and user experience.

This telecom partner is already:

 * Serving billions of tokens per day.
 * Preparing to scale further as the tool goes customer-facing.
 * Planning larger-cluster training runs with more data on TensorWave.

The upcoming results are expected to be showcased at GSMA 2026, highlighting how domain-specialized, open-source stacks on AMD can go toe-to-toe with frontier-scale systems; at a fraction of the cost and complexity.


What Ties it All Together

Behind both case studies is a common stack:

 * Hardware: Supermicro servers with AMD Instinct MI300- and MI350-series GPUs, including direct liquid-cooled MI325 and MI355.
 * Networking: Fully interconnected clusters designed for large-scale training and inference.
 * Software: Projects like Modular and ScalarLM that embrace openness and portability.
 * Operations: Data centers in Tucson and Miami, built for density, efficiency, and comfort, with direct liquid cooling enabling faster, larger buildouts.

These aren’t isolated experiments, they’re signals of a broader shift:

 * Away from proprietary lock-in.
 * Toward open, AMD-first ecosystems.
 * Where billions of tokens per day, 2× throughput, and up to 60% lower costs are very much on the table.


About TensorWave

TensorWave is the AI neocloud purpose-built for performance. Powered exclusively by AMD Instinct™ Series GPUs, we deliver high-bandwidth, memory optimized infrastructure that scales with your most demanding models–training or inference. 

Ready to get started? Connect with a sales engineer. 

Real AI Workloads on AMD GPUs: Inference, Training, and Scaling

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company