PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Ryder Hankins

At this year’s Beyond Summit, Neha Prakriya, a member of AMD's technical staff working on large-scale pre-training, gave a technical deep dive into how AMD is using AI agents to automatically optimize both training and inference workloads from top to bottom across the stack.

AMD's Agentic Optimization Loop: Closing the Gap and Going Beyond

At this year’s Beyond Summit, Neha Prakriya, a member of AMD's technical staff working on large-scale pre-training, gave a technical deep dive into how AMD is using AI agents to automatically optimize both training and inference workloads from top to bottom across the stack.

The result? 

Performance improvements that don't just close the gap with competitors, but push past it.


An Iterative, Closed-Loop Optimization Agent

The core of AMD's approach is a closed-loop optimization agent that autonomously cycles through idea generation, implementation, testing, and decision-making. The agent operates at every level of the stack; from application code, to ROCm, all the way down to individual kernel optimizations.

"Given enough time, we optimize across the stack to make sure that the performance is as high as possible."

The agent builds a dynamic tree of ideas, where each branch can be explored, deepened, or abandoned based on real-time scoring. When an optimization attempt succeeds, the agent doubles down on related ideas in that branch. When it fails, it backtracks and reprioritizes. Every action triggers a rescore of the remaining candidates, making the search smarter over time.

Crucially, the agent also maintains a growing knowledge base that accumulates AMD-specific learnings across every run. This means each new optimization session starts with more context than the last; models that once took 8 hours to tune now get there in 1.5.


Real-World Results on Inference X

The performance numbers from this agent are striking. On GLM5's FP8 model, AMD went from trailing a major competitor to leading it, all in a single optimization run.

+193% - tokens/sec/GPU improvement on GLM5 FP8 vs. Feb–Mar baseline+13.6% higher throughput than the leading competitor on GLM51.5 hours to fully optimize DeepSeek-R1 (down from 8 hours)

For GLM5, the agent identified that AMD's Inference X workload, backed by SGLang and AIT, was missing optimized GEMM configs for many of the shapes that GLM5 relies on. The agent systematically tuned 55 of those GEMM kernels, patched them into AIT, and wove them into the end-to-end workload. It also introduced a new FP8 quantization kernel merged directly into the framework.

For DeepSeek-R1, the agent implemented speculative decoding after noticing a competitor was using it, and when time-to-first-token spiked 10x during the attempt, the agent recovered on its own by adjusting AIT flags. It shipped seven different config optimizations in a single run, reaching performance levels AMD hadn't previously submitted to Inference X benchmarks.


How the Agent Decides What to Try

The scoring system behind the agent balances multiple factors: expected performance gain, implementation time, accuracy risk, and urgency relative to roofline or competitor targets. Operations that sit far below the MI355X roofline get a score boost - the agent is explicitly incentivized to go after the highest-leverage bottlenecks first.

Accuracy is a hard constraint. The agent is not permitted to simply drop precision to gain speed, as any optimization that risks benchmark accuracy is penalized in scoring. Likewise, the knowledge base flags operations that are historically crash-prone, keeping the agent away from known landmines.


Not Just Inference

While the session focused largely on inference, Prakriya also touched on AMD's progress in large-scale training. AMD's own training framework, Primus, has been key to validating training runs and closing gaps that previously required manual debugging. The progress is real: companies like Essential AI have already trained large models on AMD infrastructure. Prakriya credited close collaboration across software, compiler, and kernel teams for getting there.


What's next

AMD indicated that more results are coming soon, at both Inference X and MLPerf, as the agent continues to push performance across an expanding set of models. With TensorWave providing additional GPU capacity to extend agent run times, the ceiling for what these automated optimization loops can achieve is only going higher.


Want to Dive Even Deeper?

Watch the full video here:


TensorWave Cloud: Engineered for AI Velocity

TensorWave is the industry’s premier AMD-exclusive cloud provider, delivering memory optimized AI infrastructure for training, fine-tuning and inference. 

Our platform combines the raw computational power of AMD's latest GPU architectures with innovative liquid cooling technology and ultra-high-speed networking, creating an optimized environment for the most demanding AI workloads. From large language model training to real-time inference, TensorWave provides the performance, scalability, and cost-efficiency that modern AI initiatives demand.

Ready to get started? Connect with a sales engineer. 

AMD's Agentic Optimization Loop: Closing the Gap and Going Beyond

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company