PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Beirit Harvey

The title "MI355X Just Flipped the Script on B200 for FP8 DeepSeek Disagg" grabs you, right? For ages, if you wanted serious distributed inference that could actually handle real traffic, everyone just defaulted to NVIDIA. No debate. 
But the latest numbers from SemiAnalysis on their InferenceX (what used to be called InferenceMAX) are shaking that up; especially around FP8 quantized DeepSeek models running in disaggregated mode. 

MI355X Just Flipped the Script on B200 for FP8 DeepSeek Disagg

The title "MI355X Just Flipped the Script on B200 for FP8 DeepSeek Disagg" grabs you, right? For ages, if you wanted serious distributed inference that could actually handle real traffic, everyone just defaulted to NVIDIA. No debate. 

But the latest numbers from SemiAnalysis on their InferenceX (what used to be called InferenceMAX) are shaking that up; especially around FP8 quantized DeepSeek models running in disaggregated mode. SemiAnalysis’ InferenceMAX / InferenceX work tracks inference performance the way software actually improves on real engines and across real interactivity targets. 

The takeaway is hard to ignore: AMD MI355X GPUs are no longer “catching up.” They’re landing on top, right where production serving actually lives.


The Moment That Matters: the Middle of the Curve

Inference isn’t one number. It’s a trade-off curve: throughput vs interactivity (tok/s/user). And SemiAnalysis shows MI355X (SGLang + MoRI) running right alongside B200 (Dynamo + SGLang), with MI355X taking a clear edge at key mid-range interactivity levels. This is the band that tends to map to real services, not lab extremes.

That’s the shift: in FP8 DeepSeek disagg, MI355X isn’t “viable.” It’s competitive enough to win.


Why This is Happening

Disaggregated serving separates prefill and decode so they don’t step on each other. It’s how frontier providers keep latency under control while scaling throughput. If your stack can’t hold up here, you don’t get to claim “production readiness.”

The latest results show AMD’s distributed inference stack getting sharper fast, especially around the plumbing that makes disagg work at scale: KV-cache transfer, dispatch/collectives, and SGLang integration.


The Fine Print

It’s true, SemiAnalysis still notes that B200 can lead on decode-only throughput at lower interactivity, and that B200 gets another boost when Dynamo + TensorRT-LLM enters.

But the important part is what changed: On the open FP8 path (SGLang-based disagg), MI355X is no longer behind. It’s pressuring the competitor model and in the most useful band, it’s ahead.


What This Means for Buyers

If you’re serving DeepSeek-class MoE models in FP8 with modern disaggregated strategies, you now have a real alternative that isn’t theoretical.

This isn’t a “cost play.” or a “good for AMD.” story. This is a legitimate performance story that’s measured and referenced. And that’s exactly what open benchmarks are supposed to do: make the market honest.


TensorWave Cloud: Engineered for AI Velocity

TensorWave is the industry’s premier AMD-exclusive cloud provider, delivering memory optimized AI infrastructure for training, fine-tuning and inference. 

Our platform combines the raw computational power of AMD's latest GPU architectures with innovative liquid cooling technology and ultra-high-speed networking, creating an optimized environment for the most demanding AI workloads. From large language model training to real-time inference, TensorWave provides the performance, scalability, and cost-efficiency that modern AI initiatives demand.

MI355X Just Flipped the Script on B200 for FP8 DeepSeek Disagg

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company