PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Beirit Harvey

Tensorwave Powers Moreh's Frontier LLM Inference on MI355X and Reduces Model Weight Download Time by ~75%.  Read TensorWave's latest case study. 

TensorWave Powers Moreh's Frontier LLM Inference on MI355X and Reduces Model Weight Download Time by ~75%

About Moreh

Moreh is an AI infrastructure software company focused on making large-scale LLM inference faster, more reliable, and more cost-efficient on AMD-based GPU systems. They work with data centers and enterprise teams running model services or internal LLM platforms, helping optimize performance, consistency, and operational cost across large inference clusters.


APAC Requirements: Governance, Trust, Support

Moreh operates across APAC markets, where inference adoption is often shaped by data sovereignty requirements, longer decision cycles, and a higher bar for long-term partner trust and support. In practice, that means proving stability and operability matters as much as raw performance. This is especially important when validating new GPU generations early in their lifecycle.

In many APAC markets, NVIDIA has stayed the default; not always because it’s the best fit, but because smaller infrastructure teams tend to choose the most familiar ecosystem with the most established playbooks, where “operational stability” feels like the safest bet. 

That’s starting to change: as more companies move real workloads onto AMD and see the performance-per-dollar and TCO upside in production, the default is shifting. Teams that are adopting AMD will be ahead of the curve. 

TensorWave provided Moreh with a stable, production-grade AMD environment purpose-built for validating next-generation GPUs early in their lifecycle. Equally important, TensorWave paired that infrastructure with hands-on, real-time engineering support, allowing Moreh to validate demanding MI355X inference workloads without getting trapped in early-generation edge cases.

This proof of operability coupled with performance enabled Moreh to confidently reduce NVIDIA dependency in APAC deployments. The shift wasn’t about theoretical parity; it was about demonstrating reliability, consistency, and repeatable results on modern AMD infrastructure.


The Challenge: Proving Reliable, Cost-Efficient Inference on the Latest AMD GPUs

Moreh supports data centers and enterprise teams running model services and internal LLM platforms. Their customers prioritize two outcomes that are exceptionally difficult to deliver at scale::

 * Performance + cost optimization for large-scale inference on AMD infrastructure
 * Consistent, reliable inference across large clusters

As AMD releases new GPU generations, early adoption can come with real friction, especially around library compatibility and the ability to run the workloads you need without spending weeks debugging infrastructure edge cases. Moreh wanted an environment where they could test and optimize the newest AMD Instinct GPUs in a real cloud setting and turn those learnings into better inference outcomes for their customers.


Why TensorWave: MI355X Access, AMD Focus, and Real-World Optimization

Moreh’s search centered on two must-haves:

 1. Usage cost (the most important factor)
 2. Confidence their libraries and workloads would run properly (especially important early in a new GPU lifecycle)

The Moreh team chose TensorWave because they viewed it as a leading provider running state-of-the-art AMD-based infrastructure, and because TensorWave actively pursues optimization of new AMD GPU generations in real service environments; including visible collaboration in the ecosystem.

For TensorWave, optimization for new AMD GPU generations spans the full stack; from inference serving and runtime tuning to networking and storage acceleration, plus cluster-level performance baselining to keep results stable at scale.

Additionally, high GPU memory (288 GB) and strong memory bandwidth were critical for Moreh’s development and validation work on large models and inference workloads.


The Work: MI355X Inference Validation Across Multiple Frontier Models

This engagement was Moreh’s first time using MI355X nodes on TensorWave. The goal was to evaluate MI355X inference behavior and performance signals across a range of modern LLMs.

Models tested (with vLLM):

 * GPT-OSS 120B
 * Deepseek-R1 671B
 * Qwen3 235B A22B
 * Llama-4-Maverick 17B
 * Llama3.3 70B

Core software stack:

 * ROCm (for testing)

vLLM (for inference serving/measurement)


Metrics at a Glance


Because model size, duration, and runtime configuration can vary significantly by model/workload scenario, Moreh’s approach emphasized broad coverage and realistic testing rather than forcing a single “typical job” template.


Results: Faster Setup, Strong MI355X Signals, Smooth Evaluation


1) Model weight download time reduced to ~25% of prior environments

Across their testing, Moreh found that model weight download time dropped to about 25% of what they previously experienced in other environments, reducing a common bottleneck that can slow iteration and evaluation loops.


2) MI355X showed clear performance improvements vs MI300X in key areas

Moreh observed that the MI355X node demonstrated improvements over MI300X in:

 * Memory bandwidth
 * Computational performance (e.g., GEMM)

GEMM performance testing with hipBLASLt shows that the MI355X delivers a major uplift over MI300X. In the BF16 test, MI355X reaches a maximum of about 1600 TFLOPs, roughly double the BF16 performance of the MI300X machine. In the FP8 test, MI355X peaks at about 3200 TFLOPs, which is approximately 2× faster than MI355X running BF16. Overall, the results indicate that MI355X offers a significant improvement in computational capacity compared to MI300X.


3) Engineering velocity improved for MI355X validation

Because this was Moreh’s first MI355X evaluation environment, they didn’t have historical MI355X iteration cycles to compare against. However, they noted that the stability and ease of use enabled them to run analysis smoothly and “without bottlenecks,” helping them move faster through development and validation.


Onboarding & Support: Seamless Experience with Real-Time Help

Moreh described onboarding as seamless; getting access to MI355X capacity quickly, with clear setup guidance that made it easy to start running tests right away. They also emphasized that real-time Slack support removed friction during evaluation when questions came up around environment configuration, runtime settings, or job execution. 

TensorWave’s team responded quickly so Moreh could keep experiments moving instead of losing cycles to troubleshooting. The result was less downtime, faster iteration, and a smoother path through first-time MI355X validation.


What Didn’t Work: Early-Lifecycle Library Compatibility Gaps

During testing, Moreh identified a limitation: it was not possible to use certain libraries in their setup, specifically MORI (Modular RDMA Interface) for internode communication on TensorWave’s MI355X nodes.

This feedback reflects a common reality in early access periods for new GPU platforms: even when core performance is strong, specific components in the broader ecosystem may need additional enablement or alignment.


Business Impact: A Faster Path to Better Inference for Customers

While it’s still early for Moreh to translate MI355X improvements into customer-level metrics, the immediate impact is clear:

 * Hands-on MI355X experience in a stable cloud environment
 * Faster validation of how Moreh’s software behaves on next-gen AMD Instinct GPUs
 * Early “better-than-expected” performance signals that Moreh expects to convert into faster, more reliable model serving as optimization work continues

“TensorWave has given us early, hands-on access to MI355X in a real production cloud, and the results have been extremely encouraging. We’re validating and optimizing against next-generation AMD Instinct GPUs faster than we anticipated, and the performance signals we’re seeing are already exceeding expectations. While it’s still early to publish customer metrics, this work is laying a clear path to meaningfully faster and more reliable inference for our customers as we continue to optimize.”

— Gangwon Jo - CEO of Moreh


Looking Ahead: Scaling Distributed Inference on Modern AMD Infrastructure

Moreh is expanding its AMD-based distributed inference software and plans to validate it across:

 * Multi-node configurations
 * Heterogeneous node environments

They see ongoing MI355X availability as a key enabler for deeper optimization and expect continued work on TensorWave to create strong synergy as Moreh elevates MI355X performance for real-world inference deployments.

“It’s been a pleasure collaborating with the talented team at Moreh. We are excited to continue our work with them.”

— Brad Bokal - VP, Partnerships 


TensorWave Cloud: Engineered for AI Velocity

TensorWave is the industry’s premier AMD-exclusive cloud provider, delivering memory optimized AI infrastructure for training, fine-tuning and inference. 

Our platform combines the raw computational power of AMD's latest GPU architectures with innovative liquid cooling technology and ultra-high-speed networking, creating an optimized environment for the most demanding AI workloads. From large language model training to real-time inference, TensorWave provides the performance, scalability, and cost-efficiency that modern AI initiatives demand.

Ready to get started? Connect with a sales engineer. 



TensorWave Powers Moreh's Frontier LLM Inference on MI355X and Reduces Model Weight Download Time by ~75%

$350M Series B Announcement

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company