PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Alex Medick

In this Beyond CUDA podcast recap, we discuss optimizing AI deployment, and fostering a diverse, innovative AI ecosystem with Mako co-founder Waleed Atallah..

Mako’s CEO on Crossing the CUDA Moat and Automating GPU Kernels

In a recent episode of the Beyond CUDA podcast, host Jeff Tatarchuk, co-founder of TensorWave, sat down with Waleed Atallah, the co-founder and CEO of Mako, to discuss the challenges and innovations in GPU kernel generation. This episode dives into how Mako is tackling the complex problem of automating kernel generation to optimize AI model deployment across diverse hardware platforms. With Nvidia’s CUDA ecosystem dominating AI hardware acceleration, Mako’s mission to provide automated, hardware-agnostic solutions is both ambitious and timely. 

This blog post summarizes the episode’s key points and highlights the potential impact of Mako’s work on the broader AI ecosystem.









Summary

Waleed Atallah’s journey into hardware acceleration began at Intel in 2016, where he worked on FPGAs for deep learning acceleration. Observing the rise of Nvidia GPUs and the challenges of writing compute kernels, he later joined Untether AI to focus on AI-specific chips. At both companies, Atallah noticed a persistent bottleneck: the scarcity of skilled kernel engineers. This realization led him to co-found Mako with the aim to automate the generation, optimization, and deployment of GPU kernels.

Atallah explained that GPU kernels are critical for mapping algorithms to hardware, requiring deep expertise in both software and hardware. However, the rarity of such expertise (estimated at one skilled kernel engineer per 100,000 software developers) creates a significant barrier. Nvidia has mitigated this with CUDA libraries like cuBLAS and cuDNN, but these are exclusive to Nvidia hardware, limiting ecosystem diversity. Mako’s solution combines traditional compiler techniques with large language model (LLM)-based code generation, fine-tuned on a robust dataset of annotated, functionally correct kernels. This approach aims to produce compilable, performant kernels for various hardware platforms, including AMD, Intel, and others.

The episode also covered Mako’s progress, which has exceeded expectations, with a product launch initially planned for late 2025 now anticipated for summer. Atallah highlighted the importance of an end-to-end pipeline that integrates kernel generation with inference engines like vLLM or SGLang, ensuring seamless deployment. Mako is already working with hyperscalers and seeing 10-30% performance improvements through automated optimization. However, challenges remain, including the need for more compute resources to scale experiments and the risk of competing solutions like PyTorch’s inductor outperforming their efforts.

Atallah emphasized the importance of a diverse AI ecosystem, where competition drives innovation in algorithms and hardware. Mako’s name, inspired by the agile Mako shark (a predator of the barracuda… get it?) reflects their mission to challenge Nvidia’s CUDA dominance and enable broader hardware compatibility.


Key Takeaways

 * Kernel Generation Bottleneck: Writing GPU kernels is a complex task requiring rare expertise in both software and hardware, with skilled engineers being highly sought after and expensive.
 * Mako’s Approach: Mako automates kernel generation using fine-tuned LLMs integrated with traditional compilers, aiming to produce efficient, hardware-agnostic kernels.
 * Ecosystem Diversity: By enabling kernel generation for non-Nvidia hardware, Mako supports a competitive AI ecosystem, fostering innovation in algorithms and hardware.
 * Progress and Challenges: Mako’s rapid progress has moved their product launch to summer 2025, but scaling compute resources and staying ahead of traditional tools like PyTorch’s inductor remain challenges.
 * Broader Impact: Automated kernel generation could unlock new algorithms, optimize new hardware, and enhance AI deployment efficiency, benefiting researchers and companies alike.

For those interested in exploring Mako’s solutions or contributing to their mission, visit mako-dev.com or connect with Atallah on LinkedIn or Twitter. As the AI landscape evolves, Mako’s work could play a pivotal role in crossing the CUDA moat and building a more inclusive, innovative ecosystem.


About TensorWave

TensorWave is the AI neocloud purpose-built for performance. Powered exclusively by AMD Instinct™ Series GPUs, we deliver high-bandwidth, memory-optimized infrastructure that scales with your most demanding models—training or inference.

Ready to get started? Connect with a Sales Engineer.

Mako’s CEO on Crossing the CUDA Moat and Automating GPU Kernels

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company