PostHeaderSection

Symbol

Global NavBar

PostContentSection

Global Footer

Blog Detail Page

Beirit Harvey

Explore how Liquid AI designs compact models for edge devices, combining hardware-aware architecture and training to support efficient on-device inference.

Edge AI Starts With the Model Architecture

Think about the computing devices around you and what you ask them to do. Your smartphone translates a message. A smart thermostat adjusts the temperature. A smart doorbell alerts you when someone arrives. A smart fridge tracks what is inside, while a VR headset responds as you turn your head.

These devices have very different resources. A phone may have a powerful processor but still needs to preserve its battery. A thermostat has a much smaller computing budget. A VR headset has to respond quickly enough to keep the experience comfortable. Connectivity varies, too. Some devices have a reliable internet connection; others need to work when that connection is slow or unavailable.

Not every task requires AI, and not every smart device runs a foundation model. But these examples make the challenge concrete: when AI runs on a device, it has to fit that device’s hardware and the job people expect it to do. This is the practical idea behind on-device, or edge, AI.

That is the starting point for Liquid AI’s approach to foundation models. In his presentation at TensorWave’s Beyond Summit, Mathias Lechner, Co-Founder and CTO of Liquid AI, explained how the company designs models around the hardware that will run them.

LFM2, its second generation of Liquid Foundation Models, emerged from a search process that evaluates model quality alongside speed and memory use on real devices. Deployment requirements become part of architecture design from the beginning.


Start With the Device

Consider a phone translating a conversation. The model needs to produce a useful translation, respond fast enough to keep the conversation moving, and fit within the memory available alongside other applications. If the feature needs to work offline, it cannot depend on sending every request to a remote server.

A laptop, a car, and a system on a factory floor each bring a different set of constraints. Available memory, memory bandwidth, and compute capacity vary, as do expectations for response time and answer quality.

Choosing a model therefore involves more than comparing evaluation scores. A model may perform well on a benchmark while using too much memory for its intended device. Another may respond quickly but struggle with the task it needs to perform.

Lechner described three priorities that guide Liquid AI’s architecture search:

 * Quality: How well does the model handle the tasks people need?
 * Speed: How quickly can it process a prompt and generate a response on the target hardware?
 * Memory: How much memory does the full inference process require?

That last point goes beyond the model’s weights or its KV cache, which stores attention information used during generation. The software running the model also allocates working buffers and other intermediate data. Measuring actual memory use helps account for the complete workload.


Search, Measure, and Refine

A model’s architecture determines how it processes information. There are many possible building blocks, including attention, convolutions, and recurrent designs. Liquid AI uses an evolutionary search process to explore combinations of these components.

The process starts with a group of candidate architectures. Each is profiled on the target device to measure speed and memory use. Training and evaluation establish how well the candidates perform across tasks such as translation and multistep reasoning.

Those results guide the next round. The search recombines promising designs, changes individual components, and evaluates the resulting architectures. Repeating this process helps identify combinations that offer a better balance of quality and efficiency.

Quality remains part of every decision. Fast answers have limited value if they are consistently wrong. Lechner also emphasized working with customers to make evaluations reflect the applications they actually want to build.

Hardware measurements therefore have a direct role in model development. How an architecture behaves on a device influences which designs move forward.


How That Search Shaped LFM2

Lechner described using a Samsung Galaxy S24 Ultra with a Qualcomm system-on-chip as a target for the search. The resulting LFM2 architecture combines double-gated short convolutions with grouped-query attention.

The short convolutions process nearby information in a sequence efficiently, while the attention layers help the model use information across a longer context. Liquid AI’s technical report describes a design with more convolution blocks and fewer attention blocks, selected to meet device speed and memory requirements while preserving quality.

In the comparisons presented during the talk, Lechner reported improvements over contemporary models in both stages of inference: prefill, when the model processes the prompt, and decode, when it generates output tokens. He also reported a smaller memory footprint.

For someone using an application, those measurements translate into practical questions: How long do I wait for an answer? How quickly does the response appear? Can the model run within the device’s available memory?

The presentation also included results on AMD Ryzen client hardware. Testing beyond smartphones matters because an architecture’s value depends on how it performs on the systems people will actually use.


Invest in Training to Support Efficient Inference

Architecture is one part of Liquid AI’s approach. Training is another.

LFM2.5 builds on the LFM2 architecture with extended pretraining and reinforcement learning. Liquid AI’s model card for LFM2.5-1.2B-Instruct specifies that pretraining was extended from 10 trillion to 28 trillion tokens.

The reasoning is straightforward: more training can improve a compact model’s capabilities while keeping its parameter count fixed.

Lechner framed this as prioritizing inference efficiency. The investment happens during development, with the goal of producing a capable model that remains practical to run repeatedly on constrained hardware.

This connects on-device AI back to training infrastructure. Developing compact models still requires experiments, evaluation, and substantial training compute. A model’s small deployment footprint does not reflect all the work required to create it.Liquid AI continued this approach with its August 2026 updates. LFM2.5-2.6B was pretrained on approximately 34 trillion tokens, then refined through supervised fine-tuning, distillation from specialist models, and reinforcement learning for multi-step tool use. The company also released updated four-bit versions of several LFM2.5 models, using additional training to help preserve quality while keeping memory requirements low. These developments reinforce the same principle: investing more during training can help compact models deliver stronger results on everyday hardware.


Software Support Shapes the Results

One of the most useful questions came during the Q&A: could the search favor architectures simply because their software kernels were already well optimized?

Kernels are the software implementations of the operations a model performs on hardware. If one operation has mature, optimized software and another does not, measured performance can favor the better-supported option.

Lechner acknowledged that tradeoff. Liquid AI deliberately constrained its design space to include components similar to those used in established architectures such as Llama. Supporting many new operators across AMD, Qualcomm, and other hardware would create a significant engineering workload for a small team.

That choice helped keep the resulting models practical to deploy. It also means the search results reflect the software implementations available during testing. Lechner said the company was exploring a broader design space for future models.

For developers, the implication is practical: evaluate the architecture alongside the software that will run it. Better kernels can change which designs perform well on a given device.


Different Devices Need Different Models

Liquid AI applies this approach across a family of models and modalities. The presentation covered language, vision-language, audio-language, and embedding models, reflecting the range of tasks applications need to support.

It also showed that local deployment extends beyond small phone models. Lechner discussed a mixture-of-experts model with 24 billion total parameters and roughly 2 billion active per token. Using only a subset of parameters for each token reduces computation, while the full model still has a larger weight-storage requirement.

The common goal is to fit the application and its hardware. A model suited to a laptop may exceed the resources available on a smaller device. An application processing text may have different needs from one handling images or audio.

Return to the devices around you. Each has a job to do, a limited set of computing resources, and its own expectations for connectivity and response time. Those requirements provide a useful starting point for AI development.

For builders, Lechner’s presentation offers a clear approach: define where the model will run, what it needs to do, and how quickly and reliably it needs to respond. Then evaluate the architecture and its software against those requirements. Liquid AI’s work shows how making those decisions early can expand where useful AI runs.

Hear Mathias Lechner explain Liquid AI’s approach to efficient model design.


TensorWave Cloud: Engineered for AI Velocity

TensorWave is the industry’s premier AMD-exclusive cloud provider, delivering memory optimized AI infrastructure for training, fine-tuning and inference. 

Our platform combines the raw computational power of AMD's latest GPU architectures with innovative liquid cooling technology and ultra-high-speed networking, creating an optimized environment for the most demanding AI workloads. From large language model training to real-time inference, TensorWave provides the performance, scalability, and cost-efficiency that modern AI initiatives demand.

Ready to get started? Connect with sales. 

Edge AI Starts With the Model Architecture

Product

Solutions

Resources

Company

Product

Resources

Solutions

Company