A new engine makes small AI models respond in under 50 milliseconds

What you could do with it

• A payments company could swap its old fraud-check models for a generative AI model and still get answers in under 50 milliseconds, keeping the user flow smooth ¹.
• An e-commerce site like Shopify could run real-time search improvements on live traffic with sub-20 ms inference, giving shoppers faster, smarter results ¹.
• An adtech platform could rewrite user search queries on the fly inside a 50 ms latency budget, improving ad relevance without slowing down the page ¹.

What RIPL actually does

Most AI serving tools like vLLM are built for long, batched jobs — think chatbots or agentic workflows where a few seconds is acceptable. RIPL is built for the opposite: search ranking, fraud checks, payment authorization — paths where every millisecond counts ¹.

Liquid AI says its LFMs (Liquid Foundation Models) are hybrid architectures mixing convolution blocks with GQA (grouped-query attention) blocks. The LFM2.5-1.2B model has 16 layers; the LFM2.5-2.6B has 30 ¹. RIPL optimizes the full path from request arrival to completed response, not just inference speed .

How fast is it?

Liquid ran benchmarks on a single NVIDIA B200 GPU ¹. Here are the numbers the company reports:

• Processing 32 input tokens into 32 output tokens at roughly 1,000 requests per second: the 1.2B model takes 34.6 ms while the 0.8B model takes 498 ms, giving a 14.4x speedup.
• Processing 256 input tokens into 128 output tokens at about 400 requests per second: the 1.2B model takes 155.8 ms compared with 4,431 ms for the 0.8B model, a 28.4x speedup.
• Processing 32 input tokens into 32 output tokens at around 1,000 requests per second: the 2.6B model takes 76.4 ms versus 586.8 ms for the 2B model, a 7.7x speedup.
• Processing 256 input tokens into 128 output tokens at roughly 300 requests per second: the 2.6B model takes 449.1 ms compared with 2,499 ms for the 2B model, a 5.6x speedup.

Liquid also claims that using the same LFM2.5-350M model, RIPL outperforms vLLM — meaning the engine itself adds speed, not just the model architecture ¹.

Real-world use cases

Liquid mapped several real-time workloads against latency budgets ¹:

• Adtech query rewriting: 32 in, 32 out, 50 ms budget
• Search intent classification: 128 input tokens, 1 output token, 20 ms budget.
• Payments risk scoring: 512 input tokens, 16 output tokens, 50 ms budget.
• Credit decisioning: 8,000 input tokens, 64 output tokens, 150 ms budget.
• Image alt text (LFM2.5-VL-450M): 512×512 image, 100 ms budget.

Who is using it?

According to Liquid AI, Shopify is deploying LFMs in production. The company says search enhancement runs with sub-20 ms inference on live traffic, using about 50% fewer parameters than alternatives and delivering 2 to 10× faster inference ¹.

Liquid AI stresses that RIPL is enterprise-only and all figures are self-reported ¹. The LFM models use the LFM Open License v1.0, while the Qwen baselines use Apache 2.0 .

How good is it

Liquid AI reports p95 latency 5.6 to 28.4 times lower than Qwen3.5 running on vLLM, measured on a single NVIDIA B200 GPU ¹. In the biggest gap, a 1.2B model on RIPL took 155.8 ms versus 4,431 ms for a smaller 0.8B model on vLLM, a 28.4x difference . Even when using the same LFM2.5-350M model, RIPL outperformed vLLM, meaning the engine itself adds speed beyond just the model design . Shopify is already running LFMs in production search with sub-20 ms inference on live traffic, according to Liquid AI .

Key facts

• RIPL returns answers in under 50 ms for short requests on a single NVIDIA B200
• Up to 28.4× lower p95 latency than Qwen3.5 on vLLM, per Liquid's benchmarks
• Shopify reportedly runs LFM search inference at sub-20 ms on production traffic
• RIPL is enterprise-only; all benchmark figures are self-reported by Liquid AI
• LFM2.5 models range from 230M to 2.6B parameters and use a hybrid conv+GQA design

Sources

1. Liquid AI Launches RIPL, a Real-Time Inference Engine for Sub-50 ms LFMs — marktechpost.com