AIGenerated
EN · IT · ES

How Meta built TLX to help regular engineers squeeze more speed out of AI hardware

A new tool lets everyday AI developers write GPU code that matches specialist-level performance, powering the systems behind Facebook, Instagram and other services.

A figure from the paper "TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments".
Figure: Guan et al., arXiv:2605.10905, CC BY 4.0 (licence)
How it works

A programming extension that lets regular AI engineers write fast GPU code without needing hardware expertise.

  1. Spot slow parts in GPU code
  2. Rewrite the hot code using TLX
  3. Tell processor groups to share work
  4. Get specialist-level speed in hours

What you could do with it

  • An ML engineer at Meta could profile a GPU workload, identify a bottleneck, and rewrite the hottest kernel in Triton with TLX to speed it up, finishing the optimization in hours rather than days 1.
  • A regular developer could write an attention kernel for a large language model and explicitly coordinate which GPU warp groups move data asynchronously while others crunch numbers, producing specialist-level performance without writing thousands of lines of low-level code 1.
  • Meta could deploy TLX-written kernels across its production AI training and inference pipelines, powering systems behind News Feed recommendations, chatbots, and large-scale LLM pretraining on both NVIDIA and AMD GPUs 1.

Figure from the paper: Code snippets and technical diagrams from the TLX paper

Figure: Guan et al., arXiv:2605.10905, CC BY 4.0

What the problem was

Modern graphics cards (GPUs) have become incredibly complex. Instead of just doing raw math faster, they now include special hardware features designed for moving data around quickly and coordinating different parts of the chip working together at once. NVIDIA's Hopper and Blackwell architectures, for example, introduced things like warp specialization — where different groups of processors handle different jobs simultaneously — and dedicated engines for shuttling data between memory and computation units.

The trouble is that traditional programming languages for GPUs either gave developers too much detail (requiring hundreds of lines of manual code to coordinate all these new features) or hid everything behind automatic optimizations that couldn't exploit the newest hardware. Meta found itself in a gap: Triton made it productive for engineers to write optimized GPU code, but modern chips demanded coordination patterns that the compiler alone couldn't figure out 1.

At a glance

Specialist CUDA vs. regular-engineer TLX + Triton

Old way (CUDA)New way (TLX + Triton)
Who could do itHardware specialists onlyAny ML engineer
How long to optimizeWeeksHours
Performance achievedSpecialist-levelMatches specialist-level
Language complexityThousands of lines of low-level codeManaged, explicit control
GPU coverage at MetaNVIDIA focusedNVIDIA and AMD

Figures from Meta TLX paper and reported adoption, CC BY 4.0

How TLX fills the gap

TLX sits between the simplicity of Triton and the detailed control needed for modern GPUs. Rather than forcing engineers to write thousands of lines of low-level code or wait for compiler writers to catch up with each new hardware generation, TLX adds explicit but manageable ways to describe how different groups of GPU processors should cooperate 2.

In practice this means an engineer can write a GPU routine for attention calculations — the core operation behind large language models — and precisely tell the GPU which parts should move data asynchronously while others crunch numbers. The result is kernels that match what specialist teams used to spend weeks building, but that a regular ML engineer can produce in hours 1.

Why Meta started releasing it

At Meta, the shift toward Triton has been dramatic. Over a ten-month period, the number of GPU kernels written in Triton grew roughly 3.8 times, compared with about two times growth for the older CUDA language. Kernel optimization — once the job of a small group of hardware specialists — became part of the normal toolkit for any ML engineer at the company 1.

TLX powers production AI training and inference workloads across Meta, including recommendation systems and large-scale language model pretraining. It works across both NVIDIA and AMD GPUs. The code is open-sourced, meaning other companies and researchers can use and improve it 12.

The underlying idea is straightforward: if every new generation of chip introduces new ways to organize work, then giving programmers clear, consistent ways to describe that organization — without drowning them in hardware-specific details — lets them keep pace with the hardware 2.

How good is it

2. According to Meta, kernels written with TLX achieve performance on par with leading-edge implementations, but need only a small amount of development work 1.

Key facts

  • TLX is an extension to Meta's Triton compiler that adds explicit GPU warp-group and cluster-level control
  • Triton kernel usage at Meta grew 3.8x versus 2.0x for CUDA over ten months
  • TLX supports NVIDIA and AMD GPUs and powers production training and inference workloads at Meta
  • The approach lets regular ML engineers write specialist-level GPU kernels in hours instead of days
  • TLX code is open-sourced on GitHub under facebookexperimental/triton

Sources

  1. How We Built TLX: A Hardware-Native GPU Compiler for Production ML at Scale — engineering.fb.com
  2. TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments

If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work

Keep reading