H-JEPA Lets AI Plan at Multiple Levels of Abstraction
A new hierarchical approach lets AI reason about short-term moves and long-term goals at the same time.

An AI planning method that organizes thinking into layers so machines can handle both small steps and big goals at once.
- Top level picks an abstract goal.
- Lower levels break it into smaller steps.
- Lowest level turns steps into real actions.
- All levels train together at once.
Building a mental map that thinks in layers
H-JEPA is a new AI technique that helps machines plan over long stretches of time by organizing their thinking into separate layers of detail. Instead of trying to handle every small movement and big-picture goal in one giant brain, H-JEPA splits the job into a hierarchy where each level handles its own scale of time.
The idea builds on an earlier system called JEPA, short for Joint-Embedding Predictive Architecture. A JEPA model learns to predict what comes next by comparing encoded versions of the current situation and future states 1. But standard JEPA works at a single timescale — it either predicts the near future in fine detail or the distant future at a high level, but not both well at the same time.
H-JEPA fixes this by stacking multiple JEPA models on top of each other. Each higher level learns to look farther ahead in its own abstract space. Fast-changing details like leg movements get filtered out at the top, while slower, more important information like overall position stays clear 2.
How the hierarchy works
Planning with H-JEPA flows from top to bottom. The highest level sets an abstract goal, like "reach the exit of the maze." Its predictions then become subgoals for the level below, which breaks them into smaller steps. The lowest level finally turns those steps into actual actions 2.
All the levels are trained together in one shot. Gradients from the top-down objectives flow back through the lower encoders, so every layer learns to serve the whole hierarchy 3. A regularization method called SIGReg prevents the system from collapsing into a useless representation during training 2.
This layered design mirrors how humans plan. You might decide to drive to a city, then figure out which roads to take, then steer the wheel — each step lives at a different level of detail.
From the official announcementLearning abstractions
Better results in simulated worlds
The researchers tested H-JEPA across four simulated environments, including navigation and manipulation tasks 3. On Visual AntMaze, a three-level hierarchy boosted success rates from 18 percent to 73 percent, while using less planning compute than a flat JEPA model 2.
Further analysis showed the improvements came from two sources: breaking planning across time scales and letting higher levels form cleaner goal representations 3. Adding inverse-dynamics supervision also improved performance on real-robot videos from the DROID dataset, with better offline planning fidelity at lower compute cost .
Code for H-JEPA has been released as a PyTorch implementation, including a masked-patch self-supervised learning variant and an end-to-end training option 4.
From the official announcementPlanning in action
From the official announcementReal-world videos: DROID
Key facts
- H-JEPA stacks multiple JEPA models into a hierarchy for multi-scale planning
- On Visual AntMaze, a three-level hierarchy raised success from 18% to 73%
- Each higher level predicts farther ahead in its own learned latent space
- The approach also improved planning on real-robot videos from the DROID dataset
- All levels are trained end-to-end with gradients flowing through the hierarchy
Sources
If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work
Keep reading
Apps and productsGoogle opens TorchTPU, a native PyTorch backend for cloud TPUsHerald of AIhttps://github.com/soumith, CC BY-SA 4.0, via Wikimedia Commons
Google opens TorchTPU, a native PyTorch backend for cloud TPUs
A new open-source stack lets developers run PyTorch models directly on Google's TPU chips, replacing the older PyTorch/XLA system.
How it worksLeCun's Startup Unveils H-JEPA A Hierarchical World Model for Visual PlanningHerald of AI
LeCun's Startup Unveils H-JEPA A Hierarchical World Model for Visual Planning
AMI releases its first research with full code and weights open-sourced, showing that stacking prediction layers dramatically improves long-term planning in visual tasks.
Business and rulesMirendil raises $1 billion to use AI agents for faster model developmentHerald of AI
Mirendil raises $1 billion to use AI agents for faster model development
A new startup founded by former Google DeepMind and Anthropic researchers says its automated platform lets small teams build large AI models quickly and cheaply.