LeCun's World Model Startup AMI Releases H-JEPA, Open-Sources Code and Weights

What AMI delivered this time is not a product, but a paper plus a set of loadable models. H-JEPA was released by researchers from AMI together with New York University, Paris INRIA and Brown University, positioned as a hierarchical world model for visual planning; the team simultaneously made public the code and pre-trained model weights, so researchers can directly load the model to reproduce planning experiments.

This open-source route was not improvised. As early as 2022, when he was still chief AI scientist at Meta, LeCun had already proposed the idea of hierarchical JEPA: letting models predict at different levels of abstraction and time scales, and then using hierarchical planning to translate complex goals into actions. Now this idea has become AMI's entrepreneurial direction, and H-JEPA is its first public presentation in the form of a paper, code and weights trio.

H-JEPA's approach is to stack multiple JEPAs layer by layer, letting each layer predict states after different time spans in its own representation space. Each layer is configured with three components: a state encoder extracts the representation of the current environment, an action encoder describes the changes brought by actions, and a predictor combines the current state and action to predict the future state. The lowest layer directly reads images, while higher layers further abstract based on lower-layer representations, retaining the more important information at that time scale.

The division of labor during planning is top-down: higher layers first formulate a rough plan, then hand the predicted intermediate states as subgoals to lower layers, which take over and refine step by step, ultimately generating executable actions. In the paper's experiments, one step of an adjacent higher layer corresponds to two steps of prediction in the lower layer.

In training, the layers learn jointly through end-to-end training, and different levels are connected through subgoals—the intermediate state predicted by a higher layer becomes the subgoal of the lower layer, and the lower layer transforms its own predicted state into the higher layer's representation space, compares it with the subgoal specified by the higher layer, and adjusts actions accordingly.

The most telling numbers come from the Visual AntMaze simulated maze task: the three-layer H-JEPA success rate reaches 73%, while the single-layer model is 18%. At the same time, across multiple simulation tasks, the hierarchical model also achieves better performance with less planning computation.

Researchers also observed the information trade-offs at different levels: in the maze task, higher layers gradually weaken leg posture but still retain the robot's position information. This exactly corresponds to the intuition of hierarchical planning—when planning a long journey, the height of a particular step underfoot is not important; but when actually walking down the stairs, this detail directly determines how to step.

It must be emphasized that both 73% and 18% come from the simulated maze task, not a real physical environment. Whether H-JEPA can maintain this success rate on real robot tasks outside Visual AntMaze is not supported by evidence in the source.

The paper itself provides a counterexample. In the Push-T task, the two-layer model performs relatively well, but when increased to three and four layers, the results instead decline. The paper believes this may be because shorter task trajectories limit the training data available to higher layers.

This means that "stacking more layers" is not a rule that can be extrapolated unconditionally. There is currently no general rule for choosing the optimal number of hierarchical layers, and the reason hierarchy fails in Push-T is only the paper's speculation, not a verified conclusion.

Another boundary comes from the base: H-JEPA uses LeWorldModel as the lowest layer, adding more JEPAs upward to form a structure of joint training and top-down planning. Its capabilities are therefore constrained by this underlying method.

Training a world model must solve a classic problem: representation collapse. If the encoder compresses all inputs into the same vector, and the predictor only outputs this vector, it can also obtain very low prediction error, but the model has actually learned no useful information.

H-JEPA follows SIGReg, previously proposed by LeCun's team in LeJEPA, to constrain the representation distribution and prevent different states from being compressed into the same vector. This year's LeWorldModel applied this method to world model training, and H-JEPA uses it as the lowest layer and continues stacking upward.

Restraint is needed here: how exactly H-JEPA's performance increment relative to LeWorldModel and LeJEPA should be attributed is not broken down in the source.

AMI's capital and personnel configuration is not low-key. In March this year, AMI announced the completion of $1.03 billion in financing, at a pre-money valuation of $3.5 billion. The company has LeCun as chairman, Alexandre LeBrun as CEO, Xie Saining as chief scientist, Pascale Fung responsible for research and innovation, and Michael Rabbat responsible for world models, and Rabbat is also one of the authors of this H-JEPA paper.

But research results are research results. At least judging from the current official website, what AMI displays is mainly technical directions, team and recruitment information, and no public-facing product entry point has yet been seen; the source also gives no timetable for when AMI will launch a public-facing product.

As a comparison, World Labs, which also bets on world models, reached a turning point on September 28: AMD announced it had signed an all-stock acquisition agreement with it worth about $8.2 billion. One side moves toward a combination with a chip company, while the other continues to advance along its own research route. However, the source does not explain the comparability between World Labs and H-JEPA's technical route; this comparison is more a reference for industry pacing than a judgment of technical superiority.

H-JEPA continues one of JEPA's basic propositions: completing prediction in representation space. LLMs usually learn to generate content by predicting the next token; visual JEPA instead first encodes images and videos into internal representations, then predicts the representation of the target part, without generating image pixels one by one. In world models like H-JEPA, the model also combines actions to predict representations of future states, and the planner compares the consequences of different actions accordingly to find a path toward the goal.

LeCun has always believed that understanding the world, predicting the consequences of actions and planning are capabilities necessary for moving toward human-level intelligence. In September this year, at ECCV held in Malmö, Sweden, his lecture slides once again showed that piece of advice: if you are interested in human-level AI, do not study LLMs. He then added that LLM-based AI tools are very useful and everyone is using them, but he believes that LLMs alone cannot reach human-level intelligence.

This belongs to LeCun's personal advocacy, and the source provides no verification. When placing it alongside H-JEPA's 73%, two things need to be distinguished: the former is a route judgment, while the latter is an experimental result on a simulation task, and the two cannot substitute for each other as proof.