This blog post introduces GRASP, a gradient-based planner for learned world models that enables robust long-horizon planning. It addresses issues like ill-conditioned optimization and bad local minima by using virtual states, stochastic exploration, and gradient reshaping. The method improves planning with high-dimensional vision models.
SynthePulse Insight · AI deep reading
Version 1 · 1 source
As world models grow more powerful, long-horizon planning remains fragile. GRASP aims to make gradient-based planning robust in long-horizon tasks by lifting trajectories, injecting randomness, and reshaping gradients.
As world models scale up, they begin to act like general-purpose simulators, but having a powerful predictive model does not automatically make it useful for control or planning. In practice, long-horizon planning with modern world models remains fragile: optimization becomes ill-conditioned, non-greedy structures cause poor local minima, and high-dimensional latent spaces introduce subtle failure modes.
The first pain point is that long-horizon rollouts create deep and ill-conditioned computation graphs. When backpropagating through time (BPTT), derivatives with respect to early actions involve products of Jacobians whose condition numbers grow exponentially with time, leading to exploding or vanishing gradients.
The second pain point is that the loss landscape is non-greedy and full of traps. In short horizons, greedy solutions are often sufficient, but long-horizon tasks require non-greedy behaviors such as going around walls, repositioning, or backtracking to choose a better path. The dimensionality of the optimization space grows with the horizon, further expanding the space of local minima.
The core idea of GRASP is to treat dynamics constraints as soft constraints, optimizing a penalty function that acts on both actions and states. This collocation method has precedents in the planning/robotics literature.
The lifted formulation shares the same global minimum as the original rollout objective (both are zero when the trajectory is dynamically feasible), but the optimization landscape is drastically different. It brings two direct benefits: each world model evaluation depends only on local variables, so all T terms can be computed in parallel over time, greatly accelerating long-horizon planning; and there is no longer a need to backpropagate through a single deep T-step composition, because the product of Jacobians now decomposes into a sum, e.g., the derivative with respect to a0 becomes 2(F_theta(s0,a0)-s1).
Directly optimizing states also aids exploration, as one can temporarily traverse non-physical domains to find the optimal plan. Collocation allows direct perturbation of states and more efficient exploration of intermediate points.
GRASP achieves robust long-horizon planning through three key mechanisms: first, lifting trajectories into virtual states to parallelize optimization over time; second, adding randomness directly to state iterations to promote exploration; and third, reshaping gradients so that actions receive clear signals while avoiding fragile state-input gradients through high-dimensional visual models.
Together, these mechanisms address the ill-conditioned optimization, non-greedy local minima, and failure modes in high-dimensional latent spaces that plague long-horizon planning.
This article is based on a technical post from the BAIR blog, which is a self-report by the research team and does not provide independent verification or benchmark results. All statements about GRASP's performance are claims by the authors, not confirmed facts.
GRASP offers a more robust gradient-based approach to long-horizon planning through trajectory lifting, stochastic exploration, and gradient reshaping, but its actual effectiveness still requires independent validation.
Primary report
Primary source