ROMA lets robots actively explore objects through physical interaction

When humans judge how heavy a box is, they often do not stare at it but pick it up and weigh it in their hand; to find out whether there is something inside, they shake it; to confirm whether it is soft or hard, they pinch it. ROMA wants to hand this set of actions to a multi-sensor robot: when existing observations are insufficient to complete a task, the robot needs to judge for itself what information is missing, which object to interact with, how to interact, which modality to focus on, and when it can stop.

This is precisely the embodied research problem ROMA attempts to address. It integrates perception across four modalities—vision, audio, touch, and force—into the ROMA-7B model, enabling the robot to form a closed loop of "reasoning—interaction—feedback—re-reasoning," rather than passively accepting one-time sensory input. It should be noted that this closed loop is currently limited to the four modalities of vision, audio, touch, and force; the source provides no evidence for other modalities.

For a robot to learn active exploration, the prerequisite is that it knows what information different interactions will bring. To this end, the research team built ROMI-2K, covering nearly 2,000 everyday objects and contents combinations, collecting data around 6 basic physical interactions—lifting, pressing, colliding, pinching, shaking, and rotating—with collection scenarios covering both handheld devices and a real tabletop robotic arm.

Each interaction with an object synchronously collects observation information across four modalities—vision, audio, touch, and force—while also providing detailed annotations of physical properties such as material, hardness, roughness, texture, and contents. On this basis, the team further built scene-level multimodal active perception question-answering data, enabling the model to learn to select appropriate interaction behaviors and perception modalities according to the current task in order to obtain the missing key evidence. In other words, the value of ROMI-2K is not just "a large amount of data," but rather aligning interaction actions, modality observations, and physical properties within the same record.

The key to active perception is that the observation obtained from one interaction affects the choice of the next interaction. The example given by the source is: when shaking an object to determine whether it is empty, if a clear sound of contents is heard, no further interaction is needed; if there is no sound, one cannot directly rule out that the interior is filled, and it is still necessary to weigh it in hand to confirm the weight. This process, continuously linked by reasoning, interaction, and feedback, is summarized as a "perception chain."

Based on this concept, the research team built ROMA Bench on the real tabletop subset of ROMI-2K, containing 2,100 scene-level active perception tasks, covering three categories: Single-Chain single-attribute reasoning, Multi-Chain multi-attribute long-chain reasoning, and Intent-Driven implicit intent reasoning, involving multiple physical attributes such as hardness, roughness, texture, contents, material, and weight. The task formats are multiple-choice, true/false, and ranking questions. In this way, evaluation is no longer "perform one interaction and then answer a question," but instead requires complete target selection, action selection, modality selection, and continuous reasoning.

ROMA Bench examines whether the model "knows how to explore," while the ROMA system aims to push this capability into the physical world. The research team built a real-world multimodal physical interaction system, connecting the four perception modalities—vision, audio, touch, and force—with a robotic arm, and designed a unified physical interaction interface so that the model can directly output interaction commands for real objects.

At the execution level, ROMA programs grasping and the six basic interaction actions in ROMI-2K, and based on the AnyGrasp grasping strategy, combines point cloud completion, grasp filtering, and enhancement mechanisms to improve stable grasping capability in the real world. On the model side, based on Qwen 2.5-Omni, ROMA-7B is trained through multi-sensor joint alignment and active perception scenario fine-tuning; it decides which object to grasp, what interaction to perform, which modalities are needed, and when to stop exploring, and dynamically adjusts subsequent actions based on the new data from each interaction. It should be pointed out that the source does not provide quantitative results on the degree of improvement in stable grasping capability.

On the 2,100 multiple-choice, true/false, and ranking questions of ROMA Bench, the authors report that ROMA-7B achieves an overall task success rate of 72.9%, with overall performance basically on par with GPT-6 Astra and clearly better than GPT-5.4 and Gemini 3.5 Flash. The source also states that GPT-6 Astra performs outstandingly on single-step, visually inferable attributes, while ROMA-7B has an advantage in Multi-Chain tasks that require continuous interaction and comprehensive multi-modal feedback, especially audio, clearly surpassing GPT-6 Astra; however, the source does not provide a verifiable numerical value for the extent of this surpass.

In real-world experiments, the evaluation covered 8 multi-object scenes and 132 open-ended questions, and the authors report that ROMA-7B achieves an overall success rate of 61.4%, approaching GPT-6 Astra and greatly leading other frontier large models. The source also mentions that Qwen 2.5-Omni and Qwen 3-Omni, which have similar parameter scales but were not fine-tuned on ROMI-2K, perform clearly insufficiently in target selection, interaction instruction following, and multimodal active reasoning; based on this, the authors believe that ROMI-2K's synchronous multimodal interaction data is the key training foundation for ROMA-7B to achieve capability approaching GPT-6 Astra at a 7B scale.

These figures and comparison conclusions all come from the paper's authors' self-reports; the source provides no third-party independent reproduction and does not elaborate on evaluation protocol details. Therefore, they should be read as results reported by the authors, not as conclusions that have been externally verified.

ROMA currently mainly revolves around 6 basic physical interactions, and the authors say this is only a starting point. The source lists richer interaction skills, unification of active perception and manipulation strategies, multi-turn dialogue and long-horizon reasoning, persistent object memory across scenes, and active perception for different robot embodiments as explorable outlooks, rather than capabilities already achieved.

The evaluation scope is likewise limited: ROMA Bench is built on the real tabletop subset of ROMI-2K, and the real-world evaluation covers only 8 multi-object scenes and 132 open-ended questions. The source provides no evidence of ROMA's active perception performance in non-tabletop scenes or on different robot embodiments, nor does it give quantitative results on the improvement in real-world stable grasping capability. These gaps determine the applicable scope of the current conclusions.