Tencent Open-Sources Three Embodied Foundation Models; Chief Scientist Zhang Zhengyou Explains How 'Three-Layer Brain' Solves Robot Latency

Zhang Zhengyou, director of Tencent Robotics X Lab, pointed out that a pain point of current VLA models is that visual understanding and action generation are often executed at the same frequency. However, in the physical world, different types of information have vastly different latency tolerances: tactile feedback is about 1 millisecond, low-level motion control runs at 500Hz to 1000Hz, while running large models at high frequencies is computationally prohibitive.

To address this, Tencent decomposes embodied intelligence into three layers that operate simultaneously but at different frequencies: the top cognitive layer wakes on demand, handling complex task understanding and deep reasoning; the middle perception-action layer continuously receives multimodal information at about 15Hz and quickly adjusts actions; the bottom execution layer handles emergencies such as collisions and imbalances at a higher frequency.

Zhang Zhengyou believes this layered architecture may remain stable for a considerable period because it corresponds not to the performance limitations of any particular model generation, but to the inherent time-scale differences in the physical world. He emphasizes 'layered operation, closed-loop training'—different models assume different responsibilities and frequencies during inference, but can still be jointly optimized during training.

The cognitive model RxBrain released by Tencent not only generates task steps and constraints using language, but also predicts the state the physical world should be in after each subtask is completed—that is, it generates in advance 'what it should look like after completion.' This effectively incorporates visual state into the information pathway, broadening cognitive bandwidth.

Zhang Zhengyou stated that language is only a low-bandwidth channel for cognition, and visual information is often much richer than textual information. For example, in the task 'set the table,' describing the distances between plates and wine glasses in words requires a lot of text and can be ambiguous, while showing the robot a photo of the completed setup provides more direct information.

RxBrain is defined as an 'embodied world cognition model,' combining Hy-Embodied-VLM (understanding the world) and Hy-Embodied-VLA (over 10,000 hours of demonstration data, ranked first overall in the RoboDojo benchmark), aiming to fill the missing information link between cognition and the body.

Zhang Zhengyou emphasized: 'A demo that scores 80 or 90 points but never gets deployed is essentially zero.' Successfully grasping an object once in the lab is completely different from completing tens of thousands of tasks on a production line. In a factory, models must face challenges such as frequent SKU changes, random placement of materials, and cycle time requirements.

Tencent disclosed that Hy-Embodied-VLA has entered a household chemical factory for production testing. On a production line with high mix, low volume, and frequent SKU iterations, the operation success rate exceeded 95%, with a cycle time faster than 6 seconds per piece. For new SKUs, the time allowed for data collection and model post-training was less than 3 days.

Chen Yudong, Tencent Cloud's director of heterogeneous computing R&D, revealed that embodied intelligence customers are beginning to demand computing resources on the scale of thousands or tens of thousands of GPUs, with related demand growing by about 200% to 300%, and some models even iterating a new version every two to three days.

Tencent chose elderly care as a technology testbed. The 'Xiao Liu' robot demonstrated can perform massage actions using both arms, but has only learned four types of dual-arm coordinated techniques. Zhang Zhengyou emphasized that Tencent does not plan to sell elderly care robot hardware; they chose elderly care because physical contact with humans concentrates the most difficult problems in embodied intelligence.

In industrial gripping, a slightly stronger force might still complete the task, but when assisting an elderly person or performing a massage, the force and posture cannot deviate significantly. Visual judgment, tactile feedback, and force control must be tightly integrated, and safety cannot be just 95%. Zhang Zhengyou stated: 'In unstructured environments like homes, safety must be 100% guaranteed.'

The elderly care scenario is an extreme test of the layered architecture: the high-level model must understand human intent, the middle-level system adjusts actions based on posture and muscle state, and the low-level system stops immediately upon abnormal force. Zhang Zhengyou said: 'Elderly care is not my personal preference, but it is very important.'