Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment

Phantom traffic jams on highways—stop-and-go waves with no clear cause—waste energy and increase accident risk. Traditional solutions like ramp metering or variable speed limits require expensive infrastructure. Autonomous vehicles (AVs) offer a flexible alternative, but their effectiveness depends on intelligence. Reinforcement learning (RL) provides a method for AVs to learn optimal driving strategies through trial and error.

The team's core hypothesis: even a small fraction of vehicles (e.g., 5%) using RL-controlled driving strategies can break the oscillation cycle of human driving, thereby improving overall traffic flow. Simulations confirmed this: in data-driven environments, these RL controllers reduced overall fuel consumption by 20%.

After learning in simulation, the RL controllers faced the sim-to-real transfer problem. The team built a simulation environment using real-world data from Tennessee's I-24 highway, allowing AVs to learn how to smooth oscillations caused by human driving trajectories. They then deployed the trained controllers on 100 regular passenger cars during morning peak traffic. This experiment, called 'MegaVanderTest', is the largest mixed-traffic smoothing trial to date.

During deployment, the controller relied only on three local measurements: the vehicle's own speed, the preceding vehicle's speed, and the distance between them, using standard radar sensors. This design allows the controller to be integrated into most modern adaptive cruise control systems without needing V2X or roadside infrastructure.

The RL controller's behavior is entirely guided by a reward function. The core challenge: how to make AVs learn to simultaneously smooth traffic, save energy, ensure safety, provide comfort, and conform to human driving norms? If energy saving is overemphasized, the AV might stop in the middle of the road (the most energy-efficient state); if ignoring human-following behavior, it could cause discomfort or danger for nearby drivers.

To address this, the researchers introduced dynamic minimum and maximum spacing thresholds, ensuring the AV optimizes energy within safe boundaries. Additionally, the reward function not only considers the AV's own fuel consumption but also penalizes fuel consumption of vehicles behind it, preventing 'selfish' driving. This behavior shaping led the AVs to maintain slightly larger following distances than humans, better absorbing deceleration waves from ahead.

Simulation results show that in the most congested scenarios, less than 5% AV penetration achieved a 20% reduction in fuel consumption for all vehicles. This figure comes from replaying real I-24 traffic data. However, simulations differ from reality: human drivers may adapt to AV behavior, altering the effect. Moreover, specific fuel savings from the field test have not yet been disclosed in this article, only 'significant improvement' is mentioned.

Another uncertainty is scalability. While 100 vehicles were successfully tested on a single highway, is local information sufficient for more complex multi-lane and interchange scenarios? Additionally, the RL controllers were trained on traffic patterns from a specific highway; would transferring to other roads require retraining? These questions await further research.