A reinforcement learning framework to enhance the ramp merging capabilities of autonomous vehicles

Figure explaining the team’s Reinforcement Learning Pipeline. They first initialize rollouts from random initial states and run an optimization on the set of collected trajectories to train a policy for each epoch. Credit: Udatha, Lyu & Dolan.

While many automotive companies are now invested in the development of self-driving cars, the vehicles created so far have not yet attained the safety levels necessary for them to be deployed on a large-scale. For this to happen, the vehicles will need to be able to tackle a wide variety of challenges on the road both safely and effectively.

Researchers at Carnegie Mellon University have recently developed a reinforcement learning (RL)-based framework that could help to improve the performance of autonomous vehicles in ramp merging scenarios, instances where vehicles on a ramp road are deviated onto a main road. Their framework, presented in a paper pre-published on arXiv, could potentially help to improve the safety of autonomous vehicles in these particularly salient times, reducing the risk of accidents.

“Prof. John Dolan’s lab at CMU has been working on various autonomous driving applications for quite some time,” Soumith Udatha, one of the researchers who developed the model, told TechXplore. “The application that we focused on in this paper is that of freeway merging because of the challenges involved with vehicles at high speeds, drivers with various styles, and uncertainties involved.”

The overreaching objective of the research efforts by Udatha and his colleagues is to improve the safety of autonomous vehicles. In their recent paper, they specifically tried to devise a framework that could effectively capture ramp merging scenarios and plan a vehicle’s actions based on its analyses of any uncertainties and possible risks.

“RL models interact with an environment and collect data to optimize their rewards, but this data exploration encounters some problems when deployed in real-world settings,” Udatha explained. “This is partly because not all the states that the agent encounters are safe. We constrained our RL policy with control barrier functions (CBFs) to ensure safety at a specified distance. So, with the environmental constraints, we ignore the unsafe states and enhance a system’s ability to learn how to navigate.”

CBFs are a class of fairly new computational methods designed to enhance the safe control of autonomous systems. CBFs can be directly applied to different optimization problems, including ramp-merging. Despite their promising qualities, the optimizations they perform do not account for the data gathered by a system while it is exploring an environment. RL methods can help to fill this gap.

“We found that our algorithm can be extended to both offline and online RL environments,” Udatha said. “Since we have enormous amounts of data now for offline RL, training on offline datasets can eventually lead to better models. With our metrics, we also discovered that the inclusion of Probabilistic CBFs as constraints provides better safety, as it accounts for the driver uncertainty to an extent.”

Udatha and his colleagues tested their framework in a series of tests, using the online version of the CARLA simulator developed by a team of researchers at Intel Labs and the Computer Vision Center in Barcelona. In these simulations, their approach achieved remarkable results, highlighting its possible value for enhancing the safety of autonomous vehicles during ramp merging.

“We now plan to extend our research by training our model to merge an autonomous vehicle with multiple vehicles in a scene with driver uncertainties,” Udatha added. “We also found that a standard benchmark to compare various approaches for ramp-merging is currently missing, so we are concurrently trying to establish a ramp-merging benchmark for NGSIM, a freeway dataset released by NHTSA on the US I-80 and US 101 freeways.”

More information:
Soumith Udatha et al, Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging, arXiv (2022). DOI: 10.48550/arxiv.2212.00618

Journal information:
arXiv


© 2022 Science X Network

Citation:
A reinforcement learning framework to enhance the ramp merging capabilities of autonomous vehicles (2022, December 22)
retrieved 22 December 2022
from https://techxplore.com/news/2022-12-framework-ramp-merging-capabilities-autonomous.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

For all the latest Technology News Click Here 

 For the latest news and updates, follow us on Google News

Read original article here

Denial of responsibility! TechNewsBoy.com is an automatic aggregator around the global media. All the content are available free on Internet. We have just arranged it in one platform for educational purpose only. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials on our website, please contact us by email – [email protected]. The content will be deleted within 24 hours.