Safe Model-Based Off-Policy Reinforcement Learning for Eco-Driving in Connected and Automated Hybrid Electric Vehicles
Deep Reinforcement Learning (DRL) has recently been applied to eco-driving to intelligently reduce fuel consumption and travel time. While previous studies synthesize simulators and model-free DRL (MFDRL), this work proposes a Safe Off-policy Model-Based Reinforcement Learning (SMORL) algorithm for eco-driving. SMORL integrates three key components, namely a computationally efficient model-based trajectory optimizer, a value function learned off-policy and a learned safe set. The advantages over the existing literature are three-fold. First, the combination of off-policy learning and the use of a physics-based model improves the sample efficiency. Second, the training does not require any extrinsic rewarding mechanism for constraint satisfaction. Third, the feasibility of trajectory is guaranteed by using a safe set approximated by deep generative models. The performance of SMORL is benchmarked over 100 trips against a baseline controller representing human drivers, a non-learning-based optimal controller, a previously designed MFDRL strategy, and the wait-and-see optimal solution. In simulation, SMORL reduces the fuel consumption by more than 21% while keeping the average speed comparable while compared to the baseline controller and demonstrates a better fuel economy while driving faster compared to the MFDRL agent and the non-learning-based optimal controller.
- Record URL:
-
Availability:
- Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
-
Supplemental Notes:
- Copyright © 2022, IEEE.
-
Authors:
- Zhu, Zhaoxuan
- Pivaro, Nicola
- Gupta, Shobhit
- Gupta, Abhishek
- Canova, Marcello
- Publication Date: 2022-6
Language
- English
Media Info
- Media Type: Web
- Features: References;
- Pagination: pp 387-398
-
Serial:
- IEEE Transactions on Intelligent Vehicles
- Volume: 7
- Issue Number: 2
- Publisher: Institute of Electrical and Electronics Engineers (IEEE)
- ISSN: 2379-8858
- Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857
Subject/Index Terms
- TRT Terms: Autonomous vehicles; Connected vehicles; Ecodriving; Hybrid vehicles; Machine learning; Mathematical models; Vehicle dynamics
- Subject Areas: Data and Information Technology; Energy; Environment; Highways; Vehicles and Equipment;
Filing Info
- Accession Number: 01856158
- Record Type: Publication
- Files: TRIS
- Created Date: Aug 26 2022 2:55PM