*** Code underlying the publication: Safe, Efficient, Comfort, and Energy-Saving Automated Driving Through Roundabout Based on Deep Reinforcement Learning ***
Authors: H. Yuan, P. Li, B. Van Arem, L. Kang, H. Farah and Y. Dong

TU Delft, Faculty of Civil Engineering and Geosciences, Department of Transport and Planning
Beijing Jiaotong University, School of Traffic and Transportation

Corresponding author: Yongqi Dong
Contact Information:
qiyuandream@gmail.com


***General Introduction***
This is the code related to the publication:
H. Yuan, P. Li, B. Van Arem, L. Kang, H. Farah and Y. Dong, "Safe, Efficient, Comfort, and Energy-Saving Automated Driving Through Roundabout Based on Deep Reinforcement Learning," 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain, 2023, pp. 6074-6079, doi: 10.1109/ITSC57777.2023.10422488. 

keywords: {Road transportation;Deep learning;Energy consumption;Merging;Reinforcement learning;Safety;Testing},



The implementation is based on Python, Stable-Baselines3 (https://stable-baselines3.readthedocs.io/en/master/) and Highway_env simulation environment https://github.com/Farama-Foundation/HighwayEnv.


Name of the file or function 	Description
Manual control	Runs the highway environment with manual control enabled, for testing the reward function.
DDPG-1.py	Applies the DDPG algorithm, it can be used for both the racetrack environment and the intersection environment (continuous action space version).
highway_env_modified_reward	The highway environment with the reward function modified, if you let safety = 1, the model will punish crashes with a flat -20, regardless of lane position etc, if you let comfort = 1, the model will punish accelerating/braking beyond 2m/s^2
highway_env_modified_reward2	fixed assuming the acceleration = action[0]*5
racetrack_env_reward_modified.py	same changes as highway_env_modified_reward2
intersection_env_reward_modified.py	same changes as highway_env_modified_reward2
manual_control_continuous	takes step by step manual control of the car with the continuous action space, for each render you input 2 values in the range [-1,1]
DDPG-2.py	gave values to some other variables that i may have forgotten about
DDPG-3.py	added action noise, also increased learning_starts to 1000 which seems to help
highway_env_modified_reward3	Highway environment with many more added parts to the reward function, able to have the car drive in a rather straight manner
highway_env_reward_functional	Highway environment able to train the car to drive straight in a lane with the right settings


Python script files:
A2C_1.py, DDPG-3R.py, PPO2e5.py, Test_DDPG.py, TRPO 1e5.py are scripts implementing various deep reinforcement learning algorithms to train autonomous vehicles. Each script employs different algorithms like A2C (Advantage Actor-Critic), DDPG (Deep Deterministic Policy Gradient), PPO (Proximal Policy Optimization), TRPO, etc., to address the driving task at a roundabout intersection.
manual_control.py is likely a script that allows users to manually control a car in the simulation environment to test the environment's response and physical properties.