***Note: This is the first version of the programming codes. Some comment lines and parameter names will still be 
updated. Please always refer to the latest version.

This repository contains programming codes which were used in the master thesis titled "Multi-leader Adaptive 
Cruise Control Systems considering Sensor Measurement Uncertainties based on Deep Reinforcement Learning". The 
project was conducted at Delft University of Technology. The thesis report can be found at the TU Delft education 
repository. The users are suggested to carefully follow chapter 3 in the report, which describes the DRL training 
setup in detail, and section 4.3, which introduces the simulation evaluation framework.

##############################################################################################

These programming codes are written in Python 3.7.11. Two libraries OpenAI gym 0.19.0 and Stable baseline 2.10.2
are used.

To run the programming codes, the user is suggested to first divide several segments and run each segment step by 
step.

##############################################################################################

There are four .py files which are used to train the DRL models: 
- ACC1_training
- ACC2_training
- LSTM_ACC1_training
- LSTM_ACC2_training

Each file outputs a trained optimal DRL model, which can be used in the simulation codes. In LSTM_ACC1_training.py, 
standard deviations of 0.2 m and 0.2 m/s are included for the distance gap and speed measurements. In 
LSTM_ACC2_training.py, the standard deviations are 0.5 m and 0.5 m/s, respectively. These settings can of course 
be changed according to users consideration.

In these files, function reset() and step() in the gym environment are used in the training process, while
reset_test() and step_test() are files which can be used to quickly verify the models.

##############################################################################################

There are four kinds of .py files which can be used to simulate the performance of the DRL models:
- one_ACC_simulation, two_ACC_simulation
- one_ACC_noisy_simulation, two_ACC_noisy_simulation
- one_ACC_KF_simulation, two_ACC_KF_simulation
- one_ACC_LSTM_simulation, two_ACC_LSTM_simulation

Each of them has two files, representing systems with one or two leaders. The first cateogory uses the models 
trained in ACC1_training.py and ACC2_training.py. The second category adds measurement noise to the environment. 
The third category additionally implements a KF function to process the noisy measurement. The fourth category 
uses the models trained in LSTM_ACC1_training.py and LSTM_ACC2_training.py.

All the simulation files test a scenario with a traffic disturbance (the leader in the platoon decelerate with 
-3 m/s^2 between 3 s and 7 s, accelerate with 1.5 m/s^2 between 12 s and 20 s).

The simulation files for the two-leader systems include two customized environments, representing the two 
controllers, respectively. Different from the training files, the simulation files change the sensor delay from 
0 s to 0.2 s to reflect the real sensing situation and change the initial position of the leading vehicle from 
500 m to 800 m to accommodate more following vehicles. Two customized functions, reset_test_multi() and step_test_multi(), 
are built in these environments.

For further analysis, the recorded acceleration, speed, and position information of every vehicle in the platoon 
can be extracted. This repository does not aim to provide such codes for the users.

