It is expected that many human drivers will still prefer to drive themselves even if self-driving technologies are ready. Therefore, human-driven vehicles and Autonomous Vehicles (AVs) will coexist in mixed traffic for a long time. To enable AVs to safely and efficiently maneuver in this mixed traffic, it is critical that the AVs can understand how humans cope with risks and make driving-related decisions. On the other hand, the driving environment is highly dynamic and ever-changing, and it is thus difficult to enumerate all the scenarios and hard-code the controllers. To face up these challenges, in this work, we incorporate a human decision-making model into Reinforcement Learning (RL) to control AVs for safe and efficient operations.

We study the problem of multi-agent maneuver-level decision-making in mixed-autonomy environments and investigate how AVs can learn cooperative policies that are robust to different scenarios and driver behaviors safely. Our altruistic AVs learn the decision-making process from experience, considering the interests of all vehicles while prioritizing safety and optimizing a general decentralized social utility function. We expose the settings for our Multi-Agent Reinforcement Learning (MARL) problem in which transfer learning and domain adaptation are more feasible, and conducted a sensitivity analysis under different HVs’ behaviors.