HD Map in the Loop Framework for End-to-End Autonomous Driving

Abstract. The generalized concept of "Human in the Loop" (HITL) enhances system performance by integrating human expertise into the decision-making process of agents. In a narrower sense, HITL specifically refers to human involvement in reinforcement learning (RL) through three key mechanisms: demonstration, intervention, and evaluation, each optimizing different stages of the training process. This approach effectively incorporates prior human knowledge, mitigates risks and sample bias in RL, and improves exploration efficiency and neural network convergence. However, existing HITL methods heavily rely on human experts for real-time annotations and guidance, leading to high implementation costs and operational complexity.
In the domain of autonomous driving, traditional hierarchical decision-making frameworks depend on high-definition (HD) maps for planning and navigation. Notably, the construction of HD maps inherently embeds expert knowledge, semantic rules, and constraint information. Inspired by this observation, this study introduces an innovative approach: "HD Map in the Loop" (HMITL), leveraging HD map features as a substitute for human expertise and establishing a corresponding application framework for autonomous driving. Specifically, this research systematically investigates three core aspects of HMITL in training end-to-end decision-control models: (1) imitation learning based on expert demonstrations from HD maps; (2) Method for constructing action interference and reward function guided by HD map spatial heterogeneity; and (3) Critic priority architecture relying on expert evaluations from HD map perception and features. These three dimensions are logically interrelated and collectively form the foundational framework of HMITL. By pioneering this methodological innovation, this study provides a novel solution to reducing reliance on real-time human intervention in autonomous driving while ensuring the reliability and safety of system decision-making.