The disorderly charging behavior of large-scale electric vehicles (EVs) can lead to prolonged queuing times, increased charging costs, unbalanced utilization of charging station resources, and aggravated voltage fluctuations in the power grid, ultimately undermining system stability. To address these challenges, this paper proposes a Hierarchical Graph Reinforcement Learning (HGRL) algorithm for optimizing EV charging navigation strategies and achieving coordinated optimization of multi-agent interests. An EV charging model that accounts for user heterogeneity is developed by incorporating mileage anxiety, temporal and economic preferences, and energy consumption. Based on this model, a multi-agent objective function is formulated to minimize users’ total time cost, economic cost, mileage anxiety, and driving energy consumption. To effectively solve the problem, the charging navigation task is modeled as a two-layer stochastic optimization problem encompassing both charging station destination selection and route planning. The proposed HGRL method is designed to overcome the curse of dimensionality in high-dimensional state spaces and the challenges of stochastic uncertainty in optimization. Simulation results demonstrate that the HGRL algorithm effectively optimizes the interests of multiple stakeholders—including EV users, charging stations, and the power grid—while providing personalized and intelligent charging navigation strategies that account for user heterogeneity, and exhibits strong adaptability to varying system conditions.