The access of a high proportion of photovoltaic energy and a large number of electric vehicles imposes severe challenges to the voltage control of a distribution network. This paper presents a multi-time scale cooperative voltage control strategy based on hierarchical deep reinforcement learning. First, considering the demand response characteristics of electric vehicle users, the day-ahead time-of-use price is formulated to guide users to change their charging behaviors to achieve orderly charging of coordinated scheduling of switching capacitors. Secondly, a deep reinforcement learning strategy with a two-layer structure is constructed within the day. The upper layer guides users to respond to real-time price incentives through price adjustment to adjust electric vehicle charging load over a long time-scale, and the lower layer controls photovoltaic inverters and reactive power compensation devices on a short time-scale. By the two-layer strategy, the real-time coordinated regulation of resources with different time response characteristics is realized to reduce system voltage deviation. Finally, the effectiveness of the proposed strategy is verified on the improved IEEE 33-bus system.