With the large amount of distributed generation in the distribution network and the rapid development of urban regional loads, the operating environment of a distribution network has become increasingly complicated. At the same time, because the distribution network reconfiguration involves a large number of binary discrete variables of switch states, it is difficult for existing optimization methods to solve the large-scale urban distribution network reconfiguration problem. Thus a multi-level dynamic reconstruction method for an urban distribution network, one based on deep reinforcement learning, is proposed. First, a fast judgment model for multi-level reconstruction of the network based on deep learning is established, through which the online decision-making of the reconstruction level is realized, and the dimensionality of the action space of the agent is reduced. Second, a deep Q-network with parameter freezing and experience playback mechanisms is used to learn environmental information such as predicted load and photovoltaic energy output power. Then, with the objective of optimal operation cost, voltage offset and load balance degrees, the distribution network is dynamically reconfigured and operationally optimized via a learned strategy set. A multi-agent reinforcement learning model is established to jointly optimize different reconstruction subjects in each period. Finally, the effectiveness of the proposed method is verified by an example analysis. This work is supported by the National Natural Science Foundation of China (No. 52077146).