A multi-agent decentralized local voltage control method based on the deep reinforcement learning is proposed. This is needed because there are some problems in the regional grid with renewable energy, such as poor communication conditions, insufficient measurement equipment, and difficult coordination of voltage control equipment at different nodes. First, this method transforms the voltage control problem lacking measurement data into a partial observable Markov decision problem, and a multi-agent decentralized voltage control framework with the optimization goal of minimizing network loss is constructed. Then, a multi-agent deep deterministic policy gradient algorithm is used to train the agents offline, and the trained agents are used for online voltage control. Finally, an example is simulated and analyzed based on the improved IEEE33 bus system. The results show that each agent can solve the approximate global optimal solution according to the electrical information of its own node. This work is supported by the Science and Technology Project of State Grid Corporation of China (No. SGTJDK00DWJS2100039).