This paper researches the application of hybrid multilevel converters (HMC) in the VSC-HVDC system. First, the circuit configuration and working principle are introduced. Then the conditions about capacitor voltage balancing of each sub-module are analyzed from the power perspective. A basic control strategy of converters is introduced. An advanced control strategy of converters and a voltage-balancing scheme are proposed. Feed forward cross decoupling control strategies based on the HMC are used to design the controller which includes decoupling control of active power and reactive power, and the DC voltage control. Finally, a two terminals, three phases, 35 kV, 240 sub-modules VSC-HVDC detailed simulation model is built based on PSCAD/EMTDC. The principle of the main circuit and the control strategies are verified.