For fault characteristic of grid-connected photovoltaic (PV) systems, the sensitivity of ratio-restraining characteristic differential protection on the condition of PV inverter reactive/active power compensation is studied when the output line of grid-connected PV systems occur asymmetrical failures. The results show that sensitivity of differential protection of ratio-restraining characteristic will decrease in the state of reactive power compensation. The characteristic curve without passing through the origin has a greater advantage in terms of sensitivity, after comparing the tuning mode of traditional ratio-restraining characteristic curve. On this basis, taking into account the zero-sequence current which is less affected by fault resistance and load current in asymmetrical fault, and inverter control strategy oriented by grid voltage positive sequence component during LVRT, traditional ratio-restraining differential protection which takes zero-sequence current as protection criterion possesses a higher sensitivity, for it is not affected by reactive current and has a small protection action setting value, it is better for the protection of output lines of photovoltaic power station. This conclusion is verified by simulation model of grid-connected photovoltaic system built on Simulink.