By the case study on the protection malfunction of the unit with a neutral arc suppression coil grounding, the influence of transitive voltage on the stator earth fault protection is analyzed. The free oscillation of the transitive voltage is the main cause of the malfunction. By analyzing the characteristic frequency of the transitive voltage, a convenient method of estimation of compensation mode for the unit is proposed, which is proved by PSCAD simulation. As for the malfunction of the stator earth fault protection, the protection blocking logic is improved. The influence of transitive voltage is eliminated without sacrificing the rapidity and sensitivity of the protection. Practical application shows that it is effective by adoptinstre proposed measure.