Existing studies on harmonic instability in wind farms that consider transmission line distributed parameters do not account for the dynamic characteristics of the static var generator (SVG). However, SVG exhibits negative resistance and inductance characteristics at high frequency, making it prone to inductive-capacitance coupling with transmission line distributed parameters and other components, which can lead to harmonic instability in the system. Based on this, a frequency domain modal analysis method for wind farm harmonic instability is proposed using Taylor approximation. The system stability is determined by solving the zero point of the frequency domain node admittance matrix determinant, and the node’s participation factor in the unstable mode is defined to achieve the source location of harmonic instability and identification of the transmission path. The paper reveals that an SVG tends to interact with long-distance transmission lines and AC power grids, leading to further amplification of high-frequency harmonics in renewable energy substations until high-frequency harmonic instability occurs. Moreover, an SVG is the dominant factor in high-frequency harmonic instability in wind farms. Finally, the accuracy of the proposed method is verified through Matlab/Simulink simulations.