A novel adaptive system based on cascade detecting harmonic current in power system is proposed to solve the contradiction between rapidity and stability. In cascade neural network, a fast dynamic response of harmonic detection can be provided by normal least mean square (LMS) with large step-size, meanwhile to filter fluctuation of weight, a higher precision of adaptive detection in steady-state is introduced by embedding an average filter into LMS. By using Z-transform, this paper analyses the stabilization and derives constraint conditions of step-size of the novel neural network unit based on LMS with embedded average filter. The stability of the new system is guaranteed by the limited range of specified step-size to improve dynamic performances and reduce steady-state errors of adaptive detection. Simulation results show that the improved cascade neural network has faster dynamic response and higher accuracy in adaptive harmonic detection.