This paper proposes a new method to estimate the low frequency oscillation modes from synchrophasor measurements in power system by using Multivariate Empirical Mode Decomposition (MEMD) and Hilbert-Huang Transform (HHT). A multi-channel MEMD is proposed to suppress the influences of mode mixing caused by Empirical Mode Decomposition (EMD), and further enhance the computational efficiency of decomposing multiple measurements. In aid of the developed MEMD, the Intrinsic Mode Functions (IMF) with different frequency scales of different measurement channels are obtained and the critical IMF associated with the dominant oscillation mode are detected by the Teager energy operator. Owing to time-varying characteristics of the dominant oscillation modes, the instantaneous oscillation frequency and instantaneous damping ratio of each dominant oscillation mode are tracked by HHT. The proposed approach is analyzed and verified by using the simulation data from the 16-generator 68-bus test system as well as the field measurements from Phasor Measurement Units of Liaoning Power Grid. The results show the accuracy and effectiveness of the proposed method. This work is supported by Science and Technology Program of State Grid Corporation of China (No. SGTYHT17-JS-199) and Science and Technology Project of State Grid Liaoning Power Supply CO., LTD (No. SGTYHT17JS201).