To solve the problem of the uncertain parameters and the low precision of the single forecasting model for the traditional least squares support vector machine (LSSVM), a combined forecasting model based on ensemble empirical mode decomposition (EEMD) and the LSSVM is proposed. Firstly, the historical data is decomposed into a series of relatively stable component of the sequence by the EEMD, and then the appropriate forecasting model is established for each component of the sequence. The parameters of the LSSVM are optimized through the Bayesian evidence framework. Bayesian inferences are used to determine model parameters, regularization hyper-parameters and kernel parameters. The results of each component forecasting are superimposed to obtain the final forecasting result. Finally, a household ultra-short-term load data is used to validate the model, and the simulation results show that this model has achieved better forecasting result than a single model.