Accurate state of charge (SOC) estimation is of great significance for the safe operation of battery management systems in new energy vehicles, and reliable parameter identification is the key to SOC estimation. In this paper, a joint parameter identification algorithm based on adaptive forgetting factor recursive least squares (AFFRLS) and adaptive extended Kalman filter (AEKF) is proposed considering the multi-timescale effect of a lithium battery. First, a lithium battery dual-polarization model (DP model) is classified into fast dynamic and slow dynamic based on multi-timescale effects. Secondly, the AFFRLS and AEKF based-joint parameter identification algorithm is designed to identify the parameters of fast dynamic and slow dynamic, respectively. Finally, the SOC is estimated using an H-infinity algorithm with the parameter identification data. The results show that the proposed algorithm has good accuracy and robustness in different driving cycles and at different temperatures. The maximum root-mean-square error of the terminal voltage is only 3.329 mV, and the SOC root mean square error is less than 1%.