With the construction of smart substation and the development of monitoring technology, the Condition-Based Maintenance (CBM) of relay protection system can be implemented. However, most of the maintenance strategies of relay protection system considering the condition monitoring are studied qualitatively, without quantitative theoretical research of science and reasonableness. On this basis, according to the failure mechanism, failure types and evolution features of the components in smart substation relay protection system, the factors affecting operation condition of equipment are classified to long-term degradation failure and short-term shock failure. The failure rate distribution model of the components are described by the Degradation-Threshold and Degradation-Shock (DTDS) model under different conditions considering the influence of the aging of equipment and maintenance regression. Thereinto, Degradation Threshold (DT) model is solved by using least square method and Degradation-Shock (DS) model is described through Gamma distribution. Moreover, the risk assessment models of single and dual configuration of relay protection systems are established respectively combining the characteristics of the refuse-to-operate failure and the mal-operation failure to optimize CBM of relay protection system in smart substation. This work is supported by National Key Research and Development Program of China (No. 2016YFB0900600).