Compared with the traditional electric power communication network, the EPON network can meet the requirements of the smart substation communication system better. But the problems of security, reliability and real-time in the process layer network of the smart substation still exist, which leads to the miss tripping risk of the relay protection. The improvement measures of this problem are proposed, and the adaptability of relay protection function based on EPON network is studied. Based on the protection dynamic testing platform of the smart substation using the EPON network, the bus protection, transformer protection and line protection of 220 kV smart substation are studied and tested. The testing results show that all kinds of protection devices, merging units and intelligent terminals can deliver the sampling and tripping packets through EPON process layer network, and the performance of the protections is not affected.