Superconducting Fault Current Limiter (SFCL) can effectively limit the amplitude and derivative of the fault current in the Voltage Source Converter-High Voltage Direct Current (VSC-HVDC) transmission engineering. This paper regards SFCL as the boundary of the VSC-HVDC transmission system compared to the physical boundary of the traditional HVDC transmission system, and proposes a novel protection scheme for VSC-HVDC transmission lines based on boundary transient harmonic energy. The transient harmonic energy consumed by the boundary of the rectifier side of the positive or negative line is used as the startup component, the difference of transient harmonic energy consumed by the boundary of both rectifier side and inverter side of the bipolar lines is used to identify the internal and external fault, the ratio of transient harmonic energy consumed by the boundary of the rectifier side of the positive and negative line is used to select the fault pole in the proposed scheme. The theory analysis and simulation results show that this protection performs well under various situations, and it is almost not affected by fault resistance and fault location. In addition, high sampling frequency is not needed in this scheme, having important engineering value. This work is supported by the Key Program of National Natural Science Foundation of China (No. 61433004).