To address the problem of difficult monitoring and identification of a bearing wear damage fault in pumped storage unit, a dual-modal neural network model for unit bearing diagnosis that combines sound and vibration data is proposed. First, the sound and vibration characteristics of the unit are analyzed, and a soft threshold is combined to improve the singular value decomposition (SVD) denoising method. This effectively eliminates inherent noise interference from the non-contact sound sensor. In addition, an inverse Bark spectrum transform method is proposed, and combining with feature engineering technology such as the Bark spectrum transform and Gram’s angle summation field (GASF) transform, the sound and vibration feature map of the unit bearing is extracted. A feature map propagation network is established using a self-attention mechanism with relative position encoding and depth-wise separable convolution. Simultaneously, a time series data propagation network is constructed using a multi-head self-attention mechanism and a bidirectional long short-term memory network (BiLSTM). These components are integrated into a parallel architecture to build a fault diagnosis model for the unit bearing. Comparative analysis of results shows that the proposed method has high accuracy in fault recognition, providing an effective solution to the unit bearing monitoring problem in pumped storage power stations.