Modern power systems are incorporated with distributed energy sources to be environmental-friendly and cost-effective. However, due to the uncertainties of the system integrated with renewable energy sources, effective strategies need to be adopted to stabilize the entire power systems. Hence, the system operators need accurate prediction tools to forecast the dynamic system states effectively. In this paper, we propose a Bayesian deep learning approach to predict the dynamic system state in a general power system. First, the input system dataset with multiple system features requires the data pre-processing stage. Second, we obtain the dynamic state matrix of a general power system through the Newton-Raphson power flow model. Third, by incorporating the state matrix with the system features, we propose a Bayesian long short-term memory (BLSTM) network to predict the dynamic system state variables accurately. Simulation results show that the accurate prediction can be achieved at different scales of power systems through the proposed Bayesian deep learning approach.