The quality of molecular dynamics (MD) simulations critically depends on the employed potential energy model. Accurate uncertainty quantification (UQ) of these models could increase trust in MD simulation predictions and promote progress in the field of active learning of neural network (NN) potentials. Bayesian methods promise reliable uncertainty estimates, but the high computational cost of training via classical Markov Chain Monte Carlo (MCMC) schemes has prevented their application to deep NN potentials. In this work, we propose stochastic gradient MCMC methods as a computationally efficient option for Bayesian UQ of MD potentials. The stochastic gradient Langevin dynamics method yields promising results for a tabulated coarse-grained water model and could thus be a feasible approach for NN potentials. Additionally, we illustrate the inherent limit of Bayesian UQ imposed by the functional form of the employed model.