I will present MCMC algorithms as optimization over the KL-divergence in the space of probabilities. By incorporating a momentum variable, I will discuss an algorithm which performs accelerated gradient descent over the KL-divergence. Using optimization-like ideas, a suitable Lyapunov function is constructed to prove that an accelerated convergence rate is obtained. I will then discuss how MCMC algorithms compare against variational inference methods in parameterizing the gradient flows in the space of probabilities and how it applies to sequential decision making problems.