Let $G$ be a finite group. The faithful dimension of $G$ is defined to be the smallest possible dimension for a faithful complex representation of $G$. Aside from its intrinsic interest, the problem of determining the faithful dimension of finite groups is intimately related to the notion of essential dimension, introduced by Buhler and Reichstein. In this paper, we will address this problem for groups parameterized by a prime parameter $p$ (e.g., Heisenberg groups over finite fields with $p$) and study the question of the dependence of the essential dimension on $p$. As it will be shown, in general, this is always a piecewise polynomial function along certain ``number-theoretically defined'' sets, while, in some specific cases, it is given by a uniform polynomial in $p$. This talk is based on a joint work with Mohammad Bardestani and Hadi Salmasian.