We present the necessity of counter-terms for Quantum Finite Element (QFE) simulations of $\phi^4$ theory on non-trivial simplicial manifolds with semi-regular lattice spacing. By computing the local cut-off dependence of UV divergent diagrams we found that the symmetries of the continuum theory are restored for $\phi^4$ theory on the manifolds $S^2$ and $S^2 \times \mathbb{R}$ in the weak coupling regime. Here we consider the construction of non-perturbative local counter-terms in an attempt to approach the strong coupling Wilson-Fisher IR fixed point.