Standard local updating algorithms experience a critical slowing down close to the continuum limit, which is particularly severe for topological observables. In practice, the Markov chain tends to remain trapped in a fixed topological sector. This problem further worsens at large $N$, and is known as $\mathrm{topological}\,\,\mathrm{freezing}$. To mitigate it, we adopt the parallel tempering on boundary conditions proposed by M. Hasenbusch. This algorithm allows to obtain a reduction of the auto-correlation time of the topological charge up to several orders of magnitude. With this strategy we are able to provide the first computation of low-lying glueball masses at large $N$ free of any systematics related to topological freezing.