In high-energy scattering processes involving two or more hadrons one can measure observables that are sensitive to partonic transverse momentum, which is encoded in so-called transverse momentum dependent (TMD) parton distribution functions (PDFs), also called TMDs. These functions correspond to Fourier transforms of matrix elements that contain process-dependent gauge links. As the energy associated to the collision process increases, one becomes more sensitive to the small-$x$ region which is dominated by gluon rather than quark TMDs. In this paper we study the leading-twist gluon TMDs in the small-$x$ limit for the dipole-type gauge link structure, for both unpolarized and vector polarized hadrons. In the limit $x\to0$, the gluon-gluon correlator reduces to a correlator that consists of a single Wilson loop. This is used to obtain a simple description of gluon TMDs in the small-$x$ region: some of the functions vanish, while others become proportional to each other.