The observed azimuthal modulations of long-range correlations in pseudorapidity in small systems like pp or p-Pb collisions show strikingly similar features to those seen in heavy ion collisions. Many theoretical approaches to interpreting this effect have been developed. However, it is still unclear whether these long-range correlations are due to final or initial state effects. To further investigate these effects, we studied long-range correlations as a function of transverse momentum in very high multiplicity pp collisions at $\sqrt{s} =13$ TeV, collected with the high multiplicity event trigger during 2016 and 2017 with ALICE. In this talk, we present the near-side per-trigger yield at large pseudorapidity separation (ridge yield) as a function of transverse momentum in pp collisions at $\sqrt{s} =13$ TeV. The results are compared to previous measurements from CMS and ATLAS experiments. In addition, we present the ridge yield in events where harder fragmentation processes are present, to explore possible physical origins of long-range correlations.