High-redshift primordial galaxies have recently been found with evolved stellar populations and complex star-formation histories reaching back to 250 Myr after the Big Bang. Their intense bursts of star-formation appear to be interspersed with sustained periods of strong quenching, however the processes underlying this evolutionary behaviour remain unclear. Unlike later epochs, galaxies in the early Universe are not located in large associations like clusters. Instead, they co-evolve with their developing circumgalactic halo as relatively isolated ecosystems. Thus, the mechanisms that could bring about the downfall of their star-formation are presumably intrinsic, and feedback processes associated with their intense starburst episodes likely play an important role. Cosmic rays are a viable agent to deliver this feedback, and could account for the star-formation histories inferred for these systems. The cosmic ray impact on galaxies may be investigated using the wealth of multi-wavelength data soon to be obtained with the armada of new and upcoming facilities. Complementary approaches to probe their action across the electromagnetic spectrum can be arranged into a distance ladder of cosmic ray feedback signatures. With a clear understanding of how cosmic ray activity in primordial systems can be traced, it will be possible to extend this ladder to high redshifts and map-out the role played by cosmic rays in shaping galaxy evolution over cosmic time.