The maximum mass of a neutron star has important implications across multiple research fields, including astrophysics, nuclear physics and gravitational wave astronomy. Compact binary millisecond pulsars (with orbital periods shorter than about a day) are a rapidly-growing pulsar population, and provide a good opportunity to search for the most massive neutron stars. Applying a new method to measure the velocity of both sides of the companion star, we previously found that the compact binary millisecond pulsar PSR J2215+5135 hosts one of the most massive neutron stars known to date, with a mass of 2.27$\pm$0.16 M$_\odot$ (Linares, Shahbaz \& Casares, 2018). We reexamine the properties of the 0.33 M$_\odot$ companion star, heated by the pulsar, and argue that irradiation in this ``redback'' binary is extreme yet stable, symmetric and not necessarily produced by an extended source. We also review the neutron star mass distribution in light of this and more recent discoveries. We compile a list of all (nine) systems with published evidence for super-massive neutron stars, with masses above 2 M$_\odot$. We find that four of them are compact binary millisecond pulsars (one black widow, two redbacks and one redback candidate). This shows that compact binary millisecond pulsars are key to constraining the maximum mass of a neutron star.