The search for the Lepton Flavor Violating decay $\mu \to e \gamma$ will reach an unprecedented level of sensitivity within the next five years thanks to the MEG-II experiment.
This experiment will take data at the Paul Scherrer Institut where continuous muon beams are delivered at a rate of about $10^8$ muons per second. On the same time scale, accelerator upgrades are expected in various facilities, making feasible to reach an intensity of $10^9$ or even $10^{10}$ muons per second. We investigate the experimental limiting factors which will define the ultimate performances, and hence the sensitivity, in the search for $\mu \to e \gamma$ with these extremely high beam rates. We then consider some conceptual detector designs and evaluate the corresponding sensitivity as a function of the beam intensity.}