We solve the long-standing problem concerning the fate of the chiral $U(1)_A$ symmetry in QCD-like theories at high temperature in the chiral limit. We introduce a simple instanton based random matrix model that precisely reproduces the properties of the lowest part of the lattice overlap Dirac spectrum. We show that in the chiral limit the instanton gas splits into a free gas component with a density proportional to $m^{N_f}$ and a gas of instanton-antiinstanton molecules. While the latter do not influence the chiral properties, for any nonzero quark mass the free gas component produces a singular spectral peak at zero that dominates Banks-Casher type spectral sums. By calculating these we show that the difference of the pion and delta susceptibility vanishes only for three or more massless flavors, however, the chiral condensate is zero already for two massless flavors.