Various theories beyond the Standard Model predict new, long-lived particles with unique signatures which are difficult to reconstruct and for which estimating the background rates is also a challenge. Signatures from exotic heavy particles, or long-lived particles decaying in the inner detector, as well as those from particles with a charge larger than the elementary unit charge, are all examples of experimentally demanding signatures. Four recent results using $13$ TeV proton-proton collision data collected by the ATLAS detector are presented here.