The radio detection of high-energy neutrinos has huge potential as it allows for cost-efficient instrumentation of huge volumes leading to unprecedented sensitivity in the $10^{16}~\mathrm{eV}$ - $10^{20}~\mathrm{eV}$ energy regime. The required technological advances of the detector hardware have matured in the two pilot arrays ARA and ARIANNA and the focus shifts to analysis and reconstruction techniques. We present a new modular framework for the detector simulation and data reconstruction of radio detectors of neutrinos and cosmic rays. The software is written in Python and publicly available on github. We built on the long-standing experience of cosmic-ray detectors as many aspects of data processing, detector simulation and reconstruction are very similar for neutrinos. NuRadioReco is already being used by the ARIANNA experiment and for the detector simulation in the novel neutrino MC code NuRadioMC. It was designed to be flexible so that is can be adapted for different neutrino and cosmic-ray detectors and can serve as the software framework of a future large-scale Askaryan detector which is actively discussed in the community.