Future energy-frontier electron-positron colliders will be capable of high-precision studies of top quark properties. The measurement of the top-pair production cross section around the threshold provides access to the mass of the top quark in theoretically well-defined schemes, with statistical uncertainties of 20 MeV or less, depending on the assumed integrated luminosity of the measurement. At this level of precision, experimental and theory systematics are likely to become important or even dominant. This contribution presents a first analysis of the impact of the remaining uncertainties of the recently completed calculation of the top pair production cross section at NNNLO QCD including the exchange of Higgs bosons on the extraction of the top quark mass from a threshold scan. The analysis is based on reconstruction efficiencies and background levels obtained in full simulation studies for CLIC, combined with signal cross sections from the higher-order calculations. To assess possible differences between different collider options, the study is performed in the context of CLIC, ILC and FCC-ee by considering the differences in the luminosity spectra.