In this contribution, we investigate the complete electroweak one-loop radiative correction to the $e^+e^-\to H^+H^-$ within the Inert Higgs Doublet Model (IHDM), at the future Higgs factory such as the ILC, CLIC and CEPC. After taking account of the theoretical and experimental constraints such as LEP, LHC and Dark matter constraints, the calculations are performed using FeynArts/FormCalc (FA/FC) to evaluate the one-loop weak corrections and using Feynman Diagram Calculation (FDC) to compute the QED contribution to the Next-Leading-Order (NLO) cross section, in three energies of collisions 250, 500 and 1000 GeV. By observing the $e^+e^-\to H^+H^-$ process, the detection of the new physics of IHDM can be directly done due to the large production rate and the corrections are significant.