Future optopolarimetric surveys of stars, including PASIPHAE and SOUTH POL, offer the potential for acquiring high-quality local measurements of the Galactic magnetic field (GMF) at interstellar cloud locations. However, the inherent sparsity of these measurements raises concerns regarding the feasibility of accurately backtracking Ultra High-Energy Cosmic Rays (UHECR) to their sources, a crucial aspect of charge-particle astronomy. In this study, we assess the accuracy of UHECR backtracking using mock sparse local GMF data derived from the Jansson & Farrar 2012 (JF12) GMF model. By creating 1000 mock UHECR datasets that trace back within a $3^{\circ}$ angular range from the galaxy M82, we investigate the impact of varying GMF measurement sparsity and rescaling of the GMF strength. Our findings demonstrate that an average GMF strength of $1,\mu G$ yields satisfactory backtracking results even with sparse measurements, requiring an average GMF measurement spacing of approximately 1600 pc. However, when the average GMF strength is increased by a factor of 3 or 10, the accuracy of backtracking diminishes significantly, resulting in breakdowns at measurement spacings of 1600 pc and 400 pc, respectively. These findings emphasize the challenges associated with precise charge-particle astronomy using sparse local GMF measurements and underscore the importance of obtaining dense and high-quality GMF data through surveys like PASIPHAE and SOUTH POL to enable accurate backtracking of UHECR.