We study systematic uncertainties in the lattice QCD computation of hadronic vacuum polarization (HVP) contribution to the muon $g-2$. In this proceedings we investigate two systematic effects; finite volume (FV) effect, mass correction and cutoff effect. We evaluate FV effect at the physical pion mass on two different volumes of (5.4 fm$)^4$ and (10.8 fm$)^4$ using PACS10 configurations at the same cut-off scale, and for the mass correction, we compare two different pion mass points, $m_\pi=$135 MeV and 146 MeV pion, on large volume, $L/a=$10.8 fm and 8.1 fm respectively. Our results indicate that the chiral perturbation theory (ChPT) possibly underestimates FV effects in a long distance region, where multi-hadron state contributions including two-pion state become prominent. For the cutoff effect, we compare two forms of lattice vector operators, which are local and conserved (point-splitting) currents, by varying the cutoff scale on a more than (10 fm$)^4$ lattice at the physical point. The local-local current correlator shows smaller scaling violation than local-conserved one both in light and strange quark channels.