We investigate relations between the chiral susceptibility and axial $U(1)$ anomaly in lattice QCD at high temperatures. Employing the exactly chiral symmetric Dirac operator, we separate the purely axial $U(1)$ breaking effect in the connected and disconnected chiral susceptibilites in a theoretically clean manner. Preliminary results for two-flavor lattice QCD near the critical temperature are presented.