We report on our study of the $B \to D^{(*)}\ell\nu$ semileptonic decays at zero and nonzero recoils in 2+1 flavor QCD. The M\"obius domain-wall action is employed for light, charm and bottom quarks at lattice cutoffs $a^{-1}$ = 2.5 and 3.6 GeV. We take bottom quark masses up to $\approx$ 2.4 times the physical charm mass to control discretization effects. The pion mass is as low as $M_\pi \sim$ 310 MeV. We present our preliminary results for the relevant form factors and discuss the violation of heavy quark symmetry, which is a recent important isuue on the long-standing tension in the Cabibbo-Kobayashi-Maskawa matrix element $|V_{cb}|$ between the exclusive and inclusive decays.