The nuclear modification factor $R_{\rm AA}$ has been satisfactorily described by various jet quenching models. Nonetheless, all these formalisms, until very recently, underpredicted the high-$p_{\rm T}$ (> 10 GeV) elliptic flow $v_2$. We find that the simultaneous description of these observables requires to strongly suppress the quenching for the first $\sim 0.6$ fm after the collision. This shows the potential of jet quenching observables to constrain the dynamics of the initial stages of the evolution.