Interactions between hard partons and the quark-gluon plasma range from frequent soft interactions to rare hard interactions. While it is reasonable that hard interactions can be described perturbatively, soft interactions likely have significant non-perturbative effects. The cumulative effect of the soft interactions can be encoded into parton transport coefficients, which can be constrained from heavy ion measurements. In this work, we perform a simplified proof of principle calculations showing that the magnitude of the drag and diffusion of light partons can be constrained from the hadronic and jet $R_{AA}$, assuming a known temperature dependence of the soft transport coefficients. We show how this study can be implemented naturally in a parton energy loss model that factorizes systematically soft and hard partonic interactions. We use this proof of principle calculation to understand how reducing uncertainties on $R_{AA}$ can improve constraints on the parton's transport coefficients. We also observe the complementarity of jet and hadronic observables to constrain the light parton transport coefficients.