The energy loss mechanism of jets plays a central role in nuclear and high energy physics. We propose direct measurements of the energy loss of leading jets and perform a calculation at next-to-leading logarithmic (NLL$'$) accuracy in the vacuum. The formation of leading jets can be described by jet functions which constitute probability densities and thus allow for a perturbative calculation of the average the energy loss. We identify the following three criteria for a direct measurement of jet energy loss at the cross section level. $i)$ We measure a well defined object, the leading jet, where the formation process can be expressed in terms of a probability density. $ii)$ In addition, we need a measurement of a hard reference scale with respect to which jet energy loss is defined. $iii)$ At leading logarithmic accuracy, we require that the jet energy loss can be identified with parton energy loss. We discuss suitable observables and present numerical results including threshold corrections by making use of a parton shower Monte Carlo approach.