The suppression and modification of high-energy processes, like jets, in heavy-ion collisions provides an important window to access the degrees of freedom of this exotic material on different length scales. Despite increasingly precise and differential measurements of the properties of jets in heavy-ion collisions, however, it has remained challenging to use jets to make unambiguous and model-independent statements about the quark-gluon plasma. Here I will give a personal take on some origins of these challenges, including the difficulty of modelling and biases from jet selection that obfuscate the direct interpretation of jet modification measurements. I will discuss a few model studies that have helped to disentangle the source of non-intuitive effects in measurements, and finally highlight data-driven approaches as an interesting opportunity toward studying the quark-gluon plasma in a model-independent way.