Chimera baryons are an important element of strongly coupled theories that provide a microscopic origin for UV complete composite Higgs models (CHMs), since they play the role of top partners in top partial compositeness. In a particular interesting realisation of CHMs based upon an underlying $Sp(4)$ gauge theory, such exotic objects are composed of two fermion constituents transforming on the fundamental, and one on the 2-index antisymmetric representations. We perform lattice computations of the chimera baryon spectrum in the quenched approximation. We present preliminary results for the masses of various chimera baryons with different quantum numbers, including the one interpreted as the top partner. We test the technology needed for future calculations with dynamical fermions.