A measurement of the production cross section for high transverse momentum top quark pairs is reported. The data set was collected during 2016 with the CMS detector at the LHC from pp collisions at 13 TeV, and corresponds to an integrated luminosity of 35.9 fb-1. The measurement uses events where either both top quark candidates decay hadronically and are reconstructed as large-radius jets with pt>400 GeV, or where one top quark decays hadronically and is identified as a single large-radius jet with pt>400 GeV and the other top quark decays leptonically to a b jet, an electron or a muon, and a neutrino. The cross section is extracted differentially as a function of kinematic variables of the top quark or the top quark pair system. The results are presented at the particle level, within a region of phase space close to that of the experimental acceptance, and at the parton level, and are compared to various theoretical models. The measured differential cross sections are significantly lower in both decay channels in the phase space of interest, compared to the theory predictions, while the normalized differential cross sections are consistent between data and theory.