The introduction of non-orthogonal electric and magnetic fields in the QCD vacuum enhances the weight of topological sectors with a nonzero topological charge. For weak fields, there is a linear response for the expectation value of the topological charge. We study this linear response and relate it to the QCD correction to the axion-photon coupling. We also analyse the magnetic field dependence of the topological susceptibility for a range of temperatures around $T_c$. In this work we use lattice simulations with improved staggered quarks at physical masses, including background magnetic and (imaginary) electric fields.