A search for resonant and non-resonant di-Higgs production in the bbbb, bb$\gamma\gamma$ and $\gamma\gamma$WW channels is presented. The analysis uses 3.2-13.3 fb$^{-1}$ of proton-proton collision data at $\sqrt{s}$ = 13 TeV collected by the ATLAS detector at the Large Hadron Collider (LHC). No excess is observed and upper limits are placed on the di-Higgs production rate in the Standard Model (SM) and in various beyond SM scenarios. Future prospects for the observation of non-resonant SM di-Higgs production with the high-luminosity LHC are also presented.