We report on recent measurements for rare decays of $B$ and $B_s$ mesons at the Belle experiment; $B^{+} \rightarrow K^{+} \tau^{\pm} \ell^{\mp}$, $B^0 \rightarrow K^{*0}e^+e-$, $B^0 \rightarrow K^{*0}\tau^+\tau^-$, $B^0_s \rightarrow \pi^0\pi^0$, $B^0_s \rightarrow \ell^\mp \tau^\pm$ and $B \rightarrow ha^\prime$ where $\ell \in e,\ \mu$, $h \in \pi^\pm,\ K^\pm,\ D_s^\pm,\ D^{0}$, and $a^\prime$ is invisible. These highly suppressed decays are excellent tests of the Standard Model, and despite data-taking ceasing more than a decade ago, we show the Belle dataset is still capable of producing world-leading results. The $B$ decay measurements were performed on a dataset of 711 fb$^{-1}$ taken at the $\Upsilon(4S)$ resonance, corresponding to 772 million $B$ meson pairs, while the measurements of decays of $B_s$ mesons were performed on a dataset of 121 fb$^{-1}$ taken at the $\Upsilon(5S)$ (or $\Upsilon(10860)$ resonance, corresponding to 8.3 million $B_s$ meson pairs.