Heavy flavor has played an important role for probing the properties of the hot and dense medium created in relativistic heavy ion collisions both at RHIC and the LHC energy regime. Over the past sixteen years of data taking since year 2000, PHENIX published a broad spectrum of heavy flavor results which include $J/\psi$, $\psi'$, open heavy flavor, and $\Upsilon$ production in many colliding systems at different collision center-of-mass energies. In this talk, a summary of the most recent heavy flavor results will be given.