We present results for the light, strange and charm quark masses using $N_f=2+1+1$ twisted mass fermion ensembles at three values of the lattice spacing, including two ensembles simulated with the physical value of the pion mass. The analysis is done both in the meson and baryon sectors. The difference in the mean values found in the two sectors is included as part of the systematic error. The presentation is based on the work of Ref. [1], where more details can be found.