The RICOCHET experiment is currently being deployed at the Institut Laue-Langevin (ILL) research nuclear reactor in Grenoble, France, with the goal of performing a precise measurement of coherent elastic neutrino-nucleus scattering (CE$\nu$NS). At reactor neutrino energies ($0$–$8$ MeV), the coherence condition is fully satisfied and the cross section benefits from the expected $N^2$ enhancement, where $N$ is the neutron number of the target nucleus. A detector with a kilogram-scale target mass is therefore sufficient for a measurement. In the final configuration of RICOCHET, two cryogenic detector concepts will be deployed: an array of eighteen germanium bolometers of about $40$ g each, and nine superconducting crystals inserted in a cryostat operating at a temperature of $10$ mK. In the initial commissioning phase, up to nine germanium crystals were installed. The raw background level is significant due to the proximity of the reactor core (neutron and gamma) and to the positioning of the detector on the surface (cosmic background). To fight against the backgrounds, a three-fold strategy is used: an active muon veto, a passive shielding of lead and polyethylene to mitigate the gamma and neutron backgrounds, and the simultaneous measurement of ionization and heat to discriminate electron and nuclear recoils. Prior to the installation of the RICOCHET cryostat, a measurement campaign of the radiogenic and reactogenic gamma background was carried out inside the shielding with a dedicated High-Purity Germanium (HPGe) detector. This contribution will present the gamma background model derived from these measurements and highlight some comparisons with data recorded in the commissioning phase.