A linearized theory of gravity is constructed in the framework of Very Special Relativity (VSR), maintaining the usual gauge invariance of linearized General Relativity (GR). After finding the equations of motion of the model, choosing a suitable gauge, we prove that this extension of linearized GR allows for a graviton mass $m_g$, which could be of extreme interest in different astrophysical scenarios. Furthermore, as expected due to the gauge invariance, we verify the presence of only two physical degrees of freedom in the theory. To start grasping the possible consequences of this modification of linearized GR, we study Gravitational Waves (GW) effects through the geodesic deviation equation: what we find is that VSR signatures would be proportional to the small parameter $m_g^2/E^2$, with $E $ being the energy of a single graviton in a monochromatic GW. While this parameter is very small ($\sim 10^{-20}$) for GW detected by the interferometers LIGO and VIRGO, it seems to get better ($\sim 10^{-10}$) with the next generation of gravitational interferometers, like LISA. Therefore, this increase plus the anisotropic nature of VSR could lead to observable consequences of the VSR extension in the future.