Fast variations of gamma-ray flux from Active Galactic Nuclei and Gamma-Ray Bursts can con- strain Lorentz Invariance Violation (LIV) because of the delayed (or advanced) arrival of photons with higher energies: this approach has lead to the current world-best limits on the energy scale of Quantum Gravity. Here we report on complementary constraints on LIV by studying the gamma-ray emission up to TeV energies from the Galactic Crab pulsar, recently discovered by the MAGIC collaboration. A likelihood analysis of the pulsar events reconstructed for energies above 400 GeV finds no significant variation of the arrival time with energy, and 95% CL limits are obtained on the effective LIV energy scale taking into account systematic uncertainties. Only a factor of about two less constraining than the current world-best limit on a quadratic LIV sce- nario, pulsars are now well established as a third and independent class of astrophysical objects suitable to constrain the characteristic energy scale of LIV.