The high luminosity upgrade of the LHC (HL-LHC) will see a massive increase in the instantaneous luminosity leading to up to 200 proton-proton collisions in each bunch crossing (pileup) demanding higher performance from the LHC detectors' electronics and real-time data processing. The ATLAS Liquid Argon (LAr) calorimeter, which measures the energy of particles from LHC collisions, employs dedicated data acquisition electronic boards based on FPGAs, to process large data volumes with low latency. The optimal filtering algorithm currently used for the energy reconstruction has been found to suffer significant performance degradation under high pileup conditions. We show that small recurrent or convolutional neural networks can surpass the performance of the optimal filter. Prototype implementations of the inference code in VHDL indicate that deploying these networks on FPGAs is feasible, with the resulting firmware fitting onto the planned Intel Agilex devices. The complete design can process 384 detector cells per FPGA by integrating parallel instances of the firmware with a latency smaller than 125 ns.