In High Energy Physics, the Monte Carlo simulation of the detector response to the particles traversing it is one of the most computing intensive tasks. For some applications, a certain level of approximation is acceptable and it is therefore possible to implement fast simulation models that have the advantage of being less computationally intensive, in comparison to standard detailed Monte Carlo techniques. As a consequence, research on fast simulation solutions, including deep Generative Models, is very active. 3DGAN is a Generative Adversarial Network prototype, based on 3 dimensional convolutions, developed to simulate the response of high-granularity calorimeters. We have previously reported on its physics performance and the results we obtained by introducing a data parallel approach to accelerate training, using the Horovod library in conjunction with the Keras and Tensorflow packages.
One of the major bottleneck in the 3DGAN application is the training time. In this contribution we present improved results on 3DGAN distributed training: we report on the reduction of training time and high scaling efficiency up to 256 Intel Xeon 8260 (codenamed ``Cascade Lake'') nodes. We demonstrate how HPC centers could be utilized to globally optimize deep learning models, thanks to their large computation power and excellent connectivity. In addition we document our preliminary studies on the use of genetic algorithms as an alternative method to train and optimise neural networks.