Attention-based models are dominating the generative modeling, namely in the natural language processing domain. These models do not suffer from implicit bias and can be trained on large amounts of data thanks to an efficient parallelization of computations. The presented work explores a suitability of attention network for an image completion task on calorimeter data that is characterized by high granularity and large dynamic range of values. We demonstrate that an attention network can learn to perform the image completion task to successfully replicate average high-level features as well as to preserve quality of individual samples.