We propose a generic construction of Lie group agnostic
and gauge covariant neural networks,
and introduce constraints to make the neural networks
continuous differentiable and invertible.
We combine such neural networks and build gauge field
transformations that is suitable for Hybrid Monte Carlo (HMC).
We use HMC to sample lattice gauge configurations in the transformed
space by the neural network parameterized gauge field transformations.
Tested with 2D U(1) pure gauge systems at a range of couplings and
lattice sizes,
compared with direct HMC sampling,
the neural network transformed HMC (NTHMC) generates Markov chains
of gauge configurations with improved tunneling of topological charges,
while allowing less force calculations as the lattice coupling increases.