Recent measurements of anisotropies in the azimuthal distributions for charged particles in $pp$ and $p$+Pb collisions from the ATLAS experiment at the LHC are reported. To study the impact of hard processes on the azimuthal anisotropy, the effect of particles produced in jets on the two-particle correlations in $pp$ collisions at $\sqrt{s}$ = 13 TeV is presented. For the same collision system, the elliptic flow of muons from decays of charm and bottom hadrons is also shown. Additionally, the impact of hard processes is explored with the azimuthal anisotropy of charged hadrons measured up to a high transverse momentum in $p$+Pb collisions at \snn = 8.16 TeV. The correlation between the mean transverse momentum and the magnitudes of the flow harmonics is reported in 5.02 TeV $p$+Pb collisions, aiming to investigate the role of initial conditions in the formation of azimuthal anisotropy in small systems. The measurement of elliptic flow of charged hadrons produced in photo-nuclear interactions in ultra-peripheral Pb+Pb collisions, which may help to interpret the $pp$ and $p$+Pb data, is also discussed.