We discuss the non-eikonal effects due to a finite longitudinal width of the target on single and double inclusive gluon production in proton-proton collisions in the framework of the Color Glass Condensate. We show that these non-eikonal corrections can be summed to all orders in proton-proton collisions and can be identified as corrections to the Lipatov vertex. By using this non-eikonal Lipatov vertex, we calculate single and double inclusive gluon production in proton-proton collisions and argue that the non-eikonal corrections that we treat in this work, break the accidental symmetry of the CGC and lead to non-vanishing odd azimuthal harmonics.