Mergers of compact objects, binary black holes and mergers including at least one neutron star, are a predicted source of high-energy neutrinos. These astrophysical events are now routinely detected through observation of their gravitational wave signature and, at least in one instance, their electromagnetic counterparts were also detected. Particles accelerated during the coalescence of compact objects may also interact to produce high-energy neutrinos, which have yet to be detected, but observations are ongoing. The LIGO-Virgo-KAGRA Collaboration publicly releases information on candidate gravitational wave events from compact binary coalescences in low latency during the current observing run (O4). To aid the electromagnetic follow-up, using data from the IceCube Neutrino Observatory, we search, in real time, for neutrinos spatially and temporally coincident with these gravitational wave candidate events using a time window of 1000 seconds centered on the merger time. We use two methods, both of which have been previously used to search for neutrino emission from gravitational-wave transients: an unbinned maximum likelihood analysis applied to significant alerts and a Bayesian analysis with astrophysical priors, applied to both significant and low-significance alerts. In addition, we search for long-duration neutrino emission up to 14 days after the merging of binaries containing a neutron star. We report analysis results determined in real time for these searches, and set upper limits on both flux and isotropic-equivalent energy emitted in neutrinos.