A major background for the $K_L \to \pi^0 \nu \bar{\nu}$ search at the KOTO experiment is caused by neutrons generating two showers in the calorimeter made of 50~cm-long 2760 undoped CsI crystals stacked in a 1.9~m-diameter cylinder. Neutrons tend to have interactions deeper than photons. In 2018 we upgraded the calorimeter by installing 4080 silicon photomultipliers (SiPMs) on the front surface of the calorimeter. Each crystal has also been read out with a PMT on the back surface of the crystals. The depth and timing information on each shower development are obtained by measuring the difference between SiPM and PMT timing and the average of the timing. A clear separation of gamma- and neutron-events was achieved in 2019 data. Less than 0.3 neutron background events are expected when we reach the Standard-Model sensitivity of $K_L \to \pi^0 \nu \bar{\nu}$.