There are cosmic rays in the current universe. They play various important roles in the current universe. However, we do not understand when, where, how cosmic rays are first accelerated since the Big Bang. We show that supernova remnants of first stars can accelerate the first cosmic rays at z~20. Shock waves of the first supernova remnants are nonrelativistic Weibel mediated shock, so that the coherent length scale of magnetic field fluctuations is much smaller than the gyroradius of the accelerated particles. Therefore, the maximum energy of the first CRs becomes much smaller than that in the current universe, which is about 100 MeV. Furthermore, we discuss cosmic-ray acceleration by accretion shocks due to the cosmological structure formation at z~20. Since the accretion shocks at z~20 propagate not an ionized medium but a neutral medium, they cannot accelerate the first cosmic rays.