We study the tensor renormalization group (TRG) in the dimension larger than two as the Higher-order TRG (HOTRG) with the randomized SVD method. The randomized SVD and the detailed discussion on the low order tensor representation, we can calculate the HOTRG with the reduced computational cost. We also represent our method by using the cost function, and the details of the cost function for the isometry determine the precision, stability, and calculation time. In our study, we show calculation order improvement using randomized SVD. We also propose that the internal line respect for any TRG method improves the calculation without changing the order of the computational cost.