We report on the interaction of $D N$, $D^* N$, together with their coupled channels, by using an extension of the local hidden gauge formalism from the light meson sector, which is based on heavy quark spin symmetry. The scheme is based on the use of the impulse approximation at the quark level, with the heavy quarks acting as spectators, which occurs for the dominant terms where there is the exchange of a light vector meson. In order to perform the full coupled channels calculation, the pion exchange and the Weinberg-Tomozawa interactions are generalized to construct the effective transition potentials. With this dynamics we look for states generated from the interaction, with a unitary coupled channels approach that mixes the pseudoscalar-baryon and vector-baryon states. We find two states with nearly zero width, which are associated to the $\Lambda_c(2595)$ and $\Lambda_c(2625)$. The lower state, with $J^P = 1/2^-$, couples to $D N$ and $D^* N$, and the second one, with $J^P = 3/2^-$, to $D^* N$. In addition to these two $\Lambda_c$ states, we find four more states with $I=0$, one of them nearly degenerate in two states of $J=1/2,\ 3/2$. Furthermore we find three states in $I=1$, two of them degenerate in $J=1/2, 3/2$.