Accuracy and latency are crucial to the trigger system in high luminosity particle physics experiments. We investigate the usage of deep neural networks (DNN) to improve the accuracy of the muon track segment reconstruction process at the trigger level. Track segments, made by hits within a detector module, are the initial partial reconstructed objects which are the typical building blocks for muon triggers. Currently, these segments are coarsely reconstructed on FPGAs to keep the latency manageable. DNNs are ideal for these types of pattern recognition problems, and so we examine the potential for DNN based track segment reconstruction to be accelerated by dedicated FPGAs to improve both processing speed and latency for the trigger system.