We propose a Bayesian parameter inference approach to determine Parton Distribution Functions (PDFs) and we show that we can replace the standard $\chi^2$ minimisation used in most existing PDF global analysis procedures, by Markov chain Monte Carlo (MCMC) techniques. These methods, widely used in statistics, lead to reliable estimates of uncertainties in terms of confidence limit intervals of probability distributions, and offer additional insight into the rich field of PDFs. The formulation of PDF determination in terms of Bayesian inference, the Monte Carlo algorithm we have implemented in the xFitter code and a selection of first results we have obtained are presented in this contribution.