This textbook will present, in a rigorous way, the basic theory of the discrete-time and the continuous-time Markov chains, along with many examples and solved problems. For both the topics a simple model, the Random Walk and the Poisson Process respectively, will be used to anticipate and illustrate the most interesting concepts rigorously defined in the following sections. A great attention will be paid to the applications of the theory of the Markov chains and many classical as well as new results will be faced in the book. This textbook is intended for a basic course on stochastic processes at an advanced undergraduate level and the background needed will be a first course in probability theory. A big emphasis is given to the computational approach and to simulations.
