Course detail
Stochastic Modelling
FSI-S2M-A Acad. year: 2026/2027 Winter semester
The course focuses on Markov Chain Monte Carlo (MCMC) algorithms.
The first part deals with the fundamentals of the theory of Markov chains with continuous (real-valued) state spaces and the existence of their stationary distributions.
Next, it describes the derivation of algorithms that implement these chains and analyzes their convergence.
The final part presents examples of MCMC applications in data analysis and machine learning.
Supervisor
Department
Learning outcomes of the course unit
Prerequisites
Probability theory and mathematical statistics, mathematical and functional analysis.
Planned learning activities and teaching methods
Assesment methods and criteria linked to learning outcomes
Preparation of a semester project and an oral examination.
Language of instruction
English
Aims
Introduction of students to the basics of the theory of Markov chains with a continuous state variable and their use for sample generation. Students will gain an overview of the application of this theory in Bayesian estimation and in typical examples of engineering practice.
Specification of controlled education, way of implementation and compensation for absences
The study programmes with the given course
Programme N-MAI-A: Mathematical Engineering, Master's
branch ---: no specialisation, 3 credits, elective
Type of course unit
Exercise
26 hours, compulsory
Syllabus
Probability measure, Bayesian estimations, motivation for using MCMC
Markov chains with discrete state space (ergodic and reversible chains)
Markov chains with continuous state space
Stationary distribution of a Markov chain
Metropolis and Metropolis-Hastings algorithms
Effect of proposal density, rejection criterion, autoregressive function, Gibbs algorithm
Evaluation of MCMC algorithm results
Hamilton’s equations, Hamiltonian Monte Carlo, parameter selection in HMC, No-U-Turn algorithm
Bayesian regression, Bayesian neural networks
Natural language processing (Latent Dirichlet Allocation)
Bayesian inverse problem (parameter estimation in differential equations)
Graph tasks, combinatorial problems, traveling salesman problem