Formally, the definition is:Ī random variable is said to be k-variate normally distributed if every linear combination of its k components have a univariate normal distribution. The Multivariate Gaussian distribution is also known as the joint normal distribution, and is the generalization of the univariate Gaussian distribution to high dimensional spaces. To solve this problem, we turn to the good old Gaussians. But what about the other ones that are also pretty good? What if we observed one more points, and one of those ones end up being a much better fit than the 'best' solution? This sort of traditional non-linear regression, however, typically gives you one function that it considers to fit these observations the best. The higher degrees of polynomials you choose, the better it will fit the observations. In non-linear regression, we fit some nonlinear curves to observations. Let's revisit the problem: somebody comes to you with some data points (red points in image below), and we would like to make some prediction of the value of $y$ with a specific $x$. Of course, like almost everything in machine learning, we have to start from regression. I hope that they will help other people who are eager to more than just scratch the surface of GPs by reading some 'machine learning for dummies' tutorial, but aren't quite yet ready to take on a textbook. After watching this video, reading the Gaussian Processes for Machine Learning book became a lot easier. Richard Turner on YouTube, which was a great introduction to GPs, and some of its state-of-the-art approaches. I eventually stumbled upon the above lecture by Dr. I'm sure it's a great book, but the math is quite out of my league. My more statistical friends swear by this book, but after spending half an hour just to read two pages about linear regression I went straight into an existential crisis. In what range does the algorithm search for 'possible functions'? What gives it the capacity to model things on a continuous, infinite space?Ĭonfused, I turned to the 'the book' in this area, Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. While memorising this sentence does help if some random stranger comes up to you on the street and ask for a definition of Gaussian Process - which I'm sure happens all the time - it doesn't get you much further beyond that. The key takeaway is always,Ī Gaussian process is a probability distribution over possible functions that fit a set of points. It's not completely my fault though! Whenever I Google 'Gaussian Processes', I find well-written tutorials with vivid plots that explain everything up until non-linear regression in detail, but shy away at the very first glimpse of any sort of information theory. I recall always having this vague impression about Gaussian Processes (GPs) being a magical algorithm that is able to define probability distributions over sets of functions, but I had always procrastinated reading up on the details.

Gaussian random process