2.2. Nonnparametric Regression¶

Nonparametric regression is a data-driven alternative to least squares in which the predictor does not take a predetermined form. Kernel regression estimates the response by convolving the data with a kernel function to combine the influence of the individual data points.

An example of kernel regression is the Nadaraya-Watson regressor. The Nadaraya-Watson regressor estimates the response $$y$$ using a kernel weighted average of the individual datapoints $$(x_i, y_i)$$:

$\hat{y}(x) = \frac{ \sum_i K(x, x_i) Y_i }{ \sum_j K(x, x_j) }$

where $$K(x, x')$$ is a kernel function.

The model learned by NadarayaWatson MORE

2.2.2. Efficient Leave-one-out Cross-Validation¶

An advantage of the Nadaraya-Watson regressor is that it allows us to perform leave-one-out cross validation in a single evaluation of the regressor. This is useful because often the kernel function $$K(x, x')$$ is parameterized by a bandwidth parameter, say $$h$$ that controls the width of the kernel. The Nadaraya-Watson regressor can be written as $$\hat{y}(x) = \omega Y$$, where:

$\omega_i = \frac{ K(x, x_i) }{ \sum_j K(x, x_j) }$

MORE TO COME!!!

Examples