Fast and approximate nearest-neighbor search methods have recently become popular for scaling nonparameteric regression to more complex and high-dimensional applications. As an alternative to fast nearest neighbor

We present a novel algorithm for efficient learning and feature selection in high-dimensional regression problems. We arrive at this model through a modification of the standard regression model, enabling us to derive a probabilistic version of the well-known statistical regression technique of backfitting. Using the expectation-maximization algorithm, along with variational approximation methods to overcome intractability, we extend our algorithm to include automatic relevance detection of the input features. This variational Bayesian least squares (VBLS) approach retains its simplicity as a linear model, but offers a novel statistically robust blackbox approach to generalized linear regression with high-dimensional inputs. It can be easily extended to nonlinear regression and classification problems. In particular, we derive the framework of sparse Bayesian learning, the relevance vector machine, with VBLS at its core, offering significant computational and robustness advantages for this class of methods. The iterative nature of VBLS makes it most suitable for real-time incremental learning, which is crucial especially in the application domain of robotics, brain-machine interfaces, and neural prosthetics, where realtime learning of models for control is needed. We evaluate our algorithm on synthetic and neurophysiological data sets, as well as on standard regression and classification benchmark data sets, comparing it with other competitive statistical approaches and demonstrating its suitability as a drop-in replacement for other generalized linear regression techniques.

In June 2003 Swiss banks held over CHF 500 billions in mortgages. This important segment accounts for about 63% of all loan portfolios of Swiss banks. Since default insurance is not common in Switzerland, the corresponding risks are a severe threat for the health of the financial system. We focus the analysis on portfolios of residential mortgages and model the probability distribution of the number of defaults using a non-parametric approach, where the intensity processes associated to the time-to-default is linked to a set of predictors through general smooth functions: A generalized additive model is used to condition default intensities of mortgages on relevant economic risk drivers. We calibrate our model on a large mortgage servicing data set and compare the resulting loss distributions to a well-known benchmark, i.e. the loss distribution from CreditRisk as commonly applied in the industry. The conditional loss distribution and risk measures for a large mortgage portfolio are shown to be greatly sensitive to the prevailing socio-economic scenario. We present evidence that aggregated residential mortgage default risk is not only driven by the rating but also by variables such as the loan-to-value ratio, contract age, regional unemployment as well as contract rate changes and the contract type. Hence, it is crucial to integrate the significant factors into any reasonable future bank risk, portfolio or capital management framework or approaches for structuring and pricing of related products. We illustrate the severe shortcomings of the unconditional approaches regarding bank risk management. With our results we are able to contribute significantly to the ongoing international discussion about the appropriate drivers as well as modelling approa...