How to Build a Bayesian Ridge Regression Model with Full Hyperparameter Integration

How do we handle the hyperparameter that controls regularization strength?

In this blog post, we’ll describe an algorithm for Bayesian ridge regression where the hyperparameter representing regularization strength is fully integrated over. An implementation is available at github.com/rnburn/bbai.

Let θ = (σ², w) denote the parameters for a linear regression model with weights w and normally distributed errors of variance σ².

If X represents an n×p matrix of full rank with p regressors
and n rows, then θ …

bayesian bayesian-statistics integration machine learning programming regression ridge statistics