How do we handle the hyperparameter that controls regularization strength? — In this blog post, we’ll describe an algorithm for Bayesian ridge regression where the hyperparameter representing regularization strength is fully integrated over. An implementation is available at github.com/rnburn/bbai. Let θ = (σ², w) denote the parameters for a linear regression model with weights w and normally distributed errors of variance…

Performance has always been a high priority for C++, yet there are many examples both in the language and the standard library where compilers produce code that is significantly slower than what a machine is capable of. …

We use the module peak-engines to fit monotonic transformations to data — Ordinary Least Squares (OLS) fits a linear regression model to a dataset so as to maximize likelihood under the assumption that errors are normally distributed. Normality can come about naturally when error terms break down into sums of independent identically distributed components, but for many problems the assumption is unrealistic.

Use an optimizer to find the best performing regularization matrix — Let X and y represent a sample of training data where X is a matrix with n rows of feature vectors and y is a vector of n corresponding target values. If 𝐱′ is an out-of-sample feature vector with unknown target value y′, then we might fit a linear model…

Allocating memory from the stack in C++ is blazingly fast, but also restrictive. It costs only a few arithmetic operations but The allocation cannot persist beyond its function’s scope. The allocation must be small enough in size that it fits into the program’s stack. …

OpenTracing now provides an API to trace code written in Lua. Given Lua’s easy interoperability with C and C++, OpenTracing also provides a bridge tracer that implements the API on top of the OpenTracing C++ API, allowing you to immediately use existing tracers from Jaeger, LightStep, DataDog, Zipkin, or any…

NGINX is a versatile and popular application. Perhaps best known as a web server, it can be used to serve static file content but is also commonly used together with other services as a component in a distributed system where it functions as a reverse proxy, load balancer, or API…