A quick note on domain-specific evaluation of predictive models

A quick note on domain-specific evaluation of predictive models

This afternoon I attended a presentation on virtual machine demand prediction. The idea is to use these predictions to drive near-real-time…

Connecting a Raspberry Pi-based hygrometer to Azure IoT Central

Connecting a Raspberry Pi-based hygrometer to Azure IoT Central

Last week was hackathon week at work. I decided to do an Azure Internet of Things (IoT) project to learn more about Azure’s IoT offerings.

Auto-instrumentation with OpenTelemetry

Auto-instrumentation with OpenTelemetry

App instrumentation generally involves significant manual effort, with application code invoking logging/metrics/tracing SDKs when…

Two key challenges for time series analysis

Two key challenges for time series analysis

This post presents time series from a technical perspective, and presents two key challenges for time series analysis. It is based on the…

How to make good decisions quickly

How to make good decisions quickly

On teams, decision-making by dictator and by committee both suck. Dictators generate mediocre decisions quickly, and committees generate…

Stationarity testing using the Augmented Dickey-Fuller test

Stationarity testing using the Augmented Dickey-Fuller test

My team at work is building a time series anomaly detection system that automatically creates anomaly detectors to monitor application…

Clean up your time series data with a Hampel filter

Clean up your time series data with a Hampel filter

When building models for forecasting time series, we generally want “clean” datasets. Usually this means we don’t want missing data and we…

Reducible vs irreducible error

Reducible vs irreducible error

Suppose that we want to predict a value Y based upon a set X = (X1, X2, …, Xp) of variables. For the predictions to have any chance of…

Evaluating anomaly detection algorithms with receiver operating characteristic (ROC) curves

Evaluating anomaly detection algorithms with receiver operating characteristic (ROC) curves

Last week I wrote Evaluating anomaly detection algorithms with precision-recall curves, which explained one way of evaluating anomaly…

Evaluating anomaly detection algorithms with precision-recall curves

Evaluating anomaly detection algorithms with precision-recall curves

Ideally, we would like anomaly detection algorithms to identify all and only anomalies. But in reality this is easier said than done, as…

Nonlinear Regression in R

Nonlinear Regression in R

Yesterday I wrote about how to do polynomial regression in R, and noted that it’s really a form of linear regression.

Polynomial Regression in R

Polynomial Regression in R

At first glance, polynomial fits would appear to involve nonlinear regression. In fact, polynomial fits are just linear fits involving…

Anomaly Detection Using STL

Anomaly Detection Using STL

This post describes a way to model the midpoint of a time series involving seasonal and trend components. We’ll take a high-level look at…

On being open-minded

On being open-minded

Some years ago, my company did a leadership training event where they brought in an improvisation expert to help people practice the art of…

Transformation in neural networks

Transformation in neural networks

In Getting started with Tensorflow I mentioned that I’m taking a deep learning course by Tommy Mulc at Expedia. This week we covered…