Book 3 was the year of Automunge. The first few essays are not exactly elegant writing, as I was figuring out how to code I was parallel figuring out how to document code. So these opening chapters were really just setting the groundwork for some of the more elaborate to come. I think the creative elements made these a little more palatable, as some of the coding aspects were obviously kind of dry. Once I found my groove the essay form started to return.
The Automunge software wasn’t created in a vacuum, for instance I reference several books in essays like A Family for Data and Automunge Complete that served as inspiration. However the bulk of development was fueled by what I would characterize as independent reasoning from first principles. Once I understood the core aspects of the need, such as normalizing numeric data and encoding categoric, and established boundaries for the scope of application, fleshing everything out was really just a matter of increments upon increments, each a slight improvement compounding upon the last.
During this period I mostly followed the approach of every new software update was adjoined by a corresponding essay. This made for some really creative writing, but turned out is not a very sustainable approach for software development. After completing this book I began to detach the software updates from essay writing and productivity impact was material.
Although it wasn’t exactly beneficial to the Automunge project, the sabbatical from December to March turned out to result in a really neat literary effect somewhat like an intermission, as got to turn focus away from software to a few more creative interests. Dropping Out for Fun and Profit was my first attempt to at least approach conventions of academic writing. I had a lot of fun with this paper. It was not well received when submitted to a conference workshop, I suspect because the movie reviews were too much of a distraction from the algorithmic contributions. Only wish had followed my own advice a little sooner.
A few of the essays in this intermission period I would characterize as kind of like vacation journals. Machine Learning and Miscellanea came from a fun trip to New York. I’m a big fan of the Met, is an amazing environment. The Final Frontier from visiting NASA, and Weather Report from good old New Orleans. Yeah this was a fun respite, but finally realized that if anything was going to come of Automunge needed to go all in.
The Automunge essays after the intermission period I think did a better job of resuming the essay form. Some of these I think turned out quite well. There was almost an aspect of crescendo at play, and the closing essay was easily worth the year of labor.
Automated Data Wrangling with Machine Learning Derived Infill for Missing Values
The shiny new thing