It’s Only a Munger of Time

Wranglin’ the Night Away

Nicholas Teague
Aug 31, 2018 · 3 min read

In my last post I drew up some functions for wrangling structured datasets. An extension of this method could be to incorporate a function that evaluates columns in a dataframe to identify the presence of date or time series data in order to apply an appropriate processing algorithm. In this notebook we’ll create this function to automate the identification of time series data and update our automunge(.) function from the previous post to include this new category of data. This leaves us with the ability to automatically identify and process numerical, binary, categorical, and time series data from structured datasets. In the interest of brevity I won’t repost the processing functions that were introduced in a prior notebook, although they will be included in the companion Colaboratory notebook available [here].


1) Import data pre-processing functions from last notebook

process_numerical_class(.)
process_binary_class(.)
process_text_class(.)

2) Define process_time_class(.) function


3) Define evalcategory(.) and automunge(.) functions


4) Test Functions

Our updated train data for testing the functions.

Now let’s apply our automunge(.) and see how we did.

train
output numpy array for train set
labels
output numpy array for labels set
validation
output numpy array for validation set
validationlabels
output numpy array for validation labels
test
output numpy array for test data

Great well I think I’ll chalk this one up as a success. Until next time.



Books that were referenced here or otherwise inspired this post:

Code Complete — Steve McConnell

Code Complete

(As an Amazon Associate I earn from qualifying purchases.)



Hi, I’m a blogger writing for fun. If you enjoyed or got some value from this post feel free to like, comment, or share. I can also be reached on linkedin for professional inquiries or twitter for personal.


Automunge

Automated Data-Wrangling With Automunge

Nicholas Teague

Written by

Writing for fun and because who knows someone out there may find of value or at least amusing. I also write software to automate data wrangling at automunge.com

Automunge

Automunge

Automated Data-Wrangling With Automunge

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade