Learn to create a rigorous workflow structure with Python

Looking across a field toward mountains in Colorado, United States
Looking across a field toward mountains in Colorado, United States
Credit: Image by Galyna_Andrushko via Envato Elements

Overview

One key to open reproducible science is to provide rigorous organization of all workflow code. Not just for when you send your project to someone else. A future version of yourself will also benefit when you return to an organized workflow after some time away.

I created a (with accompanying ) that provides a Python script and Jupyter Notebook each containing an example workflow. I use this structure for all of my computational projects. The workflow contains the following stages:

  • Environment Setup
  • User-Defined Variables
  • Data Acquisition
  • Data Preprocessing
  • Data Processing
  • Data Postprocessing
  • Data Visualization
  • Data Export

Workflow Stages

Below…


Learn to create a rigorous project structure with Python

Looking through a forest toward the mountains in Colorado, United States
Looking through a forest toward the mountains in Colorado, United States
Credit: Image by lensandshutter via Envato Elements

Introduction

How many times have you returned to a programming project after time away and the analysis failed? Broken file paths. Missing data. Code execution errors. I know what you are thinking: this sounds familiar. I have also experienced computational heartbreak. Let’s avoid that going forward. I’ll start.

One aspect of open reproducible science is to provide a rigorous workflow structure. Not just for when you send your project to someone else though. Adding structure will set a future version of yourself up for success. Imagine how thrilled you will be upon returning to a well-managed, reproducible project.

Overview

I created a…


Using Object-Based Image Analysis to Extract Alpine Tundra Features on Vermont’s Highest Mountain

Mount Mansfield, Vermont, as seen through the eyes of Google Earth.

Editor’s Preface: is back, this time with a look at the alpine tundra of Mount Mansfield. Download and check out the data from this work (). It makes a nice compliment to lower-resolution , and an even better one to the high-resolution (0.5m) 2016 statewide land cover set soon available from VCGI:


Georeferenced 1962 imagery in South Burlington overlaid on the same location’s 2018 color infrared imagery.

Editor’s Preface: Obtaining, digitizing, georeferencing and publishing historical Vermont orthoimagery are some of the many things we at wish there were more time and resources to do. Fortunately, we’ve been lucky to have met , a Vermonter and current graduate student in GIS and remote sensing at Penn State. Cale has graciously volunteered not only to georeference some of the digitized old black and white VT orthoimagery we’ve collected over the years, but also document for others the process and lessons learned. …

Cale Kochenour

Scientific programmer. Interested in all things remote sensing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store