A Philosophical Primer in Pandas

Python-Pandas Evangelism From a Collegiate Historian-turned-Programmer

In 1949, a personal resume that included a legitimate High School diploma and “Typing” was worth considerably more than it’s weight in gold. Such a resume was potentially a passport to a higher professional strata; certainly for millions of young men who had returned from the battlefields in Europe and Asia, and often, for the women who had entered the workforce during that conflict.

It’s easy to see why typing was so valuable at the time — it represented a vector for information sharing that had finally been adopted almost universally in the post-war business environment. But for all the clout that “Typing” once commanded, claim it on your CV today and hiring managers will laugh you right out of an interview.

Typing died out as a special skill a victim of it’s own success — it became such a necessity, and so universal in the workforce, that it stopped being remarkable. In our own lifetimes, we’re witnessing the same process slowly-but-surely squeezing Microsoft Office expertise out of the range of “special” professional skills, to be replaced by a slew of shiny new talents that are desirable and scarce today.

Personally I view this as a minor tragedy.

Speaking from a historical perspective, Microsoft Office — or even just MS Excel — might truly be among the most effective tools ever devised by human beings. In terms of the sheer volume of data processed worldwide by any single invention, I would classify Excel in the same category as Writing, the Printing Press, and even the Internet. However, no workhorse long outlives it’s usefulness, and so “Microsoft Office” is gradually joining “Typing” in the blasé category of mandatory or expected skills. Modern workers are left to differentiate themselves with new skills, and one of the most tantalizing resume-boosters in the 2020’s will be Python 3 language programming.

Please don’t mistake my tone for mournful reminiscence, however! I mean this article to share a message of hope for the Excel master in the era of Python; the master bronze-smith in the Iron Age; the clay-cuneiform scribe in the era of papyrus! Why? Because the fact is, any true expert in the Microsoft Office Suite is a nascent Python linguist, whether they know it or not.

Each program in the MS Office suite is really nothing more than a clever shell — a mask — that makes common programming operations accessible and intuitive for a massive audience of contemporary workers.

At some level, this should strike you as a really, really obvious statement. After all, the MS office programs are just that: programs. Each one is a product of one programming language or another, and even the least-technically inclined reader will grasp that the overlay of an Excel spreadsheet is just a facade that assists information storage and recovery. But the core of my argument is this; if you are proficient with that facade, and you can use it as a platform to manipulate data using the functions and logic built into Excel, you are only a very short conceptual step away from Python- fluency. The gap between your waning “MS Office” skill, and the much more desirable “Python language” skill is more cosmetic than material, and you can make that upgrade more easily than you might think!

The best approach for translating your Excel mastery into Python, or at least the approach I am adamantly in favor of, is to pursue a specific discipline in Python that corresponds to your existing skills. If Excel is your particular field, look no further than the Pandas data manipulation library, an extension of standard Python functionality. Pandas is an excellent — if peculiarly named — tool that uses a logical framework intuitively familiar to Excel users, with a different, vastly more powerful vocabulary and range of applications.

Before I began using Python and the Pandas Library, the largest Excel file I ever worked on contained about 50,000 rows across 80 columns — approximately 400 thousand cells rich with important content. I plied that file with every scrap of MS Office talent I had, and I took great pride in the lumbering, overloaded Pivot Tables, graphs, and analyses that I was able to eke out of it with Excel. In a span of 4 days — lighting fast for my aging MacBook Pro, I cobbled together a working analysis of that file that I still regard as the pinnacle of my career in Microsoft Office. My work on that file was excellent, it was sophisticated by the standards of MS Excel, and it was grindingly inefficient by the standards of what I can achieve now, 3 months later, with an understanding of Pandas. I revisited that data file recently, and performed a similar analysis in 45 minutes.

In the end, my point is only this; upgrading your professional skillset from MS office to Python, or any programming language, entails little more than a transition from one set of similar tools to another. This is much more than a cosmetic difference, and while it will require dedicated work on your part, the benefits for your career could be the difference between gradual obsolescence and cleaving to the cutting edge! I exhort you; don’t be a Typist in twenty-twenty.

Financial tech analyst and programmer; I believe that transparency through Data Science is the defining next step in human progress this century.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store