Why Laziness, Impatience and Hubris Drives Great Developers to Script

Peter Campbell
Mar 1, 2016 · 10 min read

Laziness

The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful, and document what you wrote so you don’t have to answer so many questions about it. Hence, the first great virtue of a programmer. Also hence, this book. See also impatience and hubris.

Hubris

Excessive pride, the sort of thing Zeus zaps you for. Also the quality that makes you write (and maintain) programs that other people won’t want to say bad things about. Hence, the third great virtue of a programmer. See also laziness and impatience.

Larry Wall, Programming Perl

For many years I have enjoyed writing Perl scripts more than any other type of code. It is fast, immediate and very satisfying to automate some small niggle with a quick script. Now, developers are kind of tribal so there will be tribes who applaud this (thanks Perl tribe), others who will scoff (hello Python guys) and the others who aren’t sure what I’m talking about.

A Skip Through Scripting

Scripting languages are more popular than ever because of the rapid development possible with dynamic languages. And as software slowly evolves they are becoming ever more mainstream for production use. Lets try and categorise some uses for scripting.

Command line applications

In the beginning scripts were used for command-line processing. This is sometime called “glue” which characterises it well: it will allow you to quickly glue things together. Glue is used for handling command-line inputs and interacting with the console. File processing and string processing for example are very popular uses for command-line scripts. Many scripts for command-line processing can also be used interactively in the shell which simplifies testing and debugging.

Web development

In 2016, web development means both browser-side code and web server-side code. Scripting languages have popularised both sides.

DevOps

DevOps is something so commonly misunderstood that so much time is spent clarifying what it means. I’m going to assume you know what this is. And I’m sure you will have people in your multi-disciplinary teams who are using configuration management to describe your infrastructure, application deployment and configuration using code. Given that code is declarative not procedural it is not quite used in the same way as a typical script. Yes there is lots of YAML but scripting skills remain important to go beyond this configuration. These are mainly Ruby (Chef, Puppet) and Python (Ansible, Salt).

Embedded

It is common for software products that wish to be extensible to provide a high-level language for doing this. This fits perfectly with scripting languages. For example Lua is often chosen by vendors because of it is small, fast and built in C. However it is not uncommon to find Python or Ruby embedded. For example Mercurial the open source distributed source control tool is written in Python and can be extended with Python scripting. This is apparently why Facebook chose Hg over git so they could extend it for their unique uses.

Desktop app development

Who has written a desktop application using a scripting language? I have but this was many many years ago. I would guess that no-one under the age of 30 has. You rarely see desktop applications these days—in the old days one of the popular ways to do with was with Tcl/Tk which I haven’t seen for 20 years. I see thought that Microsoft has introduced the development of Universal Windows Apps for Windows 10 using Javascript but I haven’t seen any of these yet.

Choosing a Scripting Language

The point of scripting languages is rapid problem solving — often with an interactive session for immediate response. If a language supports rapid development and rapid problem solving I would classify this as a scripting language. But it is very very common for scripting languages to be dynamic languages and to be interpreted at runtime. The distinction between static languages and dynamic languages is becoming more grey every year as dynamic languages add more features (such as type checks and compile to IL).

Scripting With Perl

Last summer I wrote a Perl command-line script to parse AWS Elastic Load Balancer log files to identify slow response times given these are stored by default in S3 and we hadn’t ingested these to our ERK[1] cluster. It spins through some ELB log files and highlights the transactions that are slow — either upstream of the ELB, within the ELB or downstream of the ELB. It’s grep for performance in ELB log files.

=========/Users/peter/Developer/dig-scriptogram/elb/testdata/719728721003_elasticloadbalancing_eu-west-1_publishing-external-elb-https_20150828T0805Z_00.000.000.000_2a71z70z.log=========2015–08–28T08:01:42.778254Z - 3.181453 - https://yoururl:443/ (200)2015–08–28T08:03:24.892627Z - 6.553243 - https://yoururl:443/stats (200)
if ($class{body} =~ /(}[\/\*\s-=\w:\?@\.\;:]*\n\s*($keyword{classModifier})?\s*($keyword{class})\s+(\w+)(?:$keyword{extends})*(.*?){(.+)}?)/s)

Scripting Performance

When I learned Perl for the first time (circa 1997 when Perl 5 was newly released) it was considered speedy. Not speedy compared with C but better performance compared with shell scripts, AWK, SED, TCL, etc. Not challenging this I still consider Perl to be speedy in 2016. But is it?

  1. The bigger set is 24 hours worth of ELB log files for a large service. It has 1589 log files that contain 5,159,601 lines and are 1.6Gb. This set isn’t published.
$ time perl grep_elb.pl -t 3 ~/Developer/dig-scriptogram/elb/testlogs/*.log6.47 real 5.72 user 0.17 sys
$ time perl grep_elb.pl -t 3 ~/Downloads/elb-pub/*.log114.98 real 91.20 user 3.87 sys

Get Scripting

This got me thinking. What other newer languages could I have written this in? Are they more fun? Will they run faster? Is the code more terse or verbose? Would it be easier to maintain? Hubris again.


Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store