The Recent Evolution of Software Engineering
Part 1 of 3 — Owning your high tech destiny in the new world
Just over 20 years ago, software engineers were building business applications deployed on desktop computers with thick clients. Java was 3 years old, Netscape was the most common web browser, and Amazon was a 3-year-old online bookstore. If you had a fast internet connection at home, it was most likely powered by a 56kb modem leveraging your home phone line. The recent evolution of technology has changed how and what we work on, faster than most realize. Today, technology powered by Artificial Intelligence is now impacting our everyday lives and careers.
This three-part blog series aims to help you take control of your technology career. First by reviewing lessons from the past, embracing the technology of the future, and finally understanding the importance of human behavior for tomorrow’s tech professionals. It’s ok to do nothing, but if you want to make a meaningful impact on our everyday lives while having fun along the way, read on.
Why I started programming
After completing high school, I became an apprentice electrician to help fund my college education in electrical and computer engineering. I was working construction and design on large, automated electrical signaling systems for controlling coal trains from the mines to the ship loading ports. Projects took months and sometimes years to complete, and the physicality of the work was humbling, often resulting in a coat of black coal dust. Some friends of mine worked in software and I was envious of how satisfying it was for them to see results sooner, thanks to the iterative nature of software. Despite my limited programming, I was ready for the excitement, satisfaction and clean clothes a switch to software engineering would give me.
My first program
In 1997, my first real software program was written in C/C++ as an intern for Compaq Computers. I vaguely recollect this being a very small piece of an accounting business program. What I clearly remember was being focused on learning the idiosyncrasies of these programming languages. I was getting my first taste of UNIX, Object Oriented Programming and modeling classes in UML (Universal Modeling Language). Learning the basics like when to use pass-by-value versus pass-by-reference, all without scanning Stack Overflow for answers. It was a really satisfying and rewarding experience to iterate and improve the software at a much faster pace compared to electrical construction.
Looking back at how software has changed
Back then, I had no idea that many of the languages and tools that provided me a great foundation for software engineering, would become irrelevant. As advancements in hardware loosely followed Moore’s Law, so too did the software we use and develop. At the turn of the century, Java and the JVM became my game changer. I no longer had to be OS-aware and be a memory management expert; instead, I simply had to include prebuilt packages that allowed for rapid software development. Soon after that, software virtualization of hardware became the norm as machines became more powerful, eventually leading to cloud computing today. Advancements in storage capacity enabled big data and analytics. User Interface (UI) frameworks exploded with the march of JavaScript and libraries like jQuery, in support of modern browsers. Smartphones and tablets with resizing touch screens paved the way for responsive UI technologies like bootstrap and backbone. Open source was becoming popular and now beginner programmers had a sea of languages, platforms, databases, frameworks, and tools to rapidly develop software applications. I cannot count the number of languages and technologies that I have used since the 1990s, many of which are now obsolete or on the decline. Today as a leader, my teams rarely use relational databases. Everything is NoSQL, REST web-services are being replaced by GraphQL, in memory computing is common, and private data centers are being replaced by cloud technology. Recently we entered the age of Artificial Intelligence (AI) and Machine Learning (ML) which represents an entirely new world of opportunities.
Fast-forward to 2018
While having coffee with a new intern, he mentioned to me a crypto-coin-poker-game he was building in his spare time, where players bet online using currency from their own bitcoin wallets. It was born in the cloud, leveraging open-source cryptocurrency wallet technology, combined with gaming libraries. He had the basics up and running in a few weeks. I started to compare this to when I was an intern just some 20 plus years earlier, and the advancements in technology really sunk in. Rapid software development was a reality, even for an intern with limited software experience and little budget. I could not have had the money, hardware, or access to open source libraries to do the same in 1997. I can only imagine how different the software engineering landscape will look 20 years from now.
Which language should I learn?
On forums like Quora, you tend to see thousands of questions that all ask the same thing; “Which programming language should I learn?” Why do people keep asking this same question over and over again? A quick look on Quora shows you over 200K questions on the topic Programming Languages and some 1.2MM followers. People ask because they are seeking a path towards relevancy. They want career choices, respect, money and, equally important, a fun and supportive work environment that will nurture their growth. The programming language you choose to learn is the first foundational step in determining which career path to take. For example, searching for a software job in Cobol will lead you down a very different path than searching for one in NodeJs. Regardless, learning how to program and how to adapt to this fast-changing environment will serve you better than becoming a language specialist.
Following your passion will help you make the right decision. A data scientist might learn R and Python, while a web or Android developer would learn JavaScript and Kotlin. Golang could emerge as the new C++ replacement. Languages with large communities and a huge ecosystem like JavaScript, have a significantly lower barrier to entry than languages like Haskell or Rust. How will traditional software engineering change when quantum computers gain popularity? As you can see, the question of which programming language to learn is very subjective, with many applications and choices. Ultimately try and keep an open mind to learning something new.
Prepackaged technology changes coding
Practically, Java, Python or JavaScript are good languages to start with because of the wide adoption. For example, if you choose to learn JavaScript for UI development, a quick search online reveals that there are hundreds of JavaScript web framework options. Many of these frameworks solve similar problems. With so many pre-packaged technology options to choose from, software is becoming less about writing code and more about leveraging different technologies that someone else already built for you. A new software program that used to take weeks to develop and deploy, now comes prepackaged and can be downloaded, configured and deployed within hours. Out of curiosity, I counted the number of technologies bundled on a typical backend micro-services and found it was over thirty.
The cloud is changing how we build software
Cloud computing is changing the way we build software by making it accessible on a global scale. Simply put, cloud computing allows rapid software development by providing a catalog of paid, hosted hardware and software services via the internet. Before the cloud, most SaaS (Software as a Service) companies had teams that supported their data center operations, usually with a combination of network and system admin engineers. Hardware needed to be pre-ordered, racked, wired, deployed and maintained. I remember being in a startup in 2001 and having to drive thirty minutes to the collocated data center to make changes to physical servers — so inefficient. Today engineers are shifting away from deploying and managing always available software on hardware, to deploying on-demand lightweight compute functions like Amazon lambdas. So no longer do you need to spend valuable time and energy building out infrastructure and writing code from scratch. Instead, engineers can focus more on designing and developing your software applications.
Getting started with a credit card and internet connection
To get started today, all you need is an idea, a credit card and an internet connection. You can see how the various prebuilt services in the cloud have removed the obstacles I faced some 20 years earlier. Software entrepreneurs and startups now flourish with innovation, as they have a much lower bar to entry. However, you should exercise discipline and optimize your software in the cloud to avoid surprises on your credit card. Cloud providers generally have a freemium product for getting started, but be aware that the cloud is usage-based, so the more you use, the more it costs.
Changes ahead
As software engineers, we see first hand the evolution of the technology landscape as it continues to change. Sometimes we underestimate this rate of change. Step back and reflect on how your own career has changed. Open source technology and cloud computing are now table stakes, making our lives easier, removing friction, and opening us up to new ideas and innovation.
In part 2 of this series, we will examine how machine learning is changing the software landscape, at an even faster rate. I’ve added some of my own learnings and advice throughout to help you navigate this next part of your software career.
