RPA is not Screen Scraping

A failed technology is shamelessly being regurgitated as something new.

Three letter acronyms (TLAs) are part and parcel of the computer software industry and dozens come and go on a yearly basis. A recent one in particular is Robotic Process Automation, or RPA. If you do a quick search on robotic process automation, you will see ads from everyone from legacy BPM (business process management) and document capture vendors, to regurgitated QA testing tools called screen scrapers.

Worse yet, arguably reliable sources such as Wikipedia or for-pay “analysts” provide no clarification. Wikipedia’s definition is singularly focused on screen scrapers and misstates the place of APIs, while also describing the effects of RPA which are far beyond what screen scrapers could ever provide. For-pay “analysts” have magic quadrants containing anyone who paid them enough to be included — the actual software types are all over the map.

So what is RPA?

From the start, let’s agree that all software is a robot and all robots (their brains at least) are software. Therefore RPA could probably be left in the TLA dustbin of history, but instead let me at least explain what I believe it is, as my company adopted the term a couple years ago when it first appeared, and was defined much more accurately in my opinion. (I will not name my company or those of other RPA vendors as this is not a plug).

RPA, when it first was coined a couple years ago (and I admit I can’t find the original article where I found it) in my interpretation meant using new or advancing artificial intelligence (AI) techniques combined with new programming paradigms to automate labor intensive business processes. It was new-new. Examples of new or advancing AI techniques include machine learning, natural language processing, machine vision, and other statistical analysis techniques. Examples of new programming paradigms include, for example, the preponderance of REST APIs now available and easily accessible from the client.

So, RPA would seem to me to be new software brains working in concert with new application service layer technologies. Right?

The “OMG, They’re Screen Scraping” Moment

After going merrily on my company’s way for a couple years with the RPA term, I happened upon a public video of a “competitor” RPA demo on YouTube. Much to my (literal) horror, I saw a demo of a technology that had already been proven utterly ineffective in the 1990’s (or perhaps earlier).

Around the turn of the century, I was working as a programmer at a software company providing document management software in the form of a Windows desktop application. It was a typical Win32 application, with menus and lots (lots!) of dialog boxes, toolbars, etc. It did not have unit tests nor was there any API — not unusual in those days.

It was here where I first encountered a “UI test tool” — which is a software program written to act like a user and “click” buttons on another software program. A “robot” user if you will. Sound nice right? Cool!

The problem is it didn’t work. Didn’t work then, and still doesn’t work now, 20 years later, despite a crop of new engineers thinking this is some new idea, and (very!) old legacy vendors putting lipstick on their stable of pigs.

Why didn’t it work? Well, there was one guy in the QA department who managed the UI test tool process. While the tool could record a user clicking dialog box fields or toolbar buttons (sounds like the current RPA tools claiming to “watch and learn” from user interactions huh?) it would ultimately generate a script — a program — that was then “run” to perform the testing. Frighteningly, the current screen scraping RPA tools do this as well. They automate users into creating programs.

Every time development did a new release of software, the giant UI test script would break. Note that there was nothing wrong with the new release, perhaps just a label on a dialog box was renamed. But, on some releases, more time was spent simply making the test script work again than any productive testing of the actual release! The release was actually held up in QA because the test script wouldn’t run.

Screen Scraping is the Worst Possible Method of Integration

If screen scraping were such a great idea, and if it were excellent at connecting systems together, then why is there today such a push for APIs? Why do all modern JavaScript UI frameworks have built-in unit testing?

From my experience (and perhaps everyone’s experience) with UI automation techniques, be they OCR’ing screen sections or control tree interrogation, screen scraping is the worst imaginable method of integrating anything — even worse than not integrating at all. Furthermore, security constraints and related technologies are advancing counter to screen scraping technologies. Screen scraping gets around authentication, let’s just be honest about that. That’s why users sometimes have to install browser plug-ins to be able to interrogate secure Web pages. Screen scrapers work like keyloggers or other malware basically.

Not to mention it is perhaps the most brittle integration technology in existence. Any change in the underlying technology stack — from OS to Web browser to the actual application — will break the integration. The UI test tool I mentioned earlier broke entirely when applied to Windows NT because of the enhanced process security.

Finally, a little thought experiment: Imagine doing a data import into your new accounting system where you want to bring over 100,000 customer records from the old accounting system. Does anyone really believe a user could build a screen automation script between two 3rd party applications to do that? Nonsense!

Who’s the robot?

Requiring users to create unreliable scripts to link application user interfaces together is not progress. Requiring users to create a template for every document, form or screen from which they wish to extract information is not progress. Robotic Process Automation is not about turning users into robots to serve the application.

Let’s circle back to the original fair definition of RPA: Using new or advancing AI techniques that leverage new or advancing programming paradigms to automate labor intensive tasks. RPA is not, I would argue, a broken screen scraping technology from the 90s that automates users into writing programs. Nor is it user-created templates that define every variation of input for “automated extraction”.

After all, the robots on the automobile factory assembly line don’t need the same lights that humans do because they don’t have the same “eyes”.