My Challenges as a .NET RPA Architect

I often wonder what will be the great technological achievements that today are perceived as fiction, but in 50 years will be taken for granted. Since history is the most reliable predictor, I’ve asked my father about the most technologically-impossible Sci-Fi concept back in the 60s. He said: “I’ve seen in a movie 2 people hearing and seeing each other through a TV set. I could have bet my life that it was never to be accomplished in my lifetime and now I’m speaking to my grandchildren via Skype”. His answer made me widen my list, as I’m currently enabling robots to take over human jobs.

RPA in the last decade

RPA stands for “Robotic Process Automation” and refers to the automation of a computer-centric process using software robots. While concepts used in RPA, like screen scraping, data crawling or record-and-play automation have been around for a while, only in the last decade have a few RPA platforms reached the required level of maturity, resilience, scalability and reliability to make this technology viable for large enterprise use. This generated a rapidly growing demand in a market which, according to Gartner, is estimated to reach $4.9B by 2020. Forrester also provides an evaluation of the overall maturity of today’s RPA platforms:

Challenges?

The sales story around these technologies is that of a classical Rapid Application Development platform: with reduced effort and increased speed you can automate against any kind of operating environment (web, windows, Citrix/RDP etc.), interface with any kind of technology and integrate with any kind of legacy system. It sounds easy, but most RPA implementations face very similar architectural challenges.

Disaster Recovery

When trying to picture a manual process, imagine an employee operating based on some external trigger (email, spreadsheet, generic data source) into producing and/or altering data across multiple target systems (ERP, CRM, databases, various business-specific internal systems, excel sheets, emails etc.).

As opposed to classical application engineering, RPA will use various mechanisms to simulate the user’s actions and decisions. The execution of the automated process will depend on all those internal systems to be predictable, responsive and reliable, which is almost never the case. Long freezes, downtime or sporadic errors, that a user would creatively cope with, might result in a failed process step execution for an automated process.

As a result, a non-invasive technology, such as RPA, poses the challenge of ensuring atomicity for every transaction. This usually implies a state-machine-like design, breaking down the process execution into atomic segments, with retry or revert capabilities for every execution step. Strong state management is also vital for handling a wide range of business and application exceptions.

Concurrency Management

Moreover, a typical RPA candidate will be a high-volume process, which raises a requirement for workload balancing between multiple robots and, in turn, demands concurrency management. In the same way Thread Pools solve this challenge in Windows applications, the design of an automated, distributed process will usually implement a Dispatcher pattern or Producer/Consumer multi-process synchronization mechanism.

This furtherly implies the need of an external database for transaction management, coupled with a fully stateless implementation for every process step, as you have no control over which robot/environment will retry/resume the failed step execution.

Performance vs Reliability

All these design decisions might not seem like a bad idea at first, but every atomic, stateless process step adds additional layers of initialization, validation and cleanup, so the more granular the process breakdown, the bigger the performance penalty. In production, this is revealed as a difference in performance between the PoC/prototype phase, where RPA average handling time (AHT) is 3x-5x the human operator and the production version, where the scale already tips the other way.

There is no universal approach in solving this problem; reliability vs performance is a clichéd tradeoff, present in all IT engineering areas. However, driving a strong business analysis of the process is crucial for choosing an appropriate strategy in the design phase.

My 2 cents on RPA challenges

RPA is an exciting technology, with some mature, stable and reliable platforms available out there and a lot of success stories so far. Nevertheless, I often see these platforms being undermined, described as trivial drag-and-drop tools, that anyone can use. While that can be true when limited to simple, non-productive scenarios, designing a large-scale, enterprise-wide RPA project will definitely provide some interesting challenges.

I’d be more than happy to discuss further with anyone interested in the prospects and challenges of this exciting new technology. Also, if you’re looking for a career opportunity in the RPA field, check out the job opening we currently have @ Accesa.