Automation in a corporate environment using Microsoft technologies, Part 1

This is the first post of what I envision as a series of articles on building IT and business process automation solution with commonplace Microsoft products and tools found in many organizations. I will explore the ways in which SharePoint, PowerShell and MS System Center can be combined to deliver the end product of process automation, that is, happy business users AND happy IT :) In this article I will tackle the question of process automation in general and will give an overview of one possible approach to building automation solution, which we’ve successfully used in Kcell.

First of all, what this automation is all about? Any large organization has a lot business processes. Business process is when people coordinate their activities to have some work done, and the rules for how this coordination should happen can be quite complex. People also love when they can make some other guy or an automated tool to do some boring repeating work instead of them. However, people generally do not want to give up control over critical tasks, like putting a signature of approval on a financial document. Capitalizing on these facts, we, the corporate IT guys, can greatly benefit our organization and the business guys by giving them an IT system which:

  • Allows many people (both users and IT admins) to participate in the execution of a business process according to some predetermined rules.
  • Lifts the burden from their shoulders by doing some activities, which normally require manual labor, in a fully automated way by executing computer programs or scripts.
  • Enforces authorizations checks and authenticates all users before giving them a chance to interact with the system, so that only designated persons can perform certain actions, and records the history of all activities for future analysis and investigation.

Many sources and authors make a clear separation between two kinds of automation solutions:

  1. Business process automation (aka “workflows”)
  2. IT process automation (aka “orchestration”)

However, even if there are indeed a lot of specifics and peculiarities for each of these two uses of automation, I think that they are close enough so that in many cases an organization can choose to deploy just one system to cover them both. Or, as my personal experience suggests, an IT orchestration platform can become a part of the bigger organizational business process automation solution.

So what’s my favorite recipe for a proper automation solution? Enterprise IT team at Kcell, which I’m a very proud member of, hugely relies on Microsoft technologies and products to deliver best-in-the-country IT services to our internal users. A bit of self-advertising hurts no one, do it? :) Among the plethora of Microsoft products deployed in our corporate network, the following systems form the backbone of our automation solution:

  • SharePoint — as the central web and e-documents portal and the company wide workflow engine, well known by the users and used in dozens everyday business scenarios, not limited to IT processes only.
  • System Center Service Manager Automation (SMA) — as the IT orchestration platform, leveraging the power of PowerShell to complete various IT administration tasks via scripts.
  • System Center Service Manager (SCSM) with connectors configured for Operations Manager (SCOM) and Configuration Manager (SCCM) — as the ITSM solution, facilitating such things as IT tickets tracking, CMDB with automatic inventorying of IT assets and IT service health monitoring.

The trick is to use each system for what it does best and do not constrain ourselves by trying to implement the whole functionality on just one platform, even if it’s perfectly possible technically speaking. If SharePoint is good for hosting web forms and business process workflows, why misuse its great functionality by trying to squeeze some Exchange mailbox provisioning scripts into it. The same goes for SCSM, which is a great platform for implementing CMDB and also includes a built-in workflow engine and a customizable web portal, but the amount of research and arcane knowledge required for effective use of this functionality does not make it a pleasant experience. So yes, it is possible to do orchestration scripting directly in a SCSM workflow, but I found it much more easier to call external PowerShell scripts hosted on SMA (called “runbooks”) from SCSM workflows and do all the technical things like provisioning, config changes and etc. using the full might of SMA and PowerShell. And one additional benefit of using a different system for each separate aspect of automation, is that of uniformity. If people are used to approving their business-related requests in SharePoint, they will not understand why the IT guys would ask them to approve a “New mailbox” IT form via some ugly looking “Service Manager Portal” web site.

We use SharePoint to define the overall layout of business or IT process, to assign roles in the process to different users, to communicate with the users when a manual interaction (such as taking an approval decision) is required and to call the IT orchestration engine, with SMA for this role in our case, for automatic task execution. The SharePoint customization tools, ranging from SharePoint Designer for power users, to professional Visual Studio solutions with full source control, allow us to define workflow algorithms and design web forms of any complexity for every required scenario. Please note, that while it is technically possible to embed IT orchestration code or scripts (like Active Directory user account provisioning) directly into SharePoint workflows or web pages, we never do this, leaving this functionality to SMA. Also, many business processes, which we run as SharePoint workflows, do not relate to IT in any way, do not involve any scripts or automated actions and only concern people interactions. The main takeaway in our approach is that for both people-only processes like these AND IT processes where a lot of magic happens under the hood in code, we use the same common SharePoint workflow engine and web front end.

We use System Center SMA for hosting PowerShell code which is essentially IT orchestration done in the native language of the Microsoft platforms. PowerShell runs over .NET framework, is an integral part of all modern Microsoft operating systems and server products and can carry out almost any conceivable action related to IT systems configuration and administration. If something can be done via software code invocation and network calls in the Microsoft world, it can be done in PowerShell. Recently, many PowerShell modules for managing non-Microsoft systems and products have become available, so the power of PowerShell now reaches into other realms, such as HP servers hardware monitoring and VMware vSphere management. As a side note, there is an alternative IT orchestration platform from Microsoft, System Center Orchestrator (SCORCH), and several offerings from other vendors, but I’d strongly recommend to focus on SMA for many reasons which I’ll probably cover in one of the next articles.

We use System Center Service Manager as the central repository of all information related to IT processes, system and assets. It has all the tickets that the users opened with our Help Desk. It knows about all computers in the network and their configuration details as SCCM populates the database with this kind of information automatically via the connector. And it can be taught the concept of a so-called IT Service, or Distributed Application how it is named in SCOM, to manage groups of servers, databases, routers etc. as a single complex object and not as a bunch of unrelated CIs (Configuration Items). The role of SCSM in IT processes automation is that of the central register merging information from many different sources and serving it to all other parts of IT infrastructure. Please note that SCSM has its own workflow engine (!) with support for requesting user interaction via the built-in web portal, and its own ways of running PowerShell scripts and other types of code. But we minimize the use of both people-focused workflows and IT orchestration scripts in SCSM, because doing these things directly in SCSM is much more difficult from the technical point of view.

The second part of this series will have some diagrams and tables with more specific information on how the suggested automation architecture can be implemented in practice. And I will also tell the story of how SharePoint, while an imperfect product and lacking functionality in many respects, still made its way into our organization in almost a guerilla fashion and became the primary e-documents and workflow hosting platform.