AIXPERIMENTATIONLAB — Guide to User-Centered Development and Implementation of Intelligent Decision Support Systems

In an increasingly digitalized and interconnected work environment, where vast amounts of data are generated, decision support systems are becoming ever more important to provide a foundation for decision-making and to ease the workload of employees. This practical guide on user-centered development and implementation of intelligent decision support systems offers insights on how to systematically involve users throughout the entire process, from the initial idea to the evaluation of the implemented system. It presents an application-oriented mix of lessons learned, tested models, methods, and tools.

While many decision support systems exist today, only a few manage to progress beyond the initial implementation phase. The goal of this guide is to provide a practical framework for successfully implementing intelligent decision support systems in companies. This includes ensuring the acceptance of the new work system by users and improving decision quality through its use. To achieve this, the guide highlights potential challenges and provides specific recommendations for action.

This guide is designed for companies of all sizes that are considering the introduction of an intelligent decision support system or are already in the development or implementation phase. The primary target audience is those responsible for steering the project.

The guide is largely based on findings from the research project AIXPERIMENTATIONLAB. In the three-year project, interdisciplinary researchers from Augsburg University of Applied Sciences and RWTH Aachen University, in collaboration with two partner companies, tested the human-centered design approach using intelligent decision support systems as an example. The project followed a structured process that includes the following phases:

Phase 1: UNDERSTAND
The decision to introduce a new work system into an organization can be driven by various factors. To determine whether an intelligent decision support system — or any other technical solution — is the right fit for your specific challenge, we recommend the following steps:
(1) Gain a detailed understanding of the problem and define system requirements
(2) Evaluate the suitability of an intelligent decision support system
(3) Assemble the project team

Phase 2: CO-CREATE
In this phase, you can initiate all necessary planning and development activities, which focus primarily on four key areas:

(1) Technical solution
(2) Change communication
(3) Training for affected employees
(4) Project evaluation

Phase 3: REALISE
Once the new work system is fully developed and meets all formal requirements, we recommend a gradual rollout rather than immediate full-scale implementation. Begin by having a small pilot group of employees test the new system in a real work environment for about 30 days. This helps quickly identify and address any issues in a “lessons learned” process.

Phase 4: REFLECT
In the final phase, you evaluate whether the introduction of the intelligent decision support system has successfully achieved your intended objectives. A useful tool for evaluating such systems in a professional context is the Psychological assessment of AI­-based decision support systems (PAAI).

Upon completing the process steps, the system becomes fully operational and serves as a key driver in securing the future success of companies in an increasingly digital workplace by offering in-depth situation analysis and consistently engaging users. For a more in-depth look at the implementation, the full guide can be accessed here: https://doi.org/10.60524/opus-1814