Designing a data-driven company
The roadmap towards a powerful BI-powered and data-driven organization: from data capturing to automated decisions, optimal customer interaction layers, and a self-service analytics store.
A simple definition of Business Intelligence would be: ‘the technological framework which enables corporations to explore, analyze, and model large amounts of complex data towards one major goal: to improve business performance’.
Although simple and clear, there is a plethora of technologies, methodologies, and practices which, along with a rich terminology, allow gaps in business expectations.
Introducing a state-of-the-art Business Intelligence environment in the corporation also requires a new culture, a modern approach in decision-making
Based on accumulated experience — through ‘Datamine ltd’ — the most frequent misconceptions can be grouped as follows:
- Business users conceive Business Intelligence as the ‘magic box’ which somehow captures the business state of the corporation, generates predictions, and ensures business performance. Depending on their level, business users usually fail to understand that introducing a state-of-the-art Business Intelligence environment in the corporation also requires a new culture, a modern approach in decision-making.
- I.T. people typically underestimate Business Intelligence systems as ‘another database’ or some kind of ‘Reporting’. Although databases are the core elements of such environments and reports are key components, it is risky not to see the big picture: a proper Business intelligence environment consists of numerous packages encapsulating a range of technologies and applications. Also, BI involves several business processes and requires specialized human resources along with a new model of interaction with your data stores.
We see the following generic & simplified roadmap towards a powerful BI-enabled and data-driven corporation.
1. The data store
As a first step, you need tools that automatically gather your data, apply proper transformations & standardization and feed a data mart, a data warehouse, a document store, or other forms of database: the data store.
This data store must gradually be known as the central point of reference regarding corporate activity data. Every single corporate transaction, customer interaction, or business activity must be gathered in a standardized fashion and stored in analytics-optimized data structures. Ensuring Data Quality is critical here — to avoid the garbage-in-garbage-out situation.
Regarding the frequency (how often to collect/ refresh the data) this depends on the type of the underlying transaction, the business scenarios associated with the data elements, data availability in the source systems, and more. In general the more frequent the updates, the more data-ready for a ‘near real-time’ Business Intelligence environment.
2. Becoming data-aware
Having ensured clean and reliable data, you need some kind of tools enabling access — search, view, list, and also data summarization and visualization. This is where you typically have basic Reporting query capabilities. At this point, you have some basic data access & reporting, with some automation. Access by Business people is limited/ indirect since this would require additional platforms and tools, as described in the following.
3. A first Business Intelligence Layer
As your corporation becomes data-aware and the business questions set by management get smarter, you will need a family of tools, a platform, to enable business users to perform their own information exploitation, analysis, and visualization. This is a ‘dynamic Reporting environment’ which should add great value to your accumulated data. Your business users directly access your rich data store via flexible, interactive analytics tools, state their assumptions, and generate answers.
At this point, you can start communicating that you have some form of Business Intelligence infrastructure. Your business users will start realizing the value of the accumulated data and the need for some kind of additional data-analytic methodology that will somehow identify hidden patterns, trends, and critical insight: data mining.
4. Data mining & statistical modeling
Once the corporation realizes that accumulated data may contain answers to a range of business questions, data mining receives the focus. Since you have rich & clean data, (semi) automated statistical & mathematical models are applied in order to generate knowledge and meta-data such as customer classifications, churn predictions, consumer credit risk estimations, campaign response predictions, and more.
At this stage, you need a robust group of experienced data scientists ready to browse, model, and present the data in unique ways. This group must have the capacity not only to answer the questions set by the management but also to raise new ones, drive analytic incentives, and introduce data-driven ideas/ scenarios to top management.
Your business users continue to independently use the Business Intelligence platform since the data mining group acts in a supplementary way. In fact, the actual data store will be significantly enriched each time a data mining model is released to production or ‘findings’ are verified.
In the long run, the data store, the business intelligence platform, and data mining technologies, surrounded by skilled and motivated people will drive you to a new entity within the corporation: the Analytics Center.
5. The ‘Analytics Centre’
As the infrastructure gets mature, data should be enriched in terms of metrics & dimensions. This is a type of post-processing, transforming your initial data store into an Analytics Center. Data mining model outputs, new business terms, and definitions, dimensions, and measures reflecting market/ environmental changes, new types of customer transaction along with a wide range of customer metadata and business questions and ideas, analysis attempts, and results are all there, in the analytics-loop.
6. The ‘Intelligence Provider’
Now the infrastructure is a powerful analytics center that must become a corporate-level intelligence provider. Here you need robust integration with all these platforms and channels that will enable controllable diffusion of the intelligence towards the right management people.
Dashboards, micro-applications for smart devices, alerts, and messaging are typical examples of tools that will advance your infrastructure into a powerful Intelligence provider within the corporation. Your corporation at this point is Business Intelligence enabled. The prerequisite to becoming a smarter one is to use this intelligence for better product offerings, market positioning, customer management, process optimization, and decision-making in general. You need to enable your internal decision paths and your network of customer touch-points to automatically use this intelligence for real-time decisioning.
7. An Intelligent ‘Customer Interaction layer’
At the next level, you should be able to define, manage & assess packages of business rules, automatically generating proposals to your customers: a recommendation engine able to generate consistent customer communication scenarios through any channel, with feedback capturing and automated processing.
This takes into consideration user-defined business rules, customer lifecycle, and metadata along with the overall accumulated intelligence. Pricing policy, customer handling strategies, Retention and collection strategies along with specialized functions such as ‘save’ can all be designed implemented, and optimized as sets of rules, leading to the best proposal for any customer (taking also into consideration the state of the customer at the time of offering, the channel, history and lifetime metrics, product and financial goals).
8. Empowering the ‘Analytics Centre’
As the corporation grows and the market changes, it is of critical importance to identify new sources of ‘environment’ data and enlist a data feed towards your analytics center. For instance:
- Telecoms may integrate a feed with competitor’s tariffs (this would allow to evaluate competitor offerings on actual customer and traffic, and more)
- Retailers may gather, feed competitors pricing for certain categories of products (this would enable advanced strategies, pricing models, loyalty applications)
Although the above may be altered depending on the priorities and the industry, it should be clarified that:
- Your BI must be powered by a centralized, highly available, properly maintained datastore
- The internal goal should be to establish the so-called Analytics Provider, enabling the corporation for better decisioning, improved or even optimized functions. This requires a certain culture and attitude, especially in decision-making.
- The ultimate target should be to apply this intelligence and take customer experience to the next level.
As a summary, there is a clear path from initial data gathering to intelligent decisioning and intelligence diffusion, adaptable depending on the industry, level, business model, and priorities. In any case, the key elements for a data-driven culture and a successful corporate-level Business Intelligence layer, are data and people: you need as much data as possible (against time and also business activities) and creative people to (a) continuously question, analyze, model, predict and (b) take actions, apply data-driven decisioning logic, transform the corporation to a new data-aware instance.
Technology is certainly a critical factor, mainly as the set of tools to (a) make interaction with data and findings easier (b) enable high levels of automation and (b) ensure smooth, controllable, and consistent information diffusion throughout the enterprise.
