Small learning about Big data

With increasing interactions with and between devices, the volume of data being generated is way more than ever before. In the past, there were mere transactions, a finite number most probably. But now the huge volume of data can see patterns, tell stories, reveal problems, predict failures, save costs and even help in times of humanitarian crisis.

Among others, Internet of Things (IoT) is one of the many applications of Big Data. There is a lot happening around the topic of IoT, however, in the enterprise world, IoT solutions are at a stage of ‘infancy.’ Typical to this stage being — plenty of players in the market, both big and small, but a market that is evolving very fast.

It opens up plenty of opportunities to transform the way enterprises work. Rightfully so, a lot of interest and investments are being made in what is also termed as Industry 4.0 in a broad way.

Role of Empathy

Like before, transformational technologies naturally trigger the fear of technology taking over humans. Big Data based applications are often perceived as replacement of ‘expert users.’ The ethics often questions if it is right to ‘program’ the knowledge of experts and replace them.

Understanding how people work

While human behaviors and interactions with new technology evolve, Big Data applications replacing experts is a long way to go (if so).

It is nevertheless important for designers (especially during interviews etc.) and developers of such solutions to be aware of how the users feel. While some parts of the user’s tasks would be more efficient to be automated etc. it is important to showcase how the solution adds value to the user.

Having said so, Big Data applications would probably inherently force the need to talk to end users and include them in the design process — contrary to the transactional applications. Transactional enterprise solutions are conventional. Users of transactional solutions were typically provided with step-by-step instructions and training to carry out their tasks.

In Big Data based enterprise solutions however, most often, the users will be consuming a lot of analytical content — content that needs to be both explored and monitored. Different users usually do both of which differently. Therefore, it becomes almost mandatory to get a good understanding how users work with the content.

Evolution of Big Data applications for end users would be: 
Observing > Experimenting > Accepting > Trusting

Observing: They see a lot more content than before. It will be critical that the users don’t feel overwhelmed. 
Experimenting: Users will skeptically try to do some of their tasks, maybe even push the system to do things that it claims to do. At this phase, it is important that the solution plays along. It will be easy to loose the user’s interest if it doesn’t. 
Accepting: Realizing and feeling that the system in fact helps them do some of their tasks better. 
Trusting: The user works confidently with the system and relies on it. The challenge then on is to keep the user engaged.

The solution needs to bond with the user. This is very different from working through steps 1-x with conventional transaction based enterprise software.

Challenges

There are plenty of teething troubles to get past before the large data sets can really result in valuable insights for consumption. At this point in time, some of these teething troubles include:

→ On-boarding machines/equipment to a network of machines in order to collect data is a cumbersome process. A lot of manual effort is spent on this and there are very many different methods to do it.

→ Insufficient data volume: While some industries have been collecting telemetry data, the volume is often not yet enough to provide a big enough sample size to draw patterns, co-relations.

→ Incorrect data: The captured telemetry data sometimes do not correspond to the business goal customers are trying to achieve leveraging Big Data.

→ Different data formats and other data inconsistencies: Sometimes the telemetry data captured are in different formats and different time frames making it hard to aggregate.

All these make it difficult to ensure the insight provided is reliable for making business decisions (in the short term).

The basic design principle of IoT applications could be to ‘tell the user what they don’t already know/would spend too much effort to know.’

It is this aspect that has the potential to create an element of surprise — an aspect to be leveraged to engage the users, keep them engaged and gain their trust.

Initial design patterns

Configuration AND consumption

Until now, the wholehearted focus while creating the software was about consumption. This side of the solution needed to be intuitive, appealing etc. as mainly ‘non-technical business users’ would use it. The configuration side of the solution most often was created with bare necessities and with complicated content and interactions because it was mainly the ‘technical administrator/super user’ who would use it. For some reason, we accepted that the administrator didn’t deserve a good experience. 
 
In IoT based applications however, the business user would also configure some of the data. The business user would configure the thresholds of parameters; configure the visualizations of content, so on. In the enormity of the data pool, this will serve the purpose of creating the relevant context that the user would like to work with. While the ‘administrator/super user’ role wouldn’t disappear, the change in paradigm about configurations will present the opportunity to create equally compelling user experiences for both scenarios, unlike before.

Explorations and explanatory (monitoring) data

IoT solutions need to be able to tell the users both what they don’t know and what they know/need to regularly work on (explanatory data). Most often the users would work with content relevant to a given context — e.g. a given group of machines/machines in certain locations etc. Solutions like Predictive Maintenance would already add a lot of value to this existing content by showing the ‘health’ of machines, predict the potential issues with it or intelligence like ‘remaining useful life’ etc. 
 
Exploratory: In addition, with machine to machine (M2M) IoT applications offers the potential to create ‘co-relations’ between various data sets to:

→ System generated co-relations: Draw the attention of the users to situations they wouldn’t have unearthed/would have taken extremely long to do so. E.g. A the air conditioning in one of the trains in a certain route is having trouble. The problem might not be about the air conditioning system. But that of the quality of service when it was last serviced — a pattern showing other machines that were serviced at the same service station are also showing abnormal behavior.

“I love this equation where innovation is equal to creativity plus execution,” says Sam Yen, chief design officer at enterprise tech and software giant SAP. “You think about using computers to solve problems faster and faster. The data actually leads you not to solving the same old problems efficiently; thinking creatively leads you not to problem solving but problem finding. The data points you to where the real need is.” (Fast Company.com)
 
 → Combine data from various data sources to present a single insight.

Calendar Appointment reminder on iPhone

The example above of a Outlook calendar appointment reminder combines the calendar appointment time, location, traffic information and recommends me an action — leave now as the traffic is light.

Another integration example is Uber with Google Maps. It lets me compare the possible commuting options and the time each would take.

In maintenance scenarios, ‘fix rate in first visit’ is crucial. A potential comparable example could be:

A service/field technician has already followed 3–5 instructions as prescribed and yet, the task has not been marked as done. At this point, the system could prompt a message asking, “do you want to call an expert?” (providing relevant options within a context)

Exploratory scenarios could be presented us pull/ push mode and is critical to be presented as (system) ‘insights’ to the user. E.g. Auto Insights in Success Factors. It also needs to provide possibilities to allow users to interactively work with the case/modify it etc.

Auto Insights from SAP SuccessFactors — converts key analytical content into consumable ‘headlines’ concept that can be drilled down to learn more.

→ Leveraging supervised and/or unsupervised machine learning: This approach includes algorithms that find hidden patterns or structures in large data sets using clustering, classification and other statistical methods. This can greatly help providing easy and contextual content to consume for the user. E.g.

→ Alert fatigue is a very relevant scenario. Too many false/incorrect alerts are a distraction. In an ‘over simplified’ definition, these are like spam mails. Just like in email clients, by identifying certain patterns/descriptions, these could be hidden away. (spam folder)

→ When a user works with specific machines very often (frequency can be defined) these machines can automatically be displayed as ‘frequently used.’

Other (evolving) patterns

Displaying and interacting with data hierarchy

Machines are always part of a data hierarchy in the data structured — a machine is always within a logical grouping and there are always multiple smaller components within a machine. Companies hold this hierarchy very sacred. Until now, this hierarchy did not change too much. In future this could change, relatively speaking. With a larger volume of data, it is not very clear at this point if, working with such a hierarchy is very important to the user.

→ Would the user navigate this hierarchy often, to find the machine/component to work with?

→ Would it be better to provide a robust search engine as an alternative? — Hide the complexity.

On-boarding machines to a network

On-boarding new machines into the existing network and layout of machines are one of the key activities in an IoT solution. From a design perspective, it would be very important to consider how to catch the attention of the user with such an information amidst all the notifications, KPIs, status etc.

→ How would having a new machine within a categorized set of machines impact a KPI?

→ What’s the consequence if the user misses this information? Is it important for the user to know this at all?

Location details

Viewing location details is a common request from customers. However, at this point, it is not clear if location information refers to geographical location or location within a plant/site/building etc. It is not clear how valuable a generic geographical location (maps) displaying machine locations is to the user who frequently works with potentially same content.

Integrated (Google like) search

It has been observed that many content chunks within IoT based solutions will have ‘uncontrollable’ amount of data. ‘Uncontrollable’ content = content that will be infinite/will grow rapidly making it meaningless for (human) interactions like ‘sorting, filtering, re-ordering’ etc. It might be best to provide an all encompassing, Google like search engine within the solution. These scenarios might be more relevant while configuring different parts of the solution.

The combination of IoT and Cloud computing

provides a great opportunity for transforming Enterprise software to make it far more engaging to its users than ever before. Cloud offers new consumption models to enterprise software users — more potential to offer ‘consumer grade’ experiences that the users are used to in their private time. IoT is already increasingly penetrating into the day-to-day life with things like fitness apps on smart phones, intelligent devices inside homes etc. Experiences people have while using these solutions set the bar very high. It is no longer about interacting with one ‘thing.’ It is about interacting with end-to-end ‘systems.’

Unlike the On Premise world, where the users didn’t make the buying decisions of the software they wanted to use, the Cloud versions provide opportunities to change this. Newer business models e.g. ‘in-app’ purchases, ‘freemium’ could include the business users in the buying decisions. Many of such models are already extensively used in consumer (software) industry.


The basis of this article refers to my learning and observations from several co-innovation (with large enterprises) projects between 2011–2015. These projects were with customers across industry. I had the privilege to work closely with both the business stakeholders and the end users to understand the various business processes and thereby potential aspects within to bring value.

I would be happy to have a dialog, learn and evolve…
 satyaviswanathan@gmail.com