Disrupting retail, warehouse automation and the supply-chain industries: Why do Task Execution software offerings need a reboot?

Seven thousand five hundred forty-eight musicians gathered in a football stadium in Frankfurt to play for 45 minutes, conducted by Wolf Kerschek. Source: EPA

What do you observe in this photograph? A single person orchestrates 7548 musicians to play various instruments in unison — any tiny misalignment can throw the symphony out of its course. The central idea here is whatever we do — cooking food to taking our kids to school requires an orchestrated delivery of a hierarchy of tasks. When decisions are often time-sensitive and have a chain of consequences, we use aids like writing tasks on a piece of paper, prioritising them, creating a sequence where the output from one task becomes the input of another, etc.

Likewise, in the industry — retail, warehousing, transport & logistics, etc. — there exists software that aims to manage the number of agents (people, robots, devices, etc.) required for an enterprise (workforce management et al.) and what these agents end up doing (task execution et al.). Today’s piece will use the scaffold of product-market fit and utilise ideas from metaverse (or rather the subset that I call digital twin) to rethink building the next generation of workforce management and task execution software. For those fixated by ‘what is the value’ of such a product, I have worn a market research hat, and as a technologist, I go back to my core and wear my machine learning hat. In a subsequent piece, I intend to combine the solution architecture under an edge AI framework, the ML modules and a cloud architecture that such a product requires.

The product-market fit

Industries such as retail, warehousing, transport and logistics, supply chain are based on fine-tuned trade-offs between supply and demand. A delay can have a ripple effect throughout the chain of micro supply-demand trade-offs. I say micro because every stage of a chain has its supply-demand trade-offs to optimise for.

Managing a warehouse or a retail store with a high flux of products comprises solving optimisation problems at various levels. Optimising the supply chain feeding to the warehouse and the retail or online front feeding out of the warehouse to the robots used for surveilling and restocking items requires exquisite planning and prioritising tasks. Tasks come in different flavours — some are not efficiently preemptable, for example, an actuation task involving a physical resource movement. Other duties can hold the resource for stochastic periods; for example, time taken to pack order is a function of a worker’s age, agility, skill-set, etc. Similarly, tasks have a structure that can be linked in time (sequential) and space (multi-layer). Most task/workforce management systems have limitations to handle preemptable, dynamic and hierarchical tasks. They are often based upon hand-crafted heuristics or, at best classical operational research methodologies.

Preemptable, dynamic and hierarchical tasks require real-time decision-making algorithms.

To formulate a framework that can learn to orchestrate tasks, let us illustrate the shortcomings of the current workforce and task management systems. Most workforce optimisation and task execution platforms are static in a commercial environment! The agents demarcated for the tasks and tasks themselves are scheduled weeks in advance. Schedule malleability includes manual tweaks of the proposed schedule (mostly heuristic or OR based), defeating the purpose of predictive and prescriptive scheduling. The following issues compound the suite of scheduling solutions, necessitating a gap in the market for real-time task guided workforce orchestration:

Coupling workforce management and task scheduling systems:

★ Manual adjustment (based on local constraints) of a schedule by a store manager or a warehouse supervisor defeats the purpose of software-based scheduling (Butterfly effect).

★ Hierarchical task structure: refilling the shelf of cornflakes constitutes walking to the back-store, finding the shelf with the cornflakes, remembering the retail front-end shelf that requires a refill, walking to the stand, restocking the item and finally surfacing other products around the cornflakes. Most task execution ignores the hierarchy and sequence of sub-tasks.

★ Multi-echelon supply chain feeding a store/warehouse: a store often has multiple suppliers; small supply chain changes disturb the establishment’s day-to-day operation. The ripple effect is ignored as external disturbances are often not modelled.

★ Shades of constraints (managers vs individual contributors): People need to be scheduled differently depending on seniority or personal conditions. Personalisation of a schedule to a worker’s capability, designation and skill-set is often ignored.

★ Coupled schedules: a person is working two jobs, with one schedule hidden to the scheduler — creating partially observable constraints for scheduling.

★ Staff turnover: retail/warehouse jobs typically have high turnover where we can not expect the same level of prowess from a new employee as from a retail/warehouse veteran.

Resource utilisation under atomic and dynamic constraints:

★ Atomic resource constraints: workforce need to be brought in for a minimum number of hours. This is less of a problem for classic task execution systems as this typically ends up being a constraint for your mixed-integer program. However, having stochastic process modelling constraints is closer to the real-life scenario.

★ Variable timing constraints: the atomic constraint on resources puts a subsequent restriction on the end horizon. Stochastic end times for a given task are not modelled in industry-wide static schedulers.

Event-based workforce modelling:

★ Nuances of the current situation are ignored: stale historical data may not have a causal effect on the schedule. Just because you had a slew of orders in your warehouse last Monday does not mean the current Monday will reflect similar demand. The absence of real-time forecasting feeding into the scheduling algorithm is not optimal.

★ Unable to take care of edge events like spillage: health and safety events are punctuated and are always stochastic by their very nature.

Prioritisation of task backlog:

★ Ranking tasks to prioritise: Large organisations have long backlogs often not pruned by their relevancy or priority. If you have a limited workforce, should it not be prudent to use them for the highest priority tasks?

★ Unable to get feedback on completed tasks: Once a job is dispatched to a worker, the scheduler does not know when the task starts and ends. This is where computer vision comes in handy and tells us if a dispatched job has finished within a given service level agreement.

Gamification of cognitive psychology:

★ Opportunity for matchmaking: scheduler does not match the skill-sets of the agents and the task specification. This is where your workforce Tinder (not a love interest, but skill-set) matchmaking come into play.

★ Effective team composition: tasks that require more than an individual to work together requires the correct combination of team members to bring the task to closure (RL folks wear your co-operative game-theory hats).

★ Enable crowd-sourcing the training set of human behaviour: gamifying the retail, or a warehouse environment helps us collect data about human behaviour that becomes increasingly relevant from an operations point of view.

The Market value

Now that we have described the problem in the retail/warehouse/supply-chain task execution segment, let us see if the product can make money if built. Remember, there are tons of companies, big and small, that offer workforce management and task execution — these solutions have worked well for decades. Why would customers move away to something more risky, unproven and technologically bleeding-edge but “raw”? In the end, if we can not determine a go-to-market strategy for how such a product solves one or many of the problems described above, we need to pause and think before we end up building it up. Always start with: will I buy it whenever it is out in the market? And for how much? Does it solve the pain points of current scheduling and planning software?

Key Performance Indicators (KPI) and possible Markets and their TAM/CAGR for a Real-time Task Execution Product.

The TAM/SAM (Total Addressable Market/Serviceable Available Market) figures are pretty unenlightening for a product that does not have a clear cut yardstick to compare against. A much smaller subset called the Serviceable Obtainable Market (SOM) should be the focus instead. But such market analysis for a product that does not exist is often a finger in the air. It is like asking Apple/PARC what happens if you introduce a mouse with your desktop or adjudging the slew of iPhones on TAM/SAM some 30 years ago. Nevertheless, for the subject under argument, even if the SOM ends up being 10% of the TAMs mentioned above, we are looking at $900M, $560M and $980M for the workforce management, warehouse robots and supply-chain markets, respectively. Not an insignificant number!

Looking at the KPIs, it is easy to come up with concrete numbers to compare them against the current state-of-the-art. This includes an increase in service rate, stock visibility index, better control of slow-moving products, picking/packing costs, replenishment rate, downtime, schedule adherence, and other metrics. I will leave further scrutiny on this matter to card-carrying product managers and business development colleagues.

So far, we have discussed the problems with current task execution software and have gauged if there is a possibility of making money. We will sketch out what such a product can look like in what follows.

The Product: Real-time Orchestration using Sensors to collect information about the world, Computer Vision and Natural Language Processing to analyse the world and finally, Deep Reinforcement Learning to act upon the sensations

A real-time orchestrator: shows that the first incoming job (coloured red) requires five humans and two robots followed by two resources for the following two units of time. Each job has a different colour, enabling us to visually see the reinforcement learning algorithm’s state-space. The task queue is finite; thus, we separate jobs in the queue and those that do not fit the queue are maintained in a task backlog. There are currently three tasks (blue boxes) waiting to be prioritised in the job queue. The queue empirically can be considered a configurable task buffer of finite size.

Often standard techniques in planning assume full observability of the environment, i.e., if we plan for inventory arriving at a warehouse, the products’ location and quantity need to be known with utmost certainty. In real-life scenarios, such full observability is often challenging to achieve. There are always decision variables that have uncertainty associated with it or task structure that can evolve with time. Can task/workforce management systems learn to schedule tasks based on interacting with the environment?

Reinforcement learning (RL) systems offer two benefits — first, akin to game-playing agents, RL algorithms enable us to create a digital twin (a subset of a metaverse — pretty glad to see Mark Zuckerberg reigniting the metaverse idea) of the retail or the warehouse environment. They are based upon building simulations of the environment by construction, allowing us to reason across a gamut of noisy and incomplete sensory observations. Second, a continuous learning system enables the inference algorithm to optimise for various workload spikes and task complexity. Such a product contributes to a new task management system that alleviates some of the problems mentioned above.

Please feel free to get in touch if you want to have a sneak preview of what we have already built in terms of a cloud-based real-time orchestration product. It utilises computer vision to determine demand in a physical space, natural language processing to analyse tasks and reinforcement learning to backtest decisions in a digital twin before taking action in a real-world environment. I am confident that some of you will find utility in it.