Casting call for bad actors — how drones and AI are spotting violent people in crowds.

Crowd control is a serious issue. Serious because whenever large groups assemble, there is a possibility of violent outbreaks between individuals which can quickly escalate to chaotic mass brawls.

Rapid response to crowd violence and deescalation offers widespread and significant protection to innocent bystanders, nearby civil and government infrastructure, and even to the perpetrators who might otherwise remain disengaged from violence when not influenced by ‘mob violence’.

Drones and AI are here to help quell violence by recognising and identifying individuals engaged in violent behaviour. Rapid identification of violence allows authorities to allocate vital human resources to other tasks unsuited to AI such as public engagement. One benefit is that AI doesn’t suffer the fatigue CCTV analysts are faced with resulting in fewer false negatives, i.e. when the tired eyes of an analyst fails to recognise when violent episodes are occurring.

Ways humans commonly recognise violent behaviour in crowds

1. Anomalous crowd behaviour such as rapid and en-mass erratic movements (groups of people moving towards/away from a location or chaotic movement of people in all directions).
2. Characteristic body positioning or stance indicating violent actions (choking, punching, shooting, etc).
3. Facial expressions are a good indicator of either imminent or in-progress violent events

Researchers from both the UK and India have collaborated in an innovative manner using consumer-grade drones to identify violent individuals through violent posture recognition.

Images of violent activities used to train the AI data set; strangling, punching, kicking, shooting, and stabbing. (via University of Cambridge/National Institute of Technology)

An illustration, for the purposes of illustration

We’ve all drawn people as stick figures. While seemingly basic they efficiently communication actions and events. Using human body nodes, i.e. ankles, knees, hips, wrists, elbows, shoulders, neck, and head, a person’s stance can characterise their current behaviour based on the relationship (angle and position) to other body nodes. The figure below illustrates how criteria such as the position of arms in relation to the body are a good indicator of the current action.

AI uses the relationship between joints (shoulders, elbows, wrists, etc) to identify violent postures

The hardware behind the AI

With new developments and drone iterations happening regularly, many consumer drones become seemingly redundant in a short time. While that’s great for drone consumers and suppliers, it’s also great for innovators who realise that even basic drone technology can be utilised to supplement complex AI. Here, developers utilised Parrot AR drones as the platform for photo acquisition and cloud uploads. By interfacing with the AR’s operating system (hooray its Linux!) via on-board Telnet and FTP servers, downloading images real-time to a cloud computing platform is no big stretch. So it’s comforting to know that while this drone was first released eight years ago, it’s still performing a useful service.

First released in 2010, the Parrot AR drone’s hackability has ensured that it will be used for years to come

How AI discerns between violent and non-violent individuals

Two thousand images, each containing one of five violent postures; strangling, punching, kicking, shooting and stabbing, were identified and utilised as the training data set for guiding AI recognition of violent people. Image processing was utilised to determine body silhouette and from that a simple skeleton, i.e. stick figure, provides the characteristic stance for each of the five violent postures.

The stats on AI to identify violent people from aerial images

1. Ability to to identify people from aerial images: 97%
2. As the number of people within an image increases, identification of violent individuals decreases.* For example: An image containing one violent person is identified 94% of the time. An image containing five people is identified 84% of the time.

*This relationship supports the value of AI in violence recognition as images containing a multiple violent individuals are easily detected by humans, it’s the isolated events and individuals that are hardest to detect.

Using an eye in the sky to look ahead …

The study represents a great leap forward in offering crowd protection. While the identification of false positives is possible, other methodologies (movement detection and facial expression recognition) used in conjunction with posture identification can only make identifying the ‘baddies’ that much easier.