AI + Forensic Facial Reconstruction
Using Artbreeder on sculptures made from unidentified remains of men found in BC, Canada. (15 faces project)
Can AI-enhanced forensic facial reconstruction help with cold cases? I’m not sure, so please tread lightly.
Families of missing persons could be traumatized if used improperly. In this instance the stakes are low because the identities of these men found in British Columbia, Canada are not known and unexpected details might jog the right person’s memory — as it did when the RCMP announced the 15 faces project.
RCMP has not responded to a request for comment.
The Idea
The idea for this came from the Colorized Statues subreddit which, in recent month, has gotten well-received artistic interpretations of Roman statues like Caracalla and Caesar.
For this experiment, I’ve only done basic brightness/contrast processing on the 15 faces before putting them into Artbreeder’s Generative Adversarial Network (GAN) — the technology behind this and similar AI/machine learning projects.
The forensic reconstructions were done in January 2020 by students at the New York Academy of Art. Each student spent five days applying clay to 3D printed skulls of unidentified remains as part of a workshop. The goal was to encourage tips from the public in hopes of bringing closure to families and loved ones. It indirectly brought closure to one family (details at end of article).
I want to be clear that I have no expertise in forensic facial reconstruction. My expertise is in image and video analysis. I do 3D reconstructions of crime scenes. I make no claims of accuracy. My gut instinct is that the results are simply easier to look at, that each step towards realism is a step away from ground-truth skull measurements.
Regardless, I think even failed experiments are worth learning from and maybe the results will jog someone’s memory.
The Results
Male, aged 40–60, discovered in Burnaby, British Columbia, 2019
- Green hoody with “Athletic” on front
- May have gone missing in 2013
- Case reference: 2019045788
Male, aged 25–40, discovered in Lytton, British Columbia, 2016
- Good dental work
- May have died several years earlier
- Case reference: 2019035507
Male, aged 45–55, discovered in Richmond, British Columbia, 2008
- Suffered from heart disease
- Full set of dentures
- Pair of boat shoes
- Case reference: 2012020145
Male, aged 30–50, discovered in Richmond, British Columbia, 2004
- Good dental work
- Gold tooth in left lower jaw
- Case reference: 2012020159
Male, aged 30–49, discovered in Delta, British Columbia, 1998
- Dark T-shirt with “Bugle Boy
- Wooden cross necklace on cord
- Case reference: 2014001023
Male, aged 30–49, discovered in Coquitlam, British Columbia, 1998
- Visibly decayed teeth
- Left knee deformity
- Osgoode-Schlater’s disease
- Case reference: 2012020165
Male, aged 50–65, discovered in West Vancouver, British Columbia, 1996
- Healed injuries possibly from major accident including:
- severe injuries to his right arm and leg
- a fractured skull, nose and ribs
- Case reference: 2012020147
Male, aged 40–60, discovered in Port Moody, British Columbia, 1995
- Osteoarthritis in neck and back
- Wearing Edmonton Psychiatric Centre underwear
- Case reference: 2012020169
Male, aged 25–40, discovered in North Vancouver, British Columbia, 1994
- Suspicious death
- Healed fracture to his right wrist
- May have had an old injury to the lower back
- Case reference: 2014001219
Male, aged 40–55, discovered in Parksville, British Columbia, 1992
- US currency in pocket
- Recently had surgery for ingrown toenails
- Healed collarbone fracture
- Case reference: 2014001116
Male, aged 45–70, discovered in Vancouver, British Columbia, 1990
- Several healed fractures, including one on the right side of the skull
- Case reference: 2014001040
Male, aged 30–49, discovered on Whistler Mountain, British Columbia, 1987
- May have been from South Africa or the United States
- Case reference: 2014001084
Male, aged 20–40, discovered on Hollyburn Mountain, British Columbia, 1984
- Tan suede cowboy boots, Double H brand
- Black leather braided belt
- Case reference: 2012020173
Male, aged 18–35, discovered in Chilliwack, British Columbia, 1972
- Brown hair, brown eyes
- Well looked-after teeth
- Case reference: 2012020177
The Process
- Step 1: Photoshop
The images provided by the New York art class lacked contrast and didn’t fare quite as well when put into the neural network. Several images would have errors processing, sometimes it would help to do a basic colorization. In this case, I increased the contrast and used the ‘shadow and highlights’ filter (similar to ‘pop’ filter on Instagram). In this case, I added Club Monaco glasses based on a RCMP photo.
- Step 2: Upload multiple angles and ‘crossbreed’
The uploaded angles are automatically cropped and mixed into the AI blender. Already, features like hair, wrinkles and eyes begin to look less uncanny.
- Step 3: ‘Edit-Genes’
This is where the artistry comes in. The ‘chaos’ control seems to revert the face back to some kind of typical/attractive face. The remaining controls are self-explanatory.
I found it was useful to use the results of this and repeat the ‘crossbreed’ step with the originals.
- Step 4: Photoshop
This was the most time-consuming process. It would be really helpful if RCMP were to photograph the clothing and glasses on a mannequin. Instead there were photographed on the ground so i had to find other source photos.
A Flawed Science
Wikipedia describes forensic facial reconstruction as “easily the most subjective — as well as one of the most controversial — techniques in the field of forensic anthropology.”
Put simply, when two experts are given the same skull the results vary significantly. Current methods are not admissible as expert testimony.
Still, public interest surrounding the project let to one person being identified. It was the boot description; not the face that led to a match. In this instance it might have helped that the sculpture lacked defining features.
In a story by The Globe and Mail, the author wrote “The reconstruction played an indirect role, at best.”
The mother said,“I was disheartened when I first saw it because it didn’t look anything like [my son],” she said. “My girls looked at it and disagreed. They said if you look at the ears and the eyes it’s him.”
Conclusion
The process remains both mysterious and unpredictable — qualities unlikely to attract law enforcement use.
In the future, I imagine someone could pair face-scans with CT scans and use machine learning to automate forensic facial reconstruction. However, it is such a niche and low-profit market that I doubt law enforcement will put any research into it. CT scans are expensive and unpleasant, so it would be hard to convince a thousand-plus participants needed to build a useful dataset.
I have only used Artbreeder for a week and I have much to learn.
ABOUT THE AUTHOR
Daniel Voshart is works in architecture and production design. He is currently on pause of his regular duties in Toronto, Canada due to the global health crisis. He writes and does forensic consultation in his spare time.
ABOUT ARTBREEDER
Created by Joel Simon while at Stochastic Labs. Generative Adversarial Networks are the main technology enabling this project. Artbreeder uses these BigGAN models and there is an open source version available.
