In 2019 QUT invited me as a technical artist to help them build what would seem to me as an alien planet — full of strange and wildly variant fungi or plant-like organisms I had never seen.
The Living Reef is an immersive and interactive 3D underwater ecosystem with complex artificial intelligence (AI) driving marine species to behave as they would in the ocean, and coral ‘grown’ using special algorithms that replicate what happens in nature.
With developers at The Cube Studio and QUT researchers including reef experts Brett Lewis and Luke Nothdurft we created 11 coral types, and 20 different fish species and other marine life ‘living’ through the 10-metre tall screens — numbering about 10,000 in total.
The secret to this success was in the ‘space colonisation algorithm’ and discovering it took me into the best rabbit hole around — YouTube.
YouTube gave me my ‘Ah-ha!’ moment
Much of my ‘research’ was done by searching programming videos, scouring old 3D forums and searching the names of technical directors from animated film credits.
My ‘Ah-ha!’ moment came when I found technical examples by Kevin McNamara who is now CEO for an autonomous data generation company in Palo Alto but in a previous life (2011) made coral generation samples which I guessed were used for Finding Dory (2016), the Finding Nemo sequel.
He simulated coral growth in the 3D animation software, Houdini, using the space colonisation algorithm — first used to model leaf vein and tree branch patterns by simulating the competition for space between those growing elements.
So, if I felt like a coral landscape was like an alien planet why shouldn’t space colonisation be the answer to my programming problem? Kevin’s examples had inspired me to learn more about the space colonisation algorithm and in doing so I discovered I had used it in my own past.
My first encounter with a space colonisation algorithm was actually in the mid-2000s where it was introduced to model leaf patterns and the branching architecture of trees. I worked in the construction visualisation sector and tree models were an important and time-consuming part of the job. At the time I didn’t know the software used this algorithm until much later.
Space colonisation algorithms have actually been used in several applications. These have also been also used to simulate crowds, mimic collision avoidance, crowd density relationships and to model crowd control. In the realm of visual effects, it could simulate inter-connected ecosystems to create environments like the machine world in The Matrix.
Building the Reef
Building a virtual reef using a space colonisation algorithm required defining the volume within which a model could grow then adding extra properties such as the availability of ‘food’ and what would happen when the algorithm reached that food. This gave it an ability to quickly simulate variations on a model that satisfied the given limitations.
Traditionally when programmers modelled an object which conformed to a given dimension or volume, they would manually create and move the model vertices and polygons to satisfy one single outcome within a space.
That was done visually — not mathematically — because 3D objects did not collide with each other the way real-world objects collided. And, while it manually achieved the same outcome as a space colonisation algorithm, this process was slower and less accurate.
By randomising certain properties using the space colonisation algorithm, objects could (and did) vary wildly. The model performed in a much more organic way and, ultimately, I produced an end product more quickly and faithfully, and with an element of surprise.
Coral reefs — the alien planet
Virtually speaking, I had lots of problems with coral. These organisms would be the landscape of our ecosystem within The Living Reef but there were so many kinds of coral it was hard to know where to start. So, I focused on creating a simple branch coral — like a tree without leaves.
I considered the classic L-System where code could iterate over a rule and create really organic results in branching trees.
I also tried parametric modelling — often referred to as biomimicry — which provided a simple model deformation like a bend or twist which, but when repeated across many objects made the underlying rhythm became very complex.
I added random values for variation, but my production generation experiments proved the parametric workflow was difficult to control.
It can be a showstopper in a production environment if you are not prepared for a direction that simultaneously commands growth in an area that cannot have growth.
How coral grows depends on such things as how much space it has, how it competes for that space, and the amount of food it can access.
My earlier processes blindly followed a directive whereas the space colonisation algorithm would start with a validity step like a set volume or amount of food to act upon.
So, we set those parameters, along with others, within the space colonisation algorithms, and generated coral of varying types and ages.
Using the algorithms, we were able to create unique pieces of coral within seconds. Digital artists could then add colour and place them throughout the reef scene.
We gave each fish species its own set of rules which followed as accurately as possible how those species behaved in nature — what they liked to eat, how they swam in schools, which of the other fish they liked to hang out with or which ones they avoided.
Within the digital environment, we let them go and they behave the way they would in nature within their rules, but without set paths. In a way, each fish was ‘thinking’ for itself.
Interactivity was built in through touchscreen activities for visitors as they explored The Living Reef. Visitors could fossick for shells in the seabed, free a turtle from a discarded fishing net and pilot an animated version of QUT’s underwater robot RangerBot to help grow coral and eliminate coral-eating starfish.
Mini-games in The Living Reef have been a big hit and there are plans to add more activities in the future, I am researching how to allow interaction with the Space Colonisation Algorithm which will expose options for editing the availability of food, show how 2 corals compete for resources within the algorithm and also introduce obstacles that the coral will need to grow around.
While it’s no Pixar production or scene from The Matrix, The Living Reef still achieved critical acclaim and great ratings.
It was the highlight of the 2019–20 QUT Summer Holiday Program at The Cube after launching on January 11th and will continue to screen throughout the year as we develop new elements.
If, like me, you are inclined to stalk developers for programming tips, here is a list of a few recommended reads:
Allan Bishop’s blog post on ‘The Living Reef’ which focuses on simulating over 700 AI-controlled fish — www.allanbishop.com/the-living-reef/
Ryan Bargiel’s ArtStation where he shares a close-up look into how much attention the different species of fish and their environments were given:
I mentioned Zaha Hadid’s parametric design applying biomimicry in architecture. Some samples:
For technical artists, I demonstrated coral creation and the interoperability of 3ds Max® and Side FX using the Houdini engine and research into runtime Houdini engine access within Unity.
About Lucas Milner
Lucas Milner is a technical artist. He creates procedural modelling solutions for The Cube, one of the world’s largest digital interactive learning and display spaces.
The Cube is housed in QUT’s Science and Engineering Centre and consists of 48 multi-touch screens across two storeys.
Collaborating with QUT researchers and drawing on knowledge and data from research areas in Science, Technology, Engineering and Mathematics (STEM), The Cube facilitates opportunities for the public to discover, visualise and contribute to research projects.