Design + AI

Automated Invention & Creativity

The traits that make us human and that we are automating little by little

Untitled #53, from a series of explorations

Invention and Creativity are traits that make us humans human. But how do you develop these? And if there’s a way to do it, can it be automated?

The following are brief thoughts I had around these questions, which were ignited after reading the latest research by OpenAI and GPT-3, DALL·E: Creating images from Text.

First, let’s start with some context and definitions.

1. Repetition & Iteration ♾

Repetition and Iteration are (some of) the basic tasks that can help us develop a resilient and creative mindset.

Here’s how I see it:

• Repetition is doing one thing, multiple times.
• Iteration is doing one thing, in multiple ways.
• Invention is doing something new.

2. Physical and mental abilities 💪 🧠

Now, let’s add the layers of our physical and mental abilities to the picture. Here’s how I would combine them:

A physical activity + Repetition = To master precision.

A physical activity + Iteration = To develop dexterity.

A mental activity + Repetition = Memorization.

A mental activity + Iteration = Exploration.

Here are some examples of the previous formulas:

• Exercising is a physical activity you improve through Repetition.
• Dancing is a physical activity you improve through Iteration.
• Learning kanji is a mental activity you improve through Repetition.
• Writing a poem is a mental activity you improve through Iteration.

On that same note:

• Learning to play guitar (Repetition) is different than composing a new song (Iteration).
• Learning to code (Repetition) is different than creative coding (Iteration).

And so on.

Here’s the idea visualized in a 2x2 matrix, try to come up with your own examples and map them to it.

Map your examples to the matrix

3. A note on Creativity ✨

All these previous definitions and examples are to say:

Creative people do not only have “Creativity” going for them. Repetition and Iteration are foundational pieces that enable minds to reflect on their explorations. This act of reflection leads to mindful creativity.

With this framing in mind, we can now we can move to the DALL·E article.

4. DALL·E vs Mind 🦾 🧠

It is both awe-inspiring and intimidating to witness a neural network like DALL·E generate a large set of iterations related to any given input.

Here’s a subset of results to the query “an armchair in the shape of an avocado.”

DALL·E’s output to the query “an armchair in the shape of an avocado”

It is fascinating and scary. It is entirely logical to think that a more sophisticated version of this neural network model could end up automating the job of interior designers. Instead of working with an agency, just ask DALL·E, and get a wide range of iterations in high-quality.

Here’s a feasible scenario for the future:

A family wants to redecorate their home. Pointing a Lidar-capable device to a space in the living room they say: “furnish this space with a modern living room ideal for a family of 3 and a corgi.” And this query would be enough for the system to generate enough variations (all of them excellent options) for the family to pick from.

They pick their favorite, and each object in the image is transferred into a production-ready 3D model that gets 3D printed and delivered on the same day.

It certainly could be a reality.

However, here’s how I’d summarize my observation/concern:

The capacity of reflection gets lost in automated iteration.

Conscious reflection is a core differentiator we have against neural networks that are capable of zero-shot reasoning. (Zero-shot reasoning is the name given to the ability of models to recognize objects they have never seen before.)

When we delegate the Repetition of mundane repetitive tasks to a machine, we may end up sacrificing our natural capacity to develop precision.

But when we delegate Iteration to a machine and forgo the experience of exploration, we end up sacrificing our natural capacity to develop reflection.

Closing thoughts

I’m excited about the outcomes and advancements in the research of neural network models. I like to think about the new questions and challenges they bring to us.

The experiments from DALL·E make me reconsider the role of designers who iterate on ideas, and the managers or directors who curate the solutions. It’s a common dynamic that exists in agencies and corporations.

The fact that DALL·E is capable of generating what could be interpreted as creative iterations makes me wonder: Who is the creative agent here? Is it the model that came up with the results? Or is it the person who crafted a clever query?

Does creativity lie in coming up with the phrase “an avocado-shaped chair”, or is it in the act of exploring what an avocado-shaped chair looks like?

…

What if “Let there be Light” was the input God gave to a model, and we are just the output that came out of it?

I use Twitter to talk about design and prototyping tools. If you liked reading this post, follow me there as I continue to document findings and observations related to Interaction Design.