We’re Shaped by Our Creative Tools
Narrowing the Creative Language Gap with Machine Intelligence
By Archie Bagnall, Lisa Jamhoury, Madeline Hsia, and Patrick Hebron
There’s a language gap present in today’s creative landscape. One that separates our creative applications from our creative intent. The gap is the reason we are required to “on-board”, the reason we find ourselves disoriented when a button moves after a software update, and the reason we watch online videos to learn features our tools have in fact offered for years. We accept the gap, because the historic nature of computers — binary execution of pre-programmed functions — has caused it to always be there. This gap demands that users speak like a computer so that they might effectively use a computer.
Today’s creative applications could be described as a collection of digital tools: a digital paintbrush, a digital typesetter, a digital brightness control, et cetera. Together, they deliver an experience that asks users to translate their creative intent into a sequence of tools. For example: to “highlight the colors of the sky” in an application such as Photoshop might translate into the following tool sequence: Magnetic Lasso Tool -> Hue/Saturation Tool -> Brightness/Contrast Tool. There are multiple tool sequences that could achieve a user’s creative intent. Users have many multiple tool combinations and sequences available to them in order to achieve their intent. Sequences can develop their own vocabularies and language shortcuts around certain creative tasks. Single tools can also achieve a wide array of outputs depending on how they are used and manipulated.
The language gap reveals limitations that users persist through as they exercise their intent within creative applications. Here’s three in particular:
- First, that their ability to enact creative intent is limited to the language of tools. Users cannot successfully use an application unless they interact with it using the language of tools.
- Second, that their ability to enact creative intent is limited to their knowledge of the tools available to them, including their ability and availability to learn new tools. If a user’s intent extends beyond their knowledge of the available tools, they must either consult documentation outside of the application, or alter their course towards use of more familiar tools.
- Third, that their ability to enact creative intent is limited to the capability of the tools available to them. If a user’s intent is greater than the function of their tools, they must either change their intent or look for new tools elsewhere.
Two decades ago, John Maeda was interviewed by the New York Times. He talked about the language gap in relation to Adobe tools:
“The problem is that most people can’t just ‘’finish’’ things with [an Adobe product]. They have to use it to start them, also… , the styles that emerge are homogeneous because the software is universal. Without being able to know how to program, you can’t break out of the technology.”
Maeda recognized a consequence of the gap: that the language of tools can subvert the language of creative intent. Rather than continue translating intent across the gap, users can instead develop a habit of formulating their intent directly in the language of the tool. Maeda describes the outcomes of this phenomenon as “homogeneous styles”, a devolved sameness that emerges when creativity starts at the opening of an application, and is then led only by the tools that the user knows. Maeda’s sentiments are echoed in this common quote:
“We shape our tools, and thereafter, they shape us”.
If the language of tools is by nature, binary and mechanical, the language of intent is contextual and human. Where a creative application hears an intent of “select the brush tool and place four brushstrokes on the artboard”, a user might speak that same intent as “turn a chair into a barstool”. Intent can also be vague and multi-layered. Vague, because another user could achieve the same outcome with “make the seat taller”. Multi-layered, because the task might contribute to a greater intent of “re-designing a restaurant interior”.
When directing his junior designers, Massimo Vignelli would famously describe a visual layout as needing to “bang” or have “reverence”. His intent could simultaneously be classed as human, contextual, vague, and multi-layered all at the same time, where his designers could only learn the true meaning of the direction through experience. It highlights the multi-dimensional complexity of creative intent, and raises question as to whether a creative tool is capable of accepting and facilitating “bang” in the way Vignelli meant? Will technology allow for the language gap to be narrowed?
At the beginning of this piece, computing was described as a historically “binary execution of pre-programmed functions”. In this style of computing functions are rules-based, and rules are pre-programmed into software before their release. The resulting functions are defined, engineered, and inflexible to adaptation.
Machine learning offers an alternative style of computing that takes a different approach. Here, intelligent computing is still rules-based, but the rules are not pre-programmed. Instead, machine intelligence infers what the rules should be after recognizing patterns and relationships in a dataset. This method allows intelligent functionality to be adaptive to the individual user, following rules that are calculated to best meet each individual request. This logic allows a pair of users to give the same search term and receive different results, or a single user to input the same search term at different points in time and be given different results. In both instances, a greater number of inputs beyond the single-given search term are utilized in calculating the most relevant result.
Through contextual learning and dynamically refined outputs, machine intelligence offers to learn the language of the user, and in doing so observe the subject and context of a user’s content. In recognizing the human language of intent, users are empowered to engage their creative applications without first needing to learn the language of tools, and creative applications are empowered to “highlight the colors of the sky” or “turn a chair into a bar stool”.
Below are visualizations of “turn a chair into a bar stool” and “highlight the colors of the sky”. The first image shows the steps taken when translating creative intent into the language of tools. The second suggests what’s possible as the language gap is narrowed.
We’re already starting to see the language gap narrowing in today’s creative applications. Below, Photoshop’s ‘Face-Aware Liquify’ feature allows for the semantic editing of human faces.
There’s a language gap present in today’s creative landscape. One that separates our creative applications from our creative intent. But advances in machine intelligence are narrowing the gap and allowing us to reimagine creative applications of tomorrow. These applications won’t just offer a collection of enhanced “smart” tools. They’ll offer a personalized relationship between a user and their creative intent. They’ll be active facilitators in the creative process and of our creative navigation. They’ll bring new creative possibilities for application users and application makers alike. They’ll bring challenges too.
Machine Intelligence Design is a team at Adobe Design, promoting a user-centered approach to machine learning and artificial intelligence in creative tools, while simultaneously working to establish deep understanding of how these technologies are changing the ways we interact and create.
Adobe Design is hiring!
