Creative Compression
Towards sustainable ways of looking
When we think of mass consumption, it’s usually the visible, physical kind that springs to mind. Towering mountains of paper and plastic. But what about the ocean of images that we encounter everyday on our screens?
Behind our online viewing habits, there is a world of invisible activity at play. From the creation of information to its upload, download, storage and consumption, energy is consumed. So much energy that data centers are now responsible for as many carbon emissions as air travel.
As a creative studio, we rely heavily on online imagery. While we may not be able to solve the global problem, can we use images to bring awareness to this growing issue? How much information, how much data, do we really need to see to understand an image, and what can we compress?
“Creative Compression”
Most compression techniques focus on preserving indiscriminately to keep as much detail as possible across the entire image. But as a creative studio, can we instead use compression as an aesthetic output? What if image compression could tell the same story as the original image? For this we needed to know what is important in an image, and what its meaning is.
This can depend on a number of things. Your article could be about ‘Furniture’, and in your main image, people might be seated on chairs in a large room surrounded by plants and other decorations. In this case, if you were to give a hierarchy of importance, the furniture, followed closely by the people sitting on furniture would be at the top. The plants, other decorations and background would be less important. However, if the same photograph was used in an article about ‘Plants and People’, the furniture is suddenly no longer the protagonist of the image.
AI Vision
What would these images look like? We wanted to use a tool that understands the nuances of a text, finds its focal point and processes the image selected. Modern AI allows computers to recognise and comprehend objects and people, which allows us to write software that ‘sees’ and thinks for itself what to preserve and what to compress. Knowing what parts of an image are interesting, or defining its ‘Regions of Interest’ (ROI), segmenting, and preserving their quality is an important step, but we still needed to define how we might effectively compress — or reduce the detail — of the rest of the image.
Some Tryouts
Pulling some of our studio’s talent together to share different expertise — spanning web experience and game design — and collaborate on some new techniques, we experimented with a few possibilities.
Cartoon Context
One thought was to use a tool that simplifies complex images into simple shapes, therefore creating a ‘cartoon’-like effect. By reducing the background to either vector shapes or basic colours with low levels of detail, significant savings in file size could be made.
Web Safe Photos
Once a computer can see into an image and work out its components, they can be swapped out for another visual element. Similar to ASCII art, which uses fonts that you have already downloaded on your PC, this technique can create recognisable images — one that might use emojis instead. Having an emoji keyboard installed on your device is now pretty common, so you could consider emojis as ‘web-safe’ and their increased visual complexity and rich colours serve to make good representations of original images.
Nested
When people talk about image optimisation, they often talk about how best to minimise the file size of an image by reducing colours, resolution, bit depth, without altering the format or layout. Image segmentation gives us the unique opportunity to rearrange and ‘nest’ the remainder of our image in an efficient and space-saving manner.
Progressive Loading
In search of ways to create an interactive, non-static method of image compression, we developed a technique to progressively load JPEGs, starting at 0 bytes up to the full resolution, 130 Kb, without adding any significant overhead. The user decides what level of detail is important to them, how much they want to see of an image — or whether they want to see the image at all.
This was a really valuable and interesting experiment, and although it’s not fit for purpose on non-custom sites such as Medium and Instagram due to its use of custom code, we want to develop it further.
Levels of Detail
We finally landed on a method that focuses on creating a background layer that progressively lowers in detail and resolution around the area of interest. Drawing on concepts from game-design and 3D design, where models and textures that are far away from the player are reduced in quality and complexity, we can hack our perception of images to sneak in compression.
Playing with how we see as humans, this technique aligns with our viewing habits. When we see an image, we don’t see everything in full detail: our eyes fall in specific locations, like people, plants, and furniture. Similarly, our AI Image Segmentation technique finds and isolates these objects of interest. A balance of aesthetics and practicalities, this outcome also does not detract from the original image.
Conclusion
Though these tools may not make a big difference on our total data-consumption, it got us playing with ways we can convey sustainability visually in order to inspire a more conscious mindset — one that encourages people to become critical of their image-usage. Thinking about our relationship to images and their meaning, against the backdrop of data-consumption and its impact on the environment, can provoke new, more active ways of looking. We’re interested in developing our Creative Compression technique as a tool to draw attention to the invisible power consumption going on behind the scenes by giving users autonomy in how much detail they choose to see.
Beyond experimenting, we’ll continue to work on these tools, figuring out how, and if, they can be applied to different content at the studio. In a digital-heavy industry like ours, thinking about what sustainability looks like in our design process on-screen is an important part of tackling our environmental impact.
