Exhibition-ready quantum image processing
Today (March 22nd, 2023) is the opening day of QUANTUM ART: Creative Expressions of the Infamously Counterintuitive. This group art exhibition in collaboration between IBM Quantum and The New School showcases artwork and projects created using IBM Quantum Computers.
I am very excited about this art show for many reasons. But for this blog, I want to talk about my art in the show, and the new quantum image processing technique that I created and is used in my art piece. If you are looking for information about the show in general, see the post here.
Computational Legacy
Computational Legacy is an homage to IBM’s history and legacy of developing and popularizing computer hardware. IBM has been around for more than 100 years, and has created many important era-defining technologies. For a long time, IBM was a vanguard of making technology accessible to the public, similar to what IBM is doing today with quantum computing. I am excited to get to be a part of that legacy by working at IBM Quantum today.
Thus, it only seems right to create an art piece that calls out that legacy for the first quantum art exhibition in New York City.
— —
I started with an image of the model 5160 IBM PC from the mid-1980s, because of its iconic form factor and recognizability — even at the low fidelity that can be achieved with the quantum image processing technique. The image is still recognizable as a PC from the 80s with a faintly recognizable “IBM” written on the screen.
The image started as a drawing (above, left) of the hardware, which I then reworked to fit on a 60x60 pixel grid (above, right) so that it could fit on the quantum computer. I will talk more about the quantum computational aspects below, but for now, it is worth mentioning that the data from the quantum computer is what is rendered as the dots on Computation Legacy. Quantum computers are still noisy, so the final image is not a perfect representation of the inputted image. For example, each dot outside of the shape of the graphic is an error from the quantum computer.
Personally, this is one of my favorite aspects of the medium. Quantum computers may one day be fault-tolerant, but for now, the fallibility of the machines is a part of the aesthetic.
The final installation is painted on aluminum paneling, created using a pen plotter, a tool that I believe is a great fit for quantum art, which I have written about before. I love the idea of using metal as a part of quantum art because metal is associated with longevity and rigidness, while quantum states are incredibly fleeting, lasting only microseconds. The idea of embedding the essence of such a fleeting moment onto the surface of a permanent surface like metal creates a juxtaposition that makes metal a fitting medium.
This is the first time I have worked with paint pens on the pen plotter, which has let me work at larger scales, with Computation Legacy being 3' x 3'. However, the pen plotter itself has a limited draw area, and thus the piece was broken up into smaller aluminum panels to solve the practical problem of not being able to draw larger than 8.5x11 at any given time (constraints breed creativity, right?). The image’s separation into panels also compositionally connects to the smaller squares that make up the image. The final image is still easily recognizable as a 1980s IBM PC, but the styling is unique to this quantum art technique.
Exhibition photos
(Edit on April 2, 2023 to add) A series of photos from the exhibition. These photos were taken from the packed out two opening nights.
Near-Term Quantum Image Processing, and why it matters
In order for a computer to manipulate an image, the computer must be able to understand the image in its own native language. For classical computers, images are stored in bits. For a quantum computer to understand an image, it must be encoded in quantum bits, or qubits. Once an image is converted to qubits, quantum computations can be performed on the image, possibly opening the door for new and different techniques from classical image processing. This is an area of active research called quantum image processing (or QIP).
QIP was first proposed in the late 1990s and has seen much theoretical development over the course of the subsequent decades. However, almost all QIP techniques have remained in the field of theory due to limitations of currently available quantum hardware, with the largest experimentally processed images being only a few pixels.
The technique being used in Computation Legacy is a new technique I created for encoding images into quantum bits that can be executed on hardware that is available today. I will refer to this technique as “Near Term Quantum Image Processing” or NTQIP for brevity.
This technique makes it possible to process images on quantum hardware available today because each pixel is its own separate quantum state, meaning the upper limit for the size of an image is not limited to the number of qubits available on a given quantum processing unit (QPU). Instead, an image can be distributed to multiple quantum processors, or a single quantum processor can recycle qubits to process larger images. The largest image I have processed with this technique is 4096 pixels, using only one quantum computer, and there is no reason a larger image couldn’t be processed.
I used IBM Quantum systems ranging from 5 qubit QPUs up to 127 qubits from the Eagle processor generation. Fundamentally this technique is agnostic to processor type so long as the qubits can be manipulated with Pauli operators.
It’s worth noting that there are many types of quantum image processing techniques such as those explained here. NTQIP takes inspiration from other techniques and builds off their successes to create something very simple that is usable on today’s hardware.
Technical deep dive
A quick summary
Before we fully dive in, here is an extremely brief summary of what is happening: A new image processing technique is used to convert a bitmap image into a series of quantum states by mapping pixel values to qubit rotations. The quantum states are measured and the final image is drawn from those measurements. The full code for the project will be available via GitHub soon, but until then the technical explanation is available below.
Quantum Image Preparation
This project requires an image of limited resolution (I’m using 60px by 60px), plus Qiskit, Processing, and some sort of image editing software such as Illustrator or Inkspace.
To begin, we must convert an image of N pixel lengths into an array, which can be done with Python. The data of each index of the array is the pixel’s color value. The value needs to come from one of the color channels, either red, green, or blue. It is possible to process a full-color image, but in this write-up, it does not matter which color channel the data comes from because the image is grayscale. The color data for an 8-bit image should range from 0 to 255. However, higher-bit images can be processed as well, so images with a value between 0-N can be processed equally.
We frequently visualize qubits as spheres, such as the Bloch sphere whose north pole represents the 0 state (or ground state), with no rotation. The south pole represents the 1 state, with a full π (aka 180-degree) rotation required to reach the south pole. The important part here is that 0 is no rotation and π is the greatest distance a rotation can be without rotating back toward the starting position.
In order to map a pixel’s color value to a qubit value, we must normalize the color value to a value between 0 and 1 by dividing the actual pixel’s value iby the upper limit of a given pixel n. The normalized value is then multiplied by π so that it maps to our range between 0 and π. Essentially, the i/n ratio represents a fraction of the full π rotation. That rotation value can then be applied to the qubit by applying an x or y gate, which performs a qubit rotation either on the x-axis or y-axis. In this case, both the x and y gate rotations would have the same effect.
Below is an example in python pseudocode
q = qubit
r = x or y rotations
i = index value
n = index value upper limit
pi = π
q.r((i/n)*pi)To give a few examples of this, an entirely black pixel has a channel value of 0, and a qubit rotation of 0. When measured, it should return 0100% of the time and 1 0% of the time. An entirely white pixel has a channel value of 255, and a qubit rotation of π. When measured, it should return 0 0% of the time and 1 100% of the time. A gray pixel in the middle may have a channel value of 128, and a qubit rotation of π/2. When measured, it should return 0 50% of the time and 1 50% of the time. It’s worth noting that the 0 and 1 values could be flipped depending on if you want to draw a dark or light image. In the case of Computation Legacy, those values actually are flipped, but I find the explanation easier to understand without flipping them.
We repeat this process for every array entry being mapped to every subsequent qubit. Although each qubit is a separate quantum system, the entire set of operations can be stored in a single quantum circuit so that the location of the pixel is intrinsically stored in the qubit ordering on the circuit instead of using quantum logic to store the location. This is a very important difference in the technique. It is not necessary to use computational energy to store pixel location information. Instead, the location is purely an order of operations. This is a lot like how jpegs store image data in classical computing.
What happens when I run out of qubits to store pixels?
Computational Legacy started with an image created with 3600 pixels. The largest quantum computer available at the time of creating this piece was 127 qubits. So how can a circuit of 3600 be fitted to only 127 qubits? One of the benefits of the qubits not being entangled into a single quantum system is that the circuit can be broken into chunks. It could either be run on separate quantum computers or run sequentially on the same quantum computer. I chose to do the latter.
Once all the quantum circuits have been executed, the returned value can be stored in a two-dimensional array of binary data where the parent array refers to each qubit, and the nested array stores the measurement data from each qubit. As briefly mentioned above, the data is flipped so all 1s become 0s and all 0s become 1s so that a 1 can represent a dot of paint while a 0 represents a gap. This felt more natural to me, but isn’t necessary. The two-dimensional array is a bit finicky at this point because, in Qiskit, qubits are counted from right to left. This means that the nested arrays need to be counted left to right while the main array is counted right to left. Not ideal, I’ll admit, but perhaps somebody smarter than I can figure out a better version.
Either way, we now have a usable dataset containing the measurements from the quantum computer. Beyond this point, we are back in the land of classical computing to create the final artwork, which ultimately is up to the artist to decide on the best creative interpretation of the quantum dataset. I opted to use Processing, a java-based framework generally used by artists to create generative art.
Turning quantum information into an artistic representation
With a little bit of clever classical manipulation, the 2D array is usable by Processing. The dataset is saved as a .txt file and stored in the Processing data folder so that we can import it into the Processing program. Each nested array item contains the data from an individual pixel. It has an array entry for each execution (or “shot”) from the quantum computer. After a lot of tweaking and testing, I decided to only use 36 shots, which created a 6x6 grid of dots. The processing script will start with the last entry in the array (remember, qubits count right to left) as the first pixel in the top left corner. I created a processing script that will look inside that array to retrieve each of the 36 results and draw a dot for a 1 and a gap for a 0. It was necessary to tell the Processing script how many qubits we used up front so it will know when to start over on the right-to-left counting process.
Once done, the final image is saved as a single SVG. The SVG can be read by any graphical software. I used a combination of Illustrator and Inkscape to prepare the SVG for plotting with the pen plotter by sizing it appropriately and breaking it up into chunks that will fit on the aluminum panels. Breaking it up into chunks here has nothing to do with the limitations of quantum hardware, or anything quantum at all. At this point, it’s just an artistic interpretation on top of the limited space of my home workshop.
And with that, we have arrived where we started. An art piece generated with the NTQIP technique — an authentically quantum art installation ready to be shown in the exhibition!
What are the advantages of NTQIP?
NTQIP allows for images to be converted to quantum states and manipulated with quantum logic. Any quantum computations performed on an image processed by NTQIP can be measured and seen as an image, making for a much more visual understanding of what is happening in quantum computers. I also very much hope that it makes for a decent entry point for artists who are curious about quantum computing but don’t know where to start.
What are the disadvantages?
NTQIP does not come without some substantial drawbacks. The technique creates a series of totally separate quantum states that are completely independent of each other, despite being represented as a single quantum circuit in Qiskit. They are not entangled with each other and thus the types of operations being performed on the states are limited. This technique can also be efficiently simulated with classical systems. However, the technique does use the non-deterministic properties of quantum systems, and the individual quantum systems could unproblematically be combined together with multi-qubit operations, creating a richer computational space in which to manipulate images.
Other examples
This is the first time a piece of this technique has been put in a gallery but it is not the first piece I have created with this technique in this style. Here are a few images of others that I created and installed in Austin, TX.
Future technique explorations and extensions
I have also been working on extending the technique to process full-color images. For full-color images, each color channel is mapped separately to the quantum processing unit and recombined with classical techniques to reassemble the image. Classical computers use individually colored pixels where each channel only interprets one color, and this is generally the same idea.
However, I haven’t explored the full extensibility of this just yet. The real importance of the technique is that we can perform quantum computations on top of the prepared qubits. In fact, this technique is fundamentally quantum image preparation. In other words, it is preparing the image to be used by a quantum computer. No real quantum “computation” is happening yet. It simply converts the image into a series of quantum states and then measures them. There is space in the middle to perform different types of computations on top of them. It is possible to combine the individual quantum states into multiple larger systems where the pixels are entangled with each other. What can be done with entangled quantum pixels? I have no idea (yet). But there is a whole body of research into the area, and I hope that others will use NTQIP to find out. Until then, I’m going to continue to enjoy creating pieces of art with this simple quantum image processing technique.
If you want to see my debut exhibition art piece made with NTQIP, come down to the Microscope Gallery between March 22nd and March 30th, from noon to 6 pm. See you all there!
