Quantum Image Manipulation
Section I: Quantum Procedural Generation
Background
Our implementation of image processing is based on the 2020 paper, “Procedural generation using quantum computation.” The paper described the process of combining quantum computing with procedural generation to create an image-blurring effect. Procedural generation is a technique used to create data algorithmically with a computer and is commonly used in video games to quickly create backgrounds and maps.
Implementation
The implementation of the algorithm in the above paper began with the encoding of the image into a multi-qubit state compatible with the circuit. To do so, it maps each pixel to a bitstring such that adjacent pixels correspond to adjacent bitstrings that differ by one. This mapping creates some entanglement between adjacent pixels, which allows for the creation of a blurry image after applying gates and rotations.
Procedure
Each image is first decomposed into its red, green, and blue pixel values. These will each go through their own circuit.
```
def _image2heights(image):
Lx,Ly = image.size
heights = [{} for j in range(3)]
for x in range(Lx):
for y in range(Ly):
rgb = image.getpixel((x,y))
for j in range(3):
heights[j][x,y] = rgb[j]
return heights
```
These values, each between 0–255, are then normalized to be between 0–1. In the quantum state, each qubit represents a bitstring, which then represents a pixel. These bit strings are mapped such that each pixel adjacent to the other both horizontally and vertically differ by one value (either 1 or 0).
def make_line ( length ):
n = int(math.ceil(math.log(length)/math.log(2)))
# iteratively build list
line = ['0','1']
for j in range(n-1):
# first append a reverse-ordered version of the current list
line = line + line[::-1]
# then add a '0' onto the end of all bit strings in the first half
for j in range(int(float(len(line))/2)):
line[j] += '0'
# and a '1' for the second half
for j in range(int(float(len(line))/2),int(len(line))):
line[j] += '1'
return line
def make_grid(Lx,Ly=None):
# set Ly if not supplied
if not Ly:
Ly = Lx
# make the lines
line_x = make_line( Lx )
line_y = make_line( Ly )
# make the grid
grid = {}
for x in range(Lx):
for y in range(Ly):
grid[ line_x[x]+line_y[y] ] = (x,y)
# determine length of the bit strings
n = len(line_x[0]+line_y[0])
return grid, nEach qubit is first set to the height, the normalized 0–255 rgb value. Next, each qubit is put through a rotation. This circuit creates the actual blur and, after some experimentation, we decided that the x-rotation best maintains the key properties of an image, such as the edges and shapes.
def partial_x(qc,fraction):
for j in range(qc.num_qubits):
qc.rz(np.pi*fraction,j)Finally, the qubit values are measured. The results are then converted back to 0–255 values, which are concatenated together to create pixels. The final result is a blurred image.
Observations
We conducted multiple experiments on different circuits to create the actual blur including x, y, and z gates and a combination of them. Furthermore, we experimented with the actual rotational values. While x-rotation generally provided the more squirrel-like blurred images, we also found that y-rotations were quite similar to x-rotational values. In addition, we found that z-rotations generally did not change the image (as is expected because it is a rotation in the i-direction); However, we did notice that after performing a z-rotation, the images tended to become lighter, an attribute we believe could simply be due to rounding errors.
When using an actual noisy simulator to create the images, we noticed that generating images requires a lot of shots, as the image is created from a probability distribution. In addition, the increase of noise means the rotation isn’t as necessary for the final image since it is already noisy enough.
When testing the model with different images, we found that images with more dull colors like white, black, and grey generally could tolerate more noise and retain their features than images with all three colors. This once again makes sense since there is generally less room for error in chromatic-colored images.
Results
Below are some of the resulting noisy images. The example below specifically are variations of x-rotations on squirrel images.
a)
b)
c)
a)
b)
c)
a)
b)
c)
d)
Relevance to Course Material
This system uses Pauli rotation gates to filter the image data in different ways. We let the qubit strings undergo a circuit, which then produces the blur through rotations. In the end, we collapse the qubit through measurement and transform it back into an image. The rest of the process can be done through classical techniques like data preprocessing and image rendering.
Limitations
Some limitations of the model are mentioned in the paper. Primarily, this requires a lot of qubits and a lot of shots (execution of the circuit). Each pixel is encoded by a qubit, which causes the number of necessary qubits to increase proportionally with the size of the images. Realistically, we can only run blurs on small full images and instead need to treat the blur as a convolution kernel to run on large images. In addition, because there are so many qubits, this also requires a large number of shots. This is because the heights are converted to the rgb values, so a large number of shots is required in order to generate a probability space.
Section II: Quantum Convolutional Neural Networks
Background
TorchQuantum
TorchQuantum is a quantum simulation framework based on the library PyTorch, designed for running fast quantum machine learning models on GPUs. It was created here at MIT in the Han Lab. Some possible quantum circuits implementation on TorchQuantum include quantum regression, Clifford gate encoder, and quantum convolutional neural networks — the last of which we decided to implement.
One of the most popular alternatives to TorchQuantum is Pennylane, a library built for quantum machine learning. However, according to their GitHub, TorchQuantum differs from Pennylane and Qiskit through “dynamic computation graph, automatic gradient computation, fast GPU support, and batch model tensorized processing.”
Quantum Convolutions
Neural networks are widely used in machine learning for classical computing for purposes such as image analysis and recognition. Convolutional neural networks assign weightings to various aspects of the image taken as input to differentiate different images. The convolution operation often implements a Frobenius inner product of the convolution filter with the input image. Through training, such an algorithm can then learn these “filtering” characteristics to classify images on its own. Directly translating the classical algorithm to quantum, however, involves challenges where inputs can only take quantum processes, and such an attempt may require “exponentially difficult quantum state or process tomography” due to the large many-body Hilbert space4.
Thus a machine learning-inspired quantum circuit model applies the same concept but within a quantum context. Convolution filters can be replaced with circuits as “quanvolutional filters”5. These filters allow more complex operations to be performed in the Hilbert space than can be done in the Frobenius inner product, thus holding more opportunity to expand and take deep learning further than traditional convolution filters.
Implementation
To implement the algorithm, we used a notebook from TorchQuantum. We began by building the four-qubit convolution filter (a two-by-two filter) which performs a y-rotation on each qubit.
self.q_device = tq.QuantumDevice(n_wires=4)
self.encoder = tq.GeneralEncoder(
[ {'input_idx': [0], 'func': 'ry', 'wires': [0]},
{'input_idx': [1], 'func': 'ry', 'wires': [1]},
{'input_idx': [2], 'func': 'ry', 'wires': [2]},
{'input_idx': [3], 'func': 'ry', 'wires': [3]},]Each filter has a stride of two, meaning the resulting value is a 14 by 14 image. After the quantum convolution is performed, the result is the input of a standard linear neural network.
class HybridModel(torch.nn.Module):
def __init__(self):
super().__init__()
self.qf = QuanvolutionFilter()
self.linear = torch.nn.Linear(4*14*14, 10)
def forward(self, x, use_qiskit=False):
with torch.no_grad():
x = self.qf(x, use_qiskit)
x = self.linear(x)
return F.log_softmax(x, -1)This is compared to running the same analysis without the quantum filtering, and overall, the two networks asymptote to roughly the same accuracy.
Limitations
Because the quantum section acts as an actual convolution, modifying sections instead of the entire image, this is overall far more practical than the procedural generation approach.
