Exploring Quantum Randomized Benchmarking
By Catherine Tang, Wilson Zhu
Introduction
In our gate-based model of quantum devices, the quantum gates lie at the core of our ability to carry out the computation. Therefore, it is imperative to evaluate the accuracy of various quantum gates, as the accuracy dictates how well the desired operations can be carried out. When deciding on a metric to measure the accuracy of a quantum computer, we need a scalable and robust algorithm for measuring the performance of our quantum gates, without being affected by state preparation and state measurement errors. Randomized Benchmarking (RB), a protocol proposed by Magesan, Gambetta, and Emerson, addresses these needs for a scalable way to benchmark the performance of quantum gates.
Protocol
In the RB protocol, we randomly sample a length m sequence of gates from the Clifford group, known as Clifford gates (described more in the next section). Based on this sequence of m gates, we choose the next gate such that the result of applying the entire m+1 sequence of gates is the identity or returns the qubits back to their original ground state. Then we run this circuit on our quantum computer repeatedly, to determine the probability of the qubit returning back to the ground state. This probability, known as the survival probability, decreases as m, the length of the gate sequence, increases due to the fact that errors are introduced with each additional gate. By plotting the decay of the survival probability against m, we understand the fidelity of our quantum gates and observe the correlation between the decay rate and the average error probability per gate.
Given this general framework for RB, we explore the nuances and details of the steps of the protocol. In our work, we ran the randomized benchmarking protocol using tools created by Qiskit to gain a better understanding of how RB is experimentally carried out.
The Clifford Group
The Pauli Group operators on n-qubits is a combination of the 4 basic single-qubit Pauli operators (I, X, Y, Z), applied on n-qubits or tensored together, with a possible phase from the set of {+/-1, +/- i}.
The Clifford group is a set of gates that normalize operators from the Pauli group. In other words, given any Pauli operator P in the n-qubit Pauli group Pₙ, if CPC† is another operator in Pₙ, then C is in the n-qubit Clifford group. In addition, the Clifford group can be universally represented with {CNOT, H, S}.
Clifford gates are particularly efficient in calculating the m+1 gate in order to “reverse” or invert the behavior of the randomly generated m length sequence. The m+1 operator can in fact be another Clifford gate, as proven by the Gottesman-Knill Theorem. The theorem can be summarized as follows:
A quantum computer can be simulated on a classical computer given that the preparation of qubits and measurements are done on a computational basis and that the circuit is composed of Clifford gates.
It is known that such Clifford circuits can be simulated in O(n log n) time. Therefore, we can reasonably infer that calculating the m+1 gate can be calculated in polynomial time as well.
Mathematical Formalism
Sequence Fidelity
Let m be the length of our sequence of gates in our Randomized Benchmarking circuit. We generate Kₘ such RB circuits, with each circuit containing m randomly sampled Clifford gates. Each operator, C_i_j, is uniformly chosen from the Clifford group associated with a n-qubit system. (In this notation, C_i_j is the j-th element of the i-th sequence.) We then calculate an additional Clifford element at the end of each sequence according to the following:
We define the average error operator
and Λ_{i_j, j} to be the time-dependent, gate-dependent error operator.
A sequence operation is defined as the sequence of gates to be applied in a circuit, taking into account the error. Then the sequence operation, S, for a single circuit is defined as
where
The average sequence operation for circuits of length m is
Having defined the average sequence operation, we can then find the average circuit fidelity for a Clifford circuit of length m with the following equation:
where S_Kₘ(ρ_ψ) is the sequence operation acting upon the initial state, ρ_ψ, that takes into account state preparation error. The additional E_ψ term accounts for errors associated with measurement. The circuit fidelity, Fₛₑᵩ, is dependent on the length of the circuit m.
Exponential Decay Model
In order to understand the measurements taken from Randomized Benchmarking, we plot the ground state population against the number of Clifford gates. From the average circuit fidelity equation
we can derive the equation to fit the plotted data. Assuming the error operator Λ_{i_j, j} = Λ is both time- and gate-independent and let
then we have
where
Hence we can model the ground state population versus the number of Clifford gates with
Error Per Clifford (EPC)
To find the amount of error present in each Clifford gate, or Error Per Clifford (EPC), we start with the average fidelity of Λ, given by
Then we can find the average error-rate r, where
and since d=2ⁿ we can simplify and get
Gate Fidelity Observations
Predicting Circuit Fidelity
Using our experimentally derived values, we can reasonably predict the fidelity of other quantum circuits that follows a similar design. We obtain the error per gate for u₁, u₂, u₃, and cₓ experimentally. Then for the new circuit, we find the number of gates per Clifford (e.g. u₁ per Clifford) which allow us to predict the EPC for the new circuit.
Algorithmic vs. Primitive Gate Fidelity
In seeking to characterize the gate fidelity of a multi-qubit system, we find that it is important to test the fidelity beyond just that of the primitive gates to take into account factors like crosstalk and addressability errors. The challenge of accurately quantifying the fidelity of a multi-qubit system can be summarized by the following results of an experiment. For a quantum computer with measured 2-qubit gate fidelities of 0.99, the state fidelity of applying a 5-qubit GHZ circuit of four 2-qubit operations was 0.82. Therefore, to predict algorithmic fidelity, we need to evaluate the performance of gates on multi-qubit systems as a whole. This is where randomized benchmarking has an advantage in that it provides a scalable way for us to measure multi-qubit fidelities while accounting for state preparation and measurement errors. The underlying connectivity of a quantum computer can greatly influence the reported EPC value. The connectivity here indicates which qubits in your quantum computer can interact with other qubits via gate operations (all-to-all vs. linear gate connectivity).
Randomized Benchmarking
Benchmarking Setup
When treating multi-qubit systems, we can benchmark the circuit by addressing the various subsystems separately. There are 8 different combinations we can consider when performing randomized benchmarking on 3 qubit systems
- Simultaneously benchmarking each qubit independently
- Benchmarking just two of the three qubits
- Simultaneously benchmarking 2 qubits and 1 qubit
- Benchmarking all 3 qubits together
Depending on the size, nᵢ, of the subsystem we choose, we sample from the corresponding nᵢ-Clifford group to create our sequence of Clifford gates.
Running the Experiment
To set up the experiment on Qiskit, we utilize the random benchmarking tools provided by the qiskit.ignis module. To get started, we define parameters related to our random benchmarking protocol as follows:
# Generate RB circuits (3Q RB)
# number of qubits
nQ = 3
rb_opts = {}
#Number of Cliffords in the sequence
rb_opts['length_vector'] = [1, 10, 20, 50, 75, 100, 125, 150, 175, 200]
# Number of seeds (random sequences)
rb_opts['nseeds'] = 5
# Default pattern
rb_opts['rb_pattern'] = [[0,1,2]]
rb_circs, xdata = rb.randomized_benchmarking_seq(**rb_opts)Here rb_opts is a dictionary storing all the parameters that define our random benchmarking protocol. The ‘length_vector’ parameter corresponds to the different length sequences we seek to benchmark. These are our various m values. In addition, to generate Kₘ sequences for each m length value, we need to have a different random seed for each sequence. So in this case, the ‘nseeds’ parameter specifying the number of random seeds is Kₘ, the number of random sequences. Finally, we also need to specify the way in which we are benchmarking our multi-qubit system. The ‘rb_pattern’ parameter accomplishes this in describing the ways we are grouping the qubits for RB (ie. we are benchmarking all 3 qubits together and treating them as one system in the above code snippet).
Taking a look at our resulting graph from running the protocol:
We note that there are 10 distinct Clifford Lengths along the x-axis, corresponding to the length_vector we specified. For each length (m), we have 5 x-markers or data points grouped together, for each of the 5 different circuits we randomly generated using 5 different random seeds. The y-axis here represents the ground state population which is the percentage of runs where the system was returned back to the ground state for the 200 shots we ran each circuit. The red dotted line connects the mean of each group of points in a line graph. The blue curve is fitted using the exponential decay model derived from the section above.
Simultaneous Random Benchmarking
When we seek to measure the gate fidelity of multi-qubit systems, we can benchmark subsystems independently, as well as the entire system simultaneously. We can intuitively understand that when benchmarking the entire system simultaneously, we will notice higher error rates compared with separately benchmarking subsystems. This is due to the fact that benchmarking individual subsystems fails to account for errors due to classical crosstalk and quantum information leakage. These fall under the category of addressability errors. Classical crosstalk is a classical error where attempts at controlling and manipulating the state of one subsystem accidentally affect the state of the other subsystem too. Quantum information leakage involves unwanted or unintentional quantum entanglement and interactions between the two subsystems. A simple experiment to characterize the addressability of our quantum system involves the following:
- Performing RB on each subsystem separately
- Performing RB on two subsystems simultaneously
Taking the difference between the error rates gives us an indication of how the error in one subsystem can affect that of the other subsystem.
In our own experiments, we witness a similar effect taking place.
Experiment Results
Results
For our 3 qubit subsystem, we measure the EPC in 3 separate instances as follows:
Analysis
If we compare results from Experiment 3 and Experiment 4, the difference between these two experiments is that we benchmarked an additional 1 qubit simultaneously alongside the two-qubit subsystem in Experiment 4. We notice that we do have a slightly lower alpha which is correlated with average gate fidelity in Experiment 4. This can be explained by the addressability of the system.
In addition, if we look at Experiment 2 where we benchmarked all 3 qubits simultaneously as single qubit systems, we have a very high alpha. This is due to the fact that we don’t have any CNOT gates in our circuit as we are only using single-qubit gates.
Finally, in Experiment 5, we have the lowest average gate fidelity as we benchmark the entire system using 3-qubit Clifford gates. We can explain this significantly lower alpha due to the fact that all the errors that arise between qubits are accounted for as we have 3 qubits interacting in our system. In addition, the connectivity of the circuit now plays a bigger role in affecting the performance of the circuit as we are performing CNOTs between all 3 pairwise combinations of qubits.
Conclusion and Related Works
In our research, we explored the random benchmarking protocol as well as different configurations for random benchmarking multi-qubit systems. We implemented and ran some of these experiments on the Qiskit quantum computer simulator using a noise model to evaluate the fidelity of our gates. Beyond the simple cases of random benchmarking we considered, there are various shortcomings for quantum randomized benchmarking, and improving this protocol is still an active area of research.
The most notable shortcoming is the limitation imposed by only using Clifford gates in the circuit. These gates are not fully universal as only with the addition of the Toffoli gate do we get a universal set of quantum gates. Without the Toffoli, all quantum operations can still be efficiently simulated on a classical computer. This means that the benchmarking results cannot be applied universally as a universal quantum computer includes additional gates. As a result, other methods for predicting circuit fidelity have been proposed. For example, there has been research exploring how to use Neural Networks to predict circuit fidelity for quantum computers containing non-Clifford gates.
Another area of research involves how to randomly sample Clifford gates for our RB protocol to optimize the configuration of our RB to reduce the confidence interval of the estimated fidelity while staying within a time budget for our circuit.
In our work, we explored the various methods by which quantum circuit fidelity can be benchmarked. Naturally, we ask how circuit fidelity can be improved. One method, Quest, aims to reduce the number of CNOT gates in a given quantum circuit in order to improve circuit fidelity. We know that circuit fidelity is inversely proportional to the number of Clifford gates as demonstrated through our experiment. Using approximate synthesis, Quest is able to reduce the number of CNOT gates by 30–80%, which in turn increases circuit fidelity and reduces noise.
As quantum computing matures and larger multi-qubit systems are developed, the need for a more robust benchmarking protocol for circuit fidelity increases and we see randomized benchmarking potentially addressing these needs.
References
Avi Vadali, Rutuja Kshirsagar, Prasanth Shyamsundar, and Gabriel N. Perdue. Quantum circuit fidelity estimation using machine learning, 2022.
“Randomized Benchmarking.” Qiskit.org, Data 100 at UC Berkeley, 29 Nov. 2022, https://qiskit.org/textbook/ch-quantum-hardware/randomized-benchmarking.html.
Easwar Magesan, J. M. Gambetta, and Joseph Emerson, Robust randomized benchmarking of quantum processes, https://arxiv.org/pdf/1009.3639
Easwar Magesan, Jay M. Gambetta, and Joseph Emerson, Characterizing Quantum Gates via Randomized Benchmarking, https://arxiv.org/pdf/1109.6887
A. D. C’orcoles, Jay M. Gambetta, Jerry M. Chow, John A. Smolin, Matthew Ware, J. D. Strand, B. L. T. Plourde, and M. Steffen, Process verification of two-qubit quantum gates by randomized benchmarking, https://arxiv.org/pdf/1210.7011
Jay M. Gambetta, A. D. C´orcoles, S. T. Merkel, B. R. Johnson, John A. Smolin, Jerry M. Chow, Colm A. Ryan, Chad Rigetti, S. Poletto, Thomas A. Ohki, Mark B. Ketchen, and M. Steffen, Characterization of addressability by simultaneous randomized benchmarking, https://arxiv.org/pdf/1204.6308
David C. McKay, Sarah Sheldon, John A. Smolin, Jerry M. Chow, and Jay M. Gambetta, Three Qubit Randomized Benchmarking, https://arxiv.org/pdf/1712.06550
Toshinari Itoko and Rudy Raymond. Sampling strategy optimization for randomized benchmark- ing. In 2021 IEEE International Conference on Quantum Computing and Engineering (QCE), pages 188–198, 2021.
Daniel Gottesman. The Heisenberg Representation of Quantum Computers. 1998., https://arxiv.org/abs/quant-ph/9807006
