Skip to content

Technical Benchmarking

Technical Benchmarking refers to tests measuring fidelities, success probabilities, or other noise measures for specific sets of gates and qubits. The Classiq Package supports, for example, randomized benchmarking, a test measuring the average error per Clifford gate on a specific, usually narrow, set of qubits.

Randomized Benchmarking

Randomized Benchmarking (RB) is a test meant to measure the average Clifford error\ fidelity on a specific set of qubits. The test is performed by applying a series of random Clifford gates, then their inverse, which is another Clifford gate, precomputed in advance. It is similar to the Mirror Benchmarking test, the difference being the inverted part, which can be thought of as highly optimized. Several analytical results allow the fitting of RB experiment results to extract the fidelity. The more mathematically inclined reader is directed to the next subsection for a summary. Following it is a usage example of RB on the Classiq platform.

Theory of Randomized Benchmarking

The Clifford group forms a 2-design, namely, a set on which all degree 2 polynomial integrals may be evaluated by a discrete sum on the set. Random samples from the Clifford group approximately have the same property [1]. In a classical paper by Nielsen [2], it is shown that when averaging over all unitary gates according to the Haar measure, the average noise channel is a depolarizing channel. This process is commonly referred to as "twirling". Direct calculation of the survival probability (the chance not to be "depolarized"), yields an exponentially decreasing success probability, with the rate given by the average Clifford fidelity \(f\)

\[ p = A f^m + B \]

Here \(m\) the number of Clifford gates, \(A\) and \(B\) are constants that depend on state preparation and measurement (SPAM) errors. This is the basic RB scheme, it may be extended. See, for example [3].


The following code presents an example of a random benchmarking test of 2-qubit Clifford fidelity comparing IBM's "Santiago" device and an AER simulator (which is noiseless, hence the trivial result). Below is a figure containing the obtained results. Note that to run the example ACCESS_TOKEN must be assigned a valid value.

from typing import Dict

from classiq.interface.backend.backend_preferences import IBMBackendPreferences
from classiq.interface.generator.function_param_list import RandomizedBenchmarking
from classiq.interface.generator.model.preferences.preferences import Preferences

from classiq import ModelDesigner
from classiq.analyzer.rb import RBAnalysis, order_executor_data_by_hardware
from classiq.executor import ExecutionPreferences, batch_execute_multiple_backends

num_of_qubits = 2
numbers_of_cliffords = [5, 10, 15, 20, 25]

params_list = [
        num_of_qubits=num_of_qubits, num_of_cliffords=num_of_cliffords
    for num_of_cliffords in numbers_of_cliffords

preferences = Preferences(
models = [ModelDesigner(preferences=preferences) for _ in numbers_of_cliffords]

for model_designer, params in zip(models, params_list):

circuits = [model_designer.synthesize() for model_designer in models]
programs = [circ.to_program() for circ in circuits]

backends_to_execute_on = IBMBackendPreferences.batch_preferences(
    backend_names=("ibmq_santiago", "aer_simulator"),

multiple_results = batch_execute_multiple_backends(

clifford_number_mapping: Dict[str, int] = {
    prog.code: num_clifford
    for prog, num_clifford in zip(programs, numbers_of_cliffords)
rb_analysis_params = order_executor_data_by_hardware(
    mixed_data=multiple_results, clifford_numbers_per_program=clifford_number_mapping
multiple_hardware_data = RBAnalysis(experiments_data=rb_analysis_params)
total_data = multiple_hardware_data.show_multiple_hardware_data()
fig = multiple_hardware_data.plot_multiple_hardware_results()



[1] C. Dankert, R. Cleve, J. Emerson, and E. Livine, “Exact and Approximate Unitary 2-Designs and their Application to Fidelity Estimation”. Available:

[2] M. A. Nielsen, "A simple formula for the average fidelity of a quantum dynamical operation". Available:

[3] J. Helsen, X. Xue, L. M. L Vandersypen, S. Wehner "A new class of efficient randomized benchmarking protocols". Available: