> ## Documentation Index
> Fetch the complete documentation index at: https://docs.classiq.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Quantum Kernels and Support Vector Machines

<Card title="View on GitHub" icon="github" href="https://github.com/Classiq/classiq-library/blob/main/applications/finance/credit_card_fraud/credit_card_fraud.ipynb">
  Open this notebook in GitHub to run it yourself
</Card>

## Detecting Credit Card Fraud

Quantum Support Vector Machines (QSVM) on Kaggle labeled data is a means to classify and detect fraudulent credit card transactions.

Quantum Machine Learning (QML) is the aspect of research that explores the consequences of implementing machine learning on a quantum computer.

SVM is a supervised machine learning method widely used for multiple labeled data classification.

The SVM algorithm can be enhanced even on a noisy intermediate scale quantum computer (NISQ) by introducing the kernel method. It can be restructured to exploit the properties of the large dimensionality of a quantum Hilbert space.

This demo presents a simple use case where a Quantum SVM (QSVM) algorithm is implemented on credit card labeled data to detect fraudulent transactions. It leverages the Classiq proprietary QSVM library and core capabilities to explore the rising potential in enhancing security applications.

This demonstration is based on work published in August 2022 \[[1](#hbc)].

*This demo uses the `sklearn` package in addition to the `classiq` package.*

```python theme={null}
!pip install -qq -U "classiq[qml]"
```

```python theme={null}

%%capture
! pip install scikit-learn
! pip install seaborn
```

Import the required resources:

```python theme={null}
# General Imports
# Visualization Imports
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# Scikit Imports
import sklearn
from sklearn import datasets
from sklearn.decomposition import PCA
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, StandardScaler
from sklearn.svm import SVC
from sklearn.utils import shuffle
```

```python theme={null}

## For TSNE visualization
from numpy import linalg
from numpy.linalg import norm
from scipy.spatial.distance import pdist, squareform
from sklearn.manifold import TSNE

# Hack the t-SNE code in sklearn 0.15.2
from sklearn.metrics.pairwise import pairwise_distances
from sklearn.preprocessing import scale

# Random state
RS = 20150101

import matplotlib
import matplotlib.colors as colors

# Use Matplotlib for graphics
import matplotlib.patheffects as PathEffects
import matplotlib.pyplot as plt

# Import Seaborn to make nice plots
try:
    import seaborn as sns
except ModuleNotFoundError:
    palette = np.array(
        [
            (0.4, 0.7607843137254902, 0.6470588235294118),
            (0.9882352941176471, 0.5529411764705883, 0.3843137254901961),
            (0.5529411764705883, 0.6274509803921569, 0.796078431372549),
            (0.9058823529411765, 0.5411764705882353, 0.7647058823529411),
            (0.6509803921568628, 0.8470588235294118, 0.32941176470588235),
            (1.0, 0.8509803921568627, 0.1843137254901961),
            (0.8980392156862745, 0.7686274509803922, 0.5803921568627451),
            (0.7019607843137254, 0.7019607843137254, 0.7019607843137254),
        ]
    )
else:
    sns.set_style("darkgrid")
    sns.set_palette("muted")
    sns.set_context("notebook", font_scale=1.5, rc={"lines.linewidth": 2.5})
    palette = np.array(sns.color_palette("Set2"))

import warnings

warnings.filterwarnings("ignore")
```

```python theme={null}

from classiq import *
```

## Data

The dataset contains transactions made by credit cards in September 2013 by European cardholders.

The transactions occurred over two days, where there were 492 frauds out of 284,807 transactions.

The dataset is highly unbalanced, such that the positive class (frauds) account for 0.172% of all transactions.

Data properties:

* The database contains only numeric input variables that are the result of a PCA transformation.
* Due to confidentiality issues, original features are not provided.
* Features V1, V2, … V28 are the principal components obtained with PCA.
* The only features that have not been transformed with PCA are 'Time' and 'Amount'.
* The 'Time' feature is the number of seconds that elapsed between each transaction and the first transaction in the dataset.
* The 'Amount' feature is the transaction amount.
* The 'Class' feature is the response variable. It takes the value of 1 in the case of fraud and 0 otherwise.

The data is freely available through Kaggle \[[2](#kaggle)].

#

## Loading the Kaggle "Credit Card Fraud Detection" Dataset

```python theme={null}
input_file = "../resources/creditcard.csv"
# comma delimited file as input
kaggle_full_set = pd.read_csv(input_file, header=0)
# presnting first 5 lines:
kaggle_full_set.head()
```

<div><style scoped> .dataframe tbody tr th:only-of-type \{ vertical-align: middle; } .dataframe tbody tr th \{ vertical-align: top; } .dataframe thead th \{ text-align: right; } </style> <table border="1" className="dataframe"> <thead> <tr style={{textAlign: "right"}}> <th /> <th>Time</th> <th>V1</th> <th>V2</th> <th>V3</th> <th>V4</th> <th>V5</th> <th>V6</th> <th>V7</th> <th>V8</th> <th>V9</th> <th>...</th> <th>V21</th> <th>V22</th> <th>V23</th> <th>V24</th> <th>V25</th> <th>V26</th> <th>V27</th> <th>V28</th> <th>Amount</th> <th>Class</th> </tr> </thead> <tbody> <tr> <th>0</th> <td>0</td> <td>-1.359807</td> <td>-0.072781</td> <td>2.536347</td> <td>1.378155</td> <td>-0.338321</td> <td>0.462388</td> <td>0.239599</td> <td>0.098698</td> <td>0.363787</td> <td>...</td> <td>-0.018307</td> <td>0.277838</td> <td>-0.110474</td> <td>0.066928</td> <td>0.128539</td> <td>-0.189115</td> <td>0.133558</td> <td>-0.021053</td> <td>149.62</td> <td>0</td> </tr> <tr> <th>1</th> <td>0</td> <td>1.191857</td> <td>0.266151</td> <td>0.166480</td> <td>0.448154</td> <td>0.060018</td> <td>-0.082361</td> <td>-0.078803</td> <td>0.085102</td> <td>-0.255425</td> <td>...</td> <td>-0.225775</td> <td>-0.638672</td> <td>0.101288</td> <td>-0.339846</td> <td>0.167170</td> <td>0.125895</td> <td>-0.008983</td> <td>0.014724</td> <td>2.69</td> <td>0</td> </tr> <tr> <th>2</th> <td>1</td> <td>-1.358354</td> <td>-1.340163</td> <td>1.773209</td> <td>0.379780</td> <td>-0.503198</td> <td>1.800499</td> <td>0.791461</td> <td>0.247676</td> <td>-1.514654</td> <td>...</td> <td>0.247998</td> <td>0.771679</td> <td>0.909412</td> <td>-0.689281</td> <td>-0.327642</td> <td>-0.139097</td> <td>-0.055353</td> <td>-0.059752</td> <td>378.66</td> <td>0</td> </tr> <tr> <th>3</th> <td>1</td> <td>-0.966272</td> <td>-0.185226</td> <td>1.792993</td> <td>-0.863291</td> <td>-0.010309</td> <td>1.247203</td> <td>0.237609</td> <td>0.377436</td> <td>-1.387024</td> <td>...</td> <td>-0.108300</td> <td>0.005274</td> <td>-0.190321</td> <td>-1.175575</td> <td>0.647376</td> <td>-0.221929</td> <td>0.062723</td> <td>0.061458</td> <td>123.50</td> <td>0</td> </tr> <tr> <th>4</th> <td>2</td> <td>-1.158233</td> <td>0.877737</td> <td>1.548718</td> <td>0.403034</td> <td>-0.407193</td> <td>0.095921</td> <td>0.592941</td> <td>-0.270533</td> <td>0.817739</td> <td>...</td> <td>-0.009431</td> <td>0.798278</td> <td>-0.137458</td> <td>0.141267</td> <td>-0.206010</td> <td>0.502292</td> <td>0.219422</td> <td>0.215153</td> <td>69.99</td> <td>0</td> </tr> </tbody> </table> <p>5 rows × 31 columns</p></div>

##

1. Data Preprocessing: Selecting Train and Test Datasets

Subsample the dataset to make it manageable for near-term quantum simulations:

```python theme={null}
TRAIN_NOMINAL_SIZE = 100
TRAIN_FRAUD_SIZE = 25

TEST_NOMINAL_SIZE = 50
TEST_FRAUD_SIZE = 10

PREDICTION_NOMINAL_SIZE = 50
PREDICTION_FRAUD_SIZE = 10

SHUFFLE_DATA = False
```

```python theme={null}

## Separating nominal ("legit") from fraud:
all_fraud_set = kaggle_full_set.loc[kaggle_full_set["Class"] == 1]
all_nominal_set = kaggle_full_set.loc[kaggle_full_set["Class"] == 0]
```

```python theme={null}

## Optionally shuffle data before selective sets
if SHUFFLE_DATA:
    all_fraud_set = shuffle(all_fraud_set, random_state=1234)
    all_nominal_set = shuffle(all_nominal_set, random_state=1234)
```

```python theme={null}

## Selecting data subsets
selected_training_set = pd.concat(
    [all_nominal_set[:TRAIN_NOMINAL_SIZE], all_fraud_set[:TRAIN_FRAUD_SIZE]]
)
selected_testing_set = pd.concat(
    [
        all_nominal_set[TRAIN_NOMINAL_SIZE : TRAIN_NOMINAL_SIZE + TEST_NOMINAL_SIZE],
        all_fraud_set[TRAIN_FRAUD_SIZE : TRAIN_FRAUD_SIZE + TEST_FRAUD_SIZE],
    ]
)
selected_prediction_set = pd.concat(
    [
        all_nominal_set[
            TRAIN_NOMINAL_SIZE
            + TEST_NOMINAL_SIZE : TRAIN_NOMINAL_SIZE
            + TEST_NOMINAL_SIZE
            + PREDICTION_NOMINAL_SIZE
        ],
        all_fraud_set[
            TRAIN_FRAUD_SIZE
            + TEST_FRAUD_SIZE : TRAIN_FRAUD_SIZE
            + TEST_FRAUD_SIZE
            + PREDICTION_FRAUD_SIZE
        ],
    ]
)
```

```python theme={null}

## Separating relevant features data (excluding the "Time" column) from label data

kaggle_headers = list(kaggle_full_set.columns.values)  # all headers
feature_cols = kaggle_headers[1:-1]  # excluding Time and Class headers
label_col = kaggle_headers[-1]  # marking Class header as label

selected_training_data = selected_training_set.loc[:, feature_cols]
selected_training_labels = selected_training_set.loc[:, label_col]

selected_testing_data = selected_testing_set.loc[:, feature_cols]
selected_testing_labels = selected_testing_set.loc[:, label_col]

selected_prediction_data = selected_prediction_set.loc[:, feature_cols]
selected_prediction_true_labels = selected_prediction_set.loc[:, label_col]
```

#

## Visualizing the Selected Datasets with t-SNE

t-SNE is a technique for dimensionality reduction that is particularly suited for the visualization of high-dimensional datasets:

```python theme={null}
def scatter(x, colors):
    # Create a scatter plot
    f = plt.figure(figsize=(8, 8))
    ax = plt.subplot(aspect="equal")
    sc = ax.scatter(x[:, 0], x[:, 1], lw=0, s=40, c=palette[colors.astype(np.int32)])
    plt.xlim(-25, 25)
    plt.ylim(-25, 25)
    ax.axis("off")
    ax.axis("tight")

    # Add the labels for each digit
    txts = []
    labels = ["Nominal", "Fraud"]
    for i in range(2):
        # Position of each label.
        xtext, ytext = np.median(x[colors == i, :], axis=0)
        txt = ax.text(xtext, ytext, labels[i], fontsize=24)
        txt.set_path_effects(
            [PathEffects.Stroke(linewidth=5, foreground="w"), PathEffects.Normal()]
        )
        txts.append(txt)

    return
```

#

### TSNE Visualization of Train Data

Observe that visually t-SNE shows a separation between nominal and anomalous samples.

**However, the sole visualization map does not allow tracking of all fraudulent transactions.**

This demonstrates the challenge of high-quality fraud detection.

For the sake of a quick demonstration, take only a very small percentage of the data.

Applying better logic for subselecting the training and testing datasets affects the quality of the results:

```python theme={null}
proj = TSNE(random_state=RS).fit_transform(selected_training_data)
scatter(proj, selected_training_labels)
```

<img src="https://mintcdn.com/classiq/6w5HDe0V8PXk6Zud/explore/applications/finance/credit_card_fraud/credit_card_fraud_files/credit_card_fraud_1.png?fit=max&auto=format&n=6w5HDe0V8PXk6Zud&q=85&s=253511fabbb5bf7be7a5090181af8100" alt="output" width="640" height="636" data-path="explore/applications/finance/credit_card_fraud/credit_card_fraud_files/credit_card_fraud_1.png" />

#

### TSNE Visualization of Test Data

```python theme={null}
proj = TSNE(random_state=RS).fit_transform(selected_testing_data)
scatter(proj, selected_testing_labels)
```

<img src="https://mintcdn.com/classiq/6w5HDe0V8PXk6Zud/explore/applications/finance/credit_card_fraud/credit_card_fraud_files/credit_card_fraud_2.png?fit=max&auto=format&n=6w5HDe0V8PXk6Zud&q=85&s=e6ba16ba5c9b82c75d351f5a8085a17f" alt="output" width="684" height="636" data-path="explore/applications/finance/credit_card_fraud/credit_card_fraud_files/credit_card_fraud_2.png" />

#

## Reducing Dimensions

Convert original features into fewer features to match the number of qubits.

Perform dimensionality reduction to match the number of features with the number of qubits used in simulation. To do this, use principal component analysis and keep only the first *N\_DIM* principal components:

```python theme={null}

## Choose a data dimension to encode
N_DIM = 3
```

```python theme={null}

sample_train = selected_training_data.values.tolist()
sample_test = selected_testing_data.values.tolist()
sample_predict = selected_prediction_data.values.tolist()

# Reduce dimensions

pca = PCA(n_components=N_DIM).fit(sample_train)
sample_train = pca.transform(sample_train)
sample_test = pca.transform(sample_test)
sample_predict = pca.transform(sample_predict)
```

#

## Normalizing

Use feature-wise standard scaling, i.e., subtract the mean and scale by the standard deviation for each feature:

```python theme={null}
# Normalize
std_scale = StandardScaler().fit(sample_train)
sample_train = std_scale.transform(sample_train)
sample_test = std_scale.transform(sample_test)
sample_predict = std_scale.transform(sample_predict)
```

#

## Scaling

Scale each feature to a range between -$\pi$ and $\pi$:

```python theme={null}
# Scale
samples = np.append(sample_train, sample_test, axis=0)
samples = np.append(samples, sample_predict, axis=0)
minmax_scale = MinMaxScaler((-np.pi, np.pi)).fit(samples)
FRAUD_TRAIN_DATA = minmax_scale.transform(sample_train)
FRAUD_TEST_DATA = minmax_scale.transform(sample_test)
FRAUD_PREDICT_DATA = minmax_scale.transform(sample_predict)
```

This is the final preprocessed dataset:

```python theme={null}
FRAUD_TRAIN_LABELS = np.array(selected_training_labels.values.tolist())
FRAUD_TEST_LABELS = np.array(selected_testing_labels.values.tolist())
```

##

2. Map the Data to a Hilbert Space

The feature map is a parameterized quantum circuit, which can be described as a unitary transformation $\mathbf{U_\phi}(\mathbf{x})$ on n qubits.

Since the data may be non-linearly separable in the original space, the feature map circuit maps the classical data into the Hilbert space.

The choice of which feature map circuit to use is key and may depend on the given dataset to classify.

You can leverage the Classiq feature map design capabilities.

#

## Designing a Feature Map

As an example, choose from the well known second-order Pauli-Z evolution encoding circuit with two repetitions or the bloch sphere circuit encoding.

Their definitions are in the [tutorial](https://github.com/Classiq/classiq-library/blob/main/algorithms/QML/qsvm/qsvm.ipynb).

Pauli feature map:

```python theme={null}
from classiq.applications.qsvm.quantum_feature_maps import pauli_feature_map

PAULIS = [[Pauli.Z], [Pauli.Z, Pauli.Z]]
CONNECTIVITY = 2  # Full
AFFINES = [[1, 0], [1, np.pi]]
REPS = 2

build_pauli_feature_map = lambda data, qba: pauli_feature_map(
    data, PAULIS, AFFINES, CONNECTIVITY, REPS, qba
)
```

Construct the quantum model for the QSVM routine:

```python theme={null}
from classiq.applications.qsvm.qsvm import QSVM

fraud_qsvm = QSVM(feature_map=build_pauli_feature_map, num_qubits=N_DIM)
```

#

## Viewing the Generated Quantum Circuit

Before training, the quantum circuit used for kernel evaluation is accessible via `fraud_qsvm.get_qprog()` and can be viewed with `show`.

For the Pauli feature map, the data dimension is the same as the number of qubits.

```python theme={null}
qprog = fraud_qsvm.get_qprog(data_dim=N_DIM)
show(qprog)
```

<Info>
  **Output:**

  ```

  Quantum program link: https://platform.classiq.io/circuit/3DywSlw95HA0z4qQJZbBRFifcro
    

  ```
</Info>

##

3. Execute QSVM

Quantum Support Vector Machines is the quantum version of SVM: a data classification method that separates the data using a hyperplane.

The algorithm performs these steps \[[3](#3)]:

1. **Estimates** the kernel matrix:

* A quantum feature map, $\phi(\mathbf{x})$, naturally gives rise to a quantum kernel, $k(\mathbf{x}_i,\mathbf{x}_j)= \phi(\mathbf{x}_j)^\dagger\phi(\mathbf{x}_i)$, which can be seen as a measure of similarity: $k(\mathbf{x}_i,\mathbf{x}_j)$ is large when $\mathbf{x}_i$ and $\mathbf{x}_j$ are close.

* When considering finite data, you can represent the quantum kernel as a matrix:
  $K_{ij} = \left| \langle \phi^\dagger(\mathbf{x}_j)| \phi(\mathbf{x}_i) \rangle \right|^{2}$.

Calculate each element of this kernel matrix on a quantum computer by calculating the transition amplitude:

$        \left| \langle \phi^\dagger(\mathbf\{x\}_j)| \phi(\mathbf\{x\}_i) \rangle \right|^\{2\} =
        \left| \langle 0^\{\otimes n\} | \mathbf\{U_\phi^\dagger\}(\mathbf\{x\}_j) \mathbf\{U_\phi\}(\mathbf\{x_i\}) | 0^\{\otimes n\} \rangle \right|^\{2\}$

This provides an estimate of the quantum kernel matrix, which will be used in the support vector classification.

2. **Optimizes** the dual problem using the classical SVM algorithm to generate a separating hyperplane and classify the data:

$$
L_D(\alpha) = \sum_{i=1}^t \alpha_i - \frac{1}{2} \sum_{i,j=1}^t y_i y_j \alpha_i \alpha_j K(\vec{x}_i \vec{x}_j)
$$

where

* $t$ is the number of data points
* $\vec{x}_i$s are the data points
* $y_i$ is the label $\in \{-1,1\}$ of each data point
* $K(\vec{x}_i \vec{x}_j)$ is the kernel matrix element between the $i$ and $j$ data points
* Optimizes over the $\alpha$s

Expect most of the $\alpha$s to be $0$.

The $\vec{x}_i$s that correspond to non-zero $\alpha_i$ are called the support vectors.

#

## Running QSVM and Analyzing the Results

#

## Training and Testing the Data

1. Build the train and test quantum kernel matrices:

2. For each pair of datapoints in the training dataset $\mathbf{x}_{i},\mathbf{x}_j$, apply the feature map and measure the transition probability: $K_{ij} = \left| \langle 0 | \mathbf{U}^\dagger_{\Phi(\mathbf{x_j})} \mathbf{U}_{\Phi(\mathbf{x_i})} | 0 \rangle \right|^2$.

3. For each training datapoint $\mathbf{x_i}$ and testing point $\mathbf{y_i}$, apply the feature map and measure the transition probability: $K_{ij} = \left| \langle 0 | \mathbf{U}^\dagger_{\Phi(\mathbf{y_i})} \mathbf{U}_{\Phi(\mathbf{x_i})} | 0 \rangle \right|^2$.

4. Use the train and test quantum kernel matrices in a classical support vector machine classification algorithm.

Execute QSVM by calling `train`, `test`, and `predict` on the `fraud_qsvm` object:

```python theme={null}
fraud_qsvm.train(FRAUD_TRAIN_DATA, FRAUD_TRAIN_LABELS)
test_score, y_test = fraud_qsvm.test(FRAUD_TEST_DATA, FRAUD_TEST_LABELS)
predicted_labels = fraud_qsvm.predict(FRAUD_PREDICT_DATA)
```

```python theme={null}

print("quantum kernel classification test score:  %0.2f" % (test_score))
```

<Info>
  **Output:**

  ```
  quantum kernel classification test score:  0.92
    

  ```
</Info>

The result seems OK.

This is a good start!

#

## Analyzing

Now analyze further by comparing the testing accuracy results to classical kernels:

```python theme={null}
classical_kernels = ["linear", "poly", "rbf", "sigmoid"]

for ckernel in classical_kernels:
    classical_svc = SVC(kernel=ckernel)
    classical_svc.fit(FRAUD_TRAIN_DATA, FRAUD_TRAIN_LABELS)
    classical_score = classical_svc.score(
        FRAUD_TEST_DATA, np.array(FRAUD_TEST_LABELS.tolist())
    )

    print("%s kernel classification test score:  %0.2f" % (ckernel, classical_score))
```

<Info>
  **Output:**

  ```
  linear kernel classification test score:  1.00
    poly kernel classification test score:  1.00
    rbf kernel classification test score:  1.00
    sigmoid kernel classification test score:  0.98
    

  ```
</Info>

Given the simple naive training and testing set example, the statement that **quantum kernel gets results at least as good as the classical kernels** can be interpreted as a promising sign and encouragement towards deeper research.

Bear in mind:

* Quantum kernel machine algorithms only have the potential of quantum advantage over classical approaches if the corresponding quantum kernel is hard to estimate classically (a necessary and not always sufficient condition to obtain a quantum advantage).

* However, it was recently proven \[[4](#4)] that learning problems exist for which learners with access to quantum kernel methods have a quantum advantage over all classical learners.

#

## Predicting Data

Finally, predict unlabeled data by calculating the kernel matrix of the new datum with respect to the support vectors:

$$
\text{Predicted Label}(\vec{s}) = \text{sign} \left( \sum_{i=1}^t y_i \alpha_i^* K(\vec{x}_i , \vec{s}) + b \right)
$$

where

* $\vec{s}$ is the datapoint to classify
* $\alpha_i^*$ are the optimized $\alpha$s
* $b$ is the bias

```python theme={null}
true_labels = np.array(selected_prediction_true_labels.values.tolist())
sklearn.metrics.accuracy_score(predicted_labels, true_labels)
```

<Info>
  **Output:**

  ```
  0.9166666666666666
    

  ```
</Info>

## Quantum Advantage Is Possible

QSVM has the potential to enhance performance, accuracy, and even efficiency in resources:

* There are limitations to the successful classical solutions when the feature space becomes large and the kernel functions become computationally expensive to estimate.

* For certain types of data, a classifier that exploits the quantum feature space shows better results.

* A necessary condition to obtain a quantum advantage is that the kernel cannot be estimated classically.

Since quantum computers are not expected to be classically simulable there is a very large design space to explore.

It becomes intriguing to find suitable feature maps for this technique with provable quantum advantages while providing significant improvement on real world datasets.

With the ubiquity of kernel methods in machine learning, in the future the techniques may produce applications even beyond binary classification.

## References

<a id="1">\[1]</a> [Oleksandr Kyriienko, Einar B. Magnusson. (2022). Unsupervised quantum machine learning for fraud detection. Preprint.](https://arxiv.org/abs/2208.01203)

<a id="kaggle">\[2]</a> \[Kaggle dataset

* Credit Card Fraud Detection:
  Anonymized credit card transactions labeled as fraudulent or genuine.]\([https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud](https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud))

<a id="3">\[3]</a> [Havlíček, V., Córcoles, A.D., Temme, K. et al. (2019). Supervised learning with quantum-enhanced feature spaces. Nature 567, 209-212.](https://doi.org/10.1038/s41586-019-0980-2)

<a id="4">\[4]</a> [Liu et al. (2020). A rigorous and robust quantum speed-up in supervised machine learning.](https://arxiv.org/pdf/2010.02174.pdf)