adversarial_training_project.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F2cK6b9bfNlb"
      },
      "source": [
        "# Adversarial training for machine learning models\n",
        "\n",
        "## Background\n",
        "\n",
        "With the prevalence of machine learning models, it is becoming increasingly important to ensure that they are not vulnerable to adversarial attacks. We will study methods to create and defend against such attacks on machine learning models.\n",
        "\n",
        "## Basic model\n",
        "We will begin by training a traditional CNN on the CIFAR-10 dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "MCG4Fw3ke30f"
      },
      "outputs": [],
      "source": [
        "from PIL import Image\n",
        "from tqdm import tqdm\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "import torchvision\n",
        "from torchvision.models import resnet18\n",
        "from torchvision import transforms as T\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZQ-H1u54gHYp",
        "outputId": "4167298a-6e35-4f1a-abc9-810e16222428"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Files already downloaded and verified\n",
            "Files already downloaded and verified\n",
            "Image batch shape: torch.Size([64, 3, 32, 32])\n",
            "Label batch shape: torch.Size([64])\n"
          ]
        }
      ],
      "source": [
        "# Load datasets\n",
        "cifar_10_train = torchvision.datasets.CIFAR10(\"CIFAR10\", train=True, transform=T.ToTensor(), download=True)\n",
        "cifar_10_test = torchvision.datasets.CIFAR10(\"CIFAR10\", train=False, transform=T.ToTensor(), download=True)\n",
        "\n",
        "batch_size = 64\n",
        "train_loader = DataLoader(cifar_10_train, batch_size=batch_size, shuffle=True)\n",
        "valid_loader = DataLoader(cifar_10_test, batch_size=batch_size, shuffle=False)\n",
        "sample_img, sample_lbl = next(iter(train_loader))\n",
        "print(f\"Image batch shape: {sample_img.size()}\")\n",
        "print(f\"Label batch shape: {sample_lbl.size()}\")\n",
        "\n",
        "transform = T.Compose(\n",
        "    [\n",
        "        # Exact transforms used were suggested by ChatGPT\n",
        "        T.ToTensor(),\n",
        "        T.RandomHorizontalFlip(),\n",
        "        T.RandomCrop(32, padding=4),\n",
        "        T.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.2, hue=0.2),\n",
        "    ]\n",
        ")\n",
        "\n",
        "cifar_10_train = torchvision.datasets.CIFAR10(\"CIFAR10\", train=True, transform=transform)\n",
        "\n",
        "batch_size = 64\n",
        "train_loader_augmented = DataLoader(cifar_10_train, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "# Load CUDA and CPU\n",
        "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "cpu_device = 'cpu'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "MFlFoDuOgypP"
      },
      "outputs": [],
      "source": [
        "class CNN(nn.Module):\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "    self.nn = nn.Sequential(\n",
        "      # use 2 convolution layers and 4 linear layers\n",
        "      nn.Conv2d(3, 16, 5),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Conv2d(16, 64, 3),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Flatten(),\n",
        "\n",
        "      nn.Linear(2304, 256),\n",
        "      nn.ReLU(),\n",
        "\n",
        "      nn.Linear(256, 256),\n",
        "      nn.ReLU(),\n",
        "\n",
        "      nn.Linear(256, 100),\n",
        "      nn.ReLU(),\n",
        "\n",
        "      nn.Linear(100, 10),\n",
        "      nn.ReLU(),\n",
        "    )\n",
        "\n",
        "  def forward(self, x):\n",
        "    return self.nn.forward(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "so_Td_uFi78u"
      },
      "outputs": [],
      "source": [
        "def evaluate(model, data_loader):\n",
        "    \"\"\"Evaluate the model on the given dataset.\"\"\"\n",
        "    # Set the model to evaluation mode.\n",
        "    model.eval()\n",
        "    correct = 0\n",
        "    # The `torch.no_grad()` context will turn off gradients for efficiency.\n",
        "    with torch.no_grad():\n",
        "        for images, labels in tqdm(data_loader):\n",
        "            images, labels = images.to(device), labels.to(device)\n",
        "            output = model(images)\n",
        "            pred = output.argmax(dim=1)\n",
        "            correct += (pred == labels).sum().item()\n",
        "    return correct / len(data_loader.dataset)\n",
        "\n",
        "def train(model, n_epoch, optimizer, scheduler, augment=False, attack_fn=None, eps=0.01):\n",
        "    \"\"\"Train the model on the given dataset.\"\"\"\n",
        "    loss_fn = nn.CrossEntropyLoss()\n",
        "\n",
        "    loader = train_loader\n",
        "    if augment:\n",
        "      loader = train_loader_augmented\n",
        "\n",
        "    for epoch in range(n_epoch):\n",
        "        # Set the model to training mode.\n",
        "        model.train()\n",
        "        for step, (images, labels) in enumerate(loader):\n",
        "            # 0. Prepare the data. Move the data to the device (CPU/GPU).\n",
        "            images, labels = images.to(device), labels.to(device)\n",
        "            # 1. Clear previous gradients.\n",
        "            optimizer.zero_grad()\n",
        "\n",
        "            # Train with defense against adversarial attacks\n",
        "            # You can ignore this for now, it will be explained later in the notebook\n",
        "            if attack_fn is not None:\n",
        "                # Loss on the FGSM/PGD example\n",
        "                model.eval()\n",
        "                images = attack_fn(model, images, labels, eps, False)\n",
        "                model.train()\n",
        "\n",
        "            # 2. Forward pass. Calculate the output of the model.\n",
        "            output = model(images)\n",
        "            # 3. Calculate the loss.\n",
        "            loss = loss_fn(output, labels)\n",
        "            # 4. Calculate the gradients. PyTorch does this for us!\n",
        "            loss.backward()\n",
        "            # 5. Update the model parameters.\n",
        "            optimizer.step()\n",
        "            if step % 700 == 0:\n",
        "                print(f\"Epoch {epoch}, Step {step}, Loss {loss.item():.4f}\")\n",
        "        # 6. (Optional) Update the learning rate.\n",
        "        scheduler.step()\n",
        "        acc = evaluate(model, valid_loader)\n",
        "        print(f\"Epoch {epoch}, Valid Accuracy {acc * 100:.2f}%\")\n",
        "\n",
        "        acc = evaluate(model, train_loader)\n",
        "        print(f\"Epoch {epoch}, Train Accuracy {acc * 100:.2f}%\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GNc1Sa51h4lK",
        "outputId": "2412221b-d85c-451f-decb-f322f84f2f3a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Step 0, Loss 2.3021\n",
            "Epoch 0, Step 700, Loss 1.3658\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 92.65it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Valid Accuracy 52.08%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 95.88it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Train Accuracy 54.27%\n",
            "Epoch 1, Step 0, Loss 1.3532\n",
            "Epoch 1, Step 700, Loss 1.0717\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 107.63it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Valid Accuracy 58.19%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 107.33it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Train Accuracy 62.83%\n",
            "Epoch 2, Step 0, Loss 1.1432\n",
            "Epoch 2, Step 700, Loss 1.1899\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 107.76it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Valid Accuracy 60.42%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 93.53it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Train Accuracy 67.86%\n",
            "Epoch 3, Step 0, Loss 0.7600\n",
            "Epoch 3, Step 700, Loss 1.0916\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 86.79it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Valid Accuracy 62.76%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 98.52it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Train Accuracy 72.92%\n",
            "Epoch 4, Step 0, Loss 0.5645\n",
            "Epoch 4, Step 700, Loss 0.7468\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 105.90it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Valid Accuracy 63.05%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 105.15it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Train Accuracy 76.67%\n",
            "Epoch 5, Step 0, Loss 0.8476\n",
            "Epoch 5, Step 700, Loss 0.6173\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 105.46it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Valid Accuracy 63.06%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 93.92it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Train Accuracy 79.23%\n",
            "Epoch 6, Step 0, Loss 0.6354\n",
            "Epoch 6, Step 700, Loss 0.4523\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 75.26it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Valid Accuracy 62.94%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 102.60it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Train Accuracy 82.31%\n",
            "Epoch 7, Step 0, Loss 0.4217\n",
            "Epoch 7, Step 700, Loss 0.4207\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 109.22it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Valid Accuracy 63.31%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 100.97it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Train Accuracy 84.55%\n",
            "Epoch 8, Step 0, Loss 0.4740\n",
            "Epoch 8, Step 700, Loss 0.3455\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 109.08it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Valid Accuracy 62.91%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 93.75it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Train Accuracy 85.93%\n",
            "Epoch 9, Step 0, Loss 0.3670\n",
            "Epoch 9, Step 700, Loss 0.2598\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 77.18it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Valid Accuracy 62.36%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 106.74it/s]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Train Accuracy 86.42%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "# setup and train the neural network\n",
        "model_normal = CNN().to(device)\n",
        "\n",
        "lr = 1e-3\n",
        "gamma = 0.9\n",
        "num_epoch = 10\n",
        "\n",
        "optimizer = torch.optim.Adam(model_normal.parameters(), lr=lr)\n",
        "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=gamma)\n",
        "\n",
        "train(model_normal, num_epoch, optimizer, scheduler)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bVAPVciRqldx"
      },
      "source": [
        "This model achieves 60% accuracy on the validation set and 90% accuracy on the training set. We will now attempt to attack the model using several different techniques.\n",
        "\n",
        "## Methodology\n",
        "We will apply a transformation to each image in the training set, and determine whether the transformed image has the same class as the original.\n",
        "\n",
        "We begin with adding noise to the image (pixel modifications with a maxiumum $L_\\infty$ norm of $\\epsilon$) as our transformation.\n",
        "\n",
        "The first technique to explore is called FGSM (Fast Gradient Sign Method) and works by taking one step in the direction of the positive gradient from the image (ie, moving in the direction of least confidence for the classifier)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "5NF5psN8sCOG"
      },
      "outputs": [],
      "source": [
        "# The last argument 'targeted' can be used to toggle between a targeted and untargeted attack.\n",
        "# A targeted attack is one where we want the classifier to misclassify as a specific class.\n",
        "# If the attack is untargeted, target should be set to the output class of the classifier.\n",
        "def fgsm(model, x, target, eps, targeted=True):\n",
        "    finalout = []\n",
        "    L = nn.CrossEntropyLoss()\n",
        "\n",
        "    x.requires_grad_()\n",
        "    loss = L(model(x), target.clone().detach())\n",
        "    loss.backward()\n",
        "\n",
        "    # take the largest possible step in the direction of the gradient\n",
        "    finalout = torch.sign(x.grad)\n",
        "\n",
        "    if targeted:\n",
        "        finalout *= -1\n",
        "\n",
        "    return (x + (eps * finalout)).detach()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C1g6dYnlttEv"
      },
      "source": [
        "We now analyze the model's accuracy on a version of the training set perturbed by FGSM. We will use an untargeted attack to get the largest effect possible."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "e5qO-BGht5Pr"
      },
      "outputs": [],
      "source": [
        "def evaluate_adversarial_attack(model_normal, attack_fn, eps):\n",
        "  image_count = 0\n",
        "  correct_count = 0\n",
        "\n",
        "  model_normal.eval()\n",
        "  model_normal = model_normal.to(device)\n",
        "\n",
        "  for step, (images, labels) in enumerate(train_loader):\n",
        "      # 0. Prepare the data. Move the data to the device (CPU/GPU).\n",
        "      images, labels = images.to(device), labels.to(device)\n",
        "      # 1. Clear previous gradients.\n",
        "      optimizer.zero_grad()\n",
        "      # Take a step away from the correct label rather than away from the predicted label.\n",
        "      perturbed_input = attack_fn(model_normal, images, labels, eps, False)\n",
        "\n",
        "      output = model_normal(perturbed_input)\n",
        "      pred = output.argmax(dim=1)\n",
        "      correct_count += (pred == labels).sum().item()\n",
        "      image_count += len(perturbed_input)\n",
        "\n",
        "  print(\"Adversarial robustness: \", correct_count / image_count)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xa-ZsVlk0jd5",
        "outputId": "da0f28d3-af71-4c06-a921-4d88029bdcc6"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Adversarial robustness:  0.2926\n"
          ]
        }
      ],
      "source": [
        "evaluate_adversarial_attack(model_normal, fgsm, eps=0.01)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GuLFhfcaxdc_"
      },
      "source": [
        "As we can see, FGSM has a drastic effect on the accuracy, reducing it to only 29%. A 1% change in the pixel value corresponds to an RGB color change of 2.55 units, which should be impossible for a human to see with the naked eye.\n",
        "\n",
        "I also tested a 3% change, which corresponds to a 7.5 pixel value change. With this, I was able to get the robustness down to 5%, but some humans might be able to spot this, and I was unable to train an accurate robust model with it. As such, we will continue using the 1% threshold for the rest of the notebook.\n",
        "\n",
        "Next, we will rerun the attack on the same model. However, this time we will use the PyTorch data augmentation feature to train our model. Our goal is to see if using data augmentation can help our model become more resistant to the FGSM attack."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_9P2XB7XjMXa",
        "outputId": "600ab391-24a6-4ad1-bbfa-8a791d6d4d3e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Step 0, Loss 2.3044\n",
            "Epoch 0, Step 700, Loss 1.5425\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 102.21it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Valid Accuracy 46.15%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 89.78it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Train Accuracy 46.85%\n",
            "Epoch 1, Step 0, Loss 1.6362\n",
            "Epoch 1, Step 700, Loss 1.5400\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 104.18it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Valid Accuracy 51.48%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 91.82it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Train Accuracy 52.00%\n",
            "Epoch 2, Step 0, Loss 1.5104\n",
            "Epoch 2, Step 700, Loss 1.4627\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 70.25it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Valid Accuracy 54.36%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 102.70it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Train Accuracy 55.52%\n",
            "Epoch 3, Step 0, Loss 1.2779\n",
            "Epoch 3, Step 700, Loss 1.4062\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 104.93it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Valid Accuracy 57.82%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 89.88it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Train Accuracy 58.89%\n",
            "Epoch 4, Step 0, Loss 1.2912\n",
            "Epoch 4, Step 700, Loss 1.0244\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 102.94it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Valid Accuracy 59.45%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 98.60it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Train Accuracy 61.34%\n",
            "Epoch 5, Step 0, Loss 1.1272\n",
            "Epoch 5, Step 700, Loss 1.0939\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 87.54it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Valid Accuracy 60.39%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 90.66it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Train Accuracy 61.92%\n",
            "Epoch 6, Step 0, Loss 1.0809\n",
            "Epoch 6, Step 700, Loss 1.3500\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 106.37it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Valid Accuracy 60.41%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 89.76it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Train Accuracy 63.09%\n",
            "Epoch 7, Step 0, Loss 1.0595\n",
            "Epoch 7, Step 700, Loss 1.3875\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 99.11it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Valid Accuracy 61.52%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 104.52it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Train Accuracy 63.67%\n",
            "Epoch 8, Step 0, Loss 1.5299\n",
            "Epoch 8, Step 700, Loss 1.2677\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:03<00:00, 43.02it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Valid Accuracy 62.59%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:19<00:00, 40.89it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Train Accuracy 65.35%\n",
            "Epoch 9, Step 0, Loss 1.1150\n",
            "Epoch 9, Step 700, Loss 1.2373\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 104.86it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Valid Accuracy 63.90%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:09<00:00, 85.62it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Train Accuracy 66.38%\n",
            "Adversarial robustness:  0.27896\n"
          ]
        }
      ],
      "source": [
        "model_normal_augmented = CNN().to(device)\n",
        "lr = 1e-3\n",
        "gamma = 0.9\n",
        "num_epoch = 10\n",
        "\n",
        "optimizer = torch.optim.Adam(model_normal_augmented.parameters(), lr=lr)\n",
        "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=gamma)\n",
        "\n",
        "train(model_normal_augmented, num_epoch, optimizer, scheduler, augment=True)\n",
        "evaluate_adversarial_attack(model_normal_augmented, fgsm, eps=0.01)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's compare some of the training set images to the adversarial images to see if they look different:"
      ],
      "metadata": {
        "id": "5BUY9YX0Cldq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for step, (images, labels) in enumerate(train_loader):\n",
        "      # 0. Prepare the data. Move the data to the device (CPU/GPU).\n",
        "      images, labels = images.to(device), labels.to(device)\n",
        "      # 1. Clear previous gradients.\n",
        "      optimizer.zero_grad()\n",
        "      # Take a step away from the correct label rather than away from the predicted label.\n",
        "      perturbed_input = fgsm(model_normal_augmented, images, labels, 0.01, False)\n",
        "\n",
        "      image_np_original = images[0].permute(1, 2, 0).cpu().detach().numpy()\n",
        "      image_np_perturbed = perturbed_input[0].permute(1, 2, 0).cpu().detach().numpy()\n",
        "\n",
        "      fig, axes = plt.subplots(1, 2, figsize=(10, 5))\n",
        "      axes[0].imshow(image_np_original)\n",
        "      axes[0].axis('off')  # Hide axes\n",
        "\n",
        "      axes[1].imshow(image_np_perturbed)\n",
        "      axes[1].axis('off')  # Hide axes\n",
        "\n",
        "      plt.show()\n",
        "\n",
        "      break"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "id": "9Y8BV_OqCtlq",
        "outputId": "44610f32-1a82-4f32-cac0-59ebf7d2b35c"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:matplotlib.image:Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1000x500 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAF0CAYAAACkIU9RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAol0lEQVR4nO3d289l+X3n9bVPz7nqqaee6urqOnRVl/tgx2k7k9iZJBMnwESZgYBELhJGJiQxg7hBCMEVEhIXSFwh/gYEQhqBAshYISETT4Y5ZCbjQ7vbdhK72+3q7uruOj1Vz3Gf915czA0SNdOf3/Dbcmd4va7fWlp777XWt771XKxO27ZtAwAAUFH3R30CAADAv3wsGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6vpp+Fv/yX8ZH3Q2PI7bwfpm3J4OR3E7Hg7jtteJ0+bCue38uL34621G4/yz9QbLuL28fyNub928Hrd33r8Tt48PT+K2aVbzovrpZFzUd/KfrultrBWcxyRvh3l7bvtSQZvfc+18FrcHx2dx23R6cbqcz/N2uYjbbjc/h9k0/y0OHj6I25/+y5+L2//2v/rP4/b/T770n/4XcTs8ymfT1s75uH1ScNzFLL+nlgX33zMFs2lZMJvm44L7b6tgNp2/Erermk0nZz/62XS2wtm0sZFfE2eT07htj/Ln4UbBbLpYMps6JbNpGrez+WpmU6+X/87d7kbczqb571Y0m34yn03/zX/9z38G+4sGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFBd/I7JP/57X40POjp+nJ9BwRtKF7P87Y5rnfwNpdub+dsorz13OW63Co47m+afrdPN31DaPc7fnvlzn301bn/4Z9+I23e//Vrcbu/sxG23k+/Jb9+5E7dN0zRHBW/wXt/ZituS33l0lL/x88L5/bi9dCF/43G/4L8iHhe8Gfz4ZBi3nTa/3ucFb1JeLvPjjsb5+Z6M8u/h4sX8eufp/vir+Ww6LphNg438+b0Y5r95Z0Wzaf7ctbjd2szn7rDgmbW2nt9Tj3bze+rnPveTcfvDt96I21XNpsW8ZDa9GbdNUzabdvby2TQ8y3/n+bBkNuVvgL9U8mbw9ThtTg7yz3ZwchS3a21+H43m+fXeK/h38XHB+Z6WzKZze3H7UfxFAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUF7/n/NLufnzQdw8exe10NI7bV155MW6fubgdt8v5LG7X+vlutrOVn8Og14vb2WIet8OzZdzev3cYt/t7l+L23NZWQZt/ZzvnduJ2f//ZuG2apjme5r/zaJn/HpOC6/2H3/t+3A56+fmurw3idjYZxu1omLcl91xJu1jk1/v2dn6tbazn9+fmdv79Hhw8iFuermQ2HR0cxe1p0Wy6Hbermk3bm6uZTc8uC2ZTt2Q25ce9f+9h3F7Z2Y3btwf5vbq7tRa36+f24vbKlYtx2zRNc3CW/87zgtl0Os+v97vf+2HcNkWzKf+3wnCc38vHs5LZFKfNrHMct/NpPpvWC2bI3qX8+jk/z7/fe8f1ZpO/aAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6vppeO3qrfigD9+/F7ejUf4K99uf+FTc3rx5JW5f+8Y34vb6rVtxe3yYf7aHjx/H7azNX2Xf6+7F7fE8vhyaay98Mm7P7eXnsL9/saC9lLeXL8dt0zRNu7YRtx8+fhS3d+78MG5/78v57zw8GcbthQvn4/bwSf7Zzu+vx+3Z2Shum+Uib9s87fV6cTsa5vfy9lZ+7RR9Np7q8s1bcfvwwUHc3n+Yt7c/mc+mF0tm0z9e1WzK77+H90tm0zRul9183hyP8vbGy6/G7V8rmE3nC2bTlSsls+la3DZN00w7+XPrw5OjuL3z5vfj9g97/2fcDgvOYXs7n02z0/xhv9/bidvDXv6sH/S243ZnRbPp+KhgNg3y2TSYjOP2o/iLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACorp+Gjx4/ig86mozidr6cxe3jJ4dxe+3qlbh9/+69uH3+2o24PTnJX+H+vbfeidvd/b24vXX7lbiddfO989aLL8btK7f/lbhdW1+P224nP99FM4/bpmmatr+M2yvXLsTtJ24+E7dX93fi9u//X1+L2w/ffS9u+/34EdF0u9OCNv/tOp1O3I7H+T3XFBx3NMqPO370OG4vPXM1bnm6o/tHcVs0m3r5bDp5cpYf9+ogbu8UzKbnrt2O25LZ9J233orb3f3LcXvrdv4snBXMhaLZ9Mt/JW5LZtNinj/fuoP8OdQ0TTNtJ3F7Y3Yhbj95M29vXsn/DfJ3/vCP4/ZRwWza7G/G7aibPyPWu/nv3FnmM2+8KJlNa3E6ny/i9uFRwWy6Wm82+YsGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFCdRQMAAKiun4Z3P/xBfNCzSf6a86af7zrXrl6K2/3zu3F769q1uD2492HcvvGtN+J2vGjj9jd+84txe+vln4jbr/wffxC3o8P8erj6xV+P24OD/No5ODiI227Ti9umaZpefz1u5538uBtra3H73Lnrcftv/tKFuP3dr/wvcdtOjuL2ycEobnfW8vt+OV/kbWcet1uD/Lfo7+7E7XHBOfQ6Zdcl/293H61mNg1WNpu24valW6uZTX/6rW/E7bBgNn3pN38rbq8Vzabfjdt5wWx65td+NW6Pj/Nr5969fDatD1Y5mwZxu9FZ1Wz6N+L2D36/YDbdHcbtk3v5M3m9ZDYVPL87R/k/FNYGebu/lc+mg3k+o2vOJn/RAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADV9dPw9OggPuh8Povb2aKN29e+9Sdxe++DD+P2i7/x78btu+/8edz+r1/5ctz+/Bf+etx+7nNfiNvvvpN/D8vOetz+77/7t+N2sVzE7ZX9y3G7ubYWt8tpnDZN0zTzWS9ur964Frf3Cu6j9999N27/2q/8fNz+9Oc/F7f/01vfj9uL5zbjdjyZxO3mhXNxe3VwKW4Xi2Xcvv3223E7neTPv0XB84+nW9VsGo4LZtPXCmZTwTP5i3/jt+P2B+98O26/8vv5bPrLP5PPps8Wzab8+bZc24nbktk0nozj9saqZlM7iNumKZxNV/LZ9N7wXtw+eDe/hv+1Xy6YTT/xo59NJ5PTuN3cPR+3V5/LZ1N3kc/H733vMG7bH9Fs8hcNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFBdPy7nizhd6+eve18s8leiv/7a63F7/+pZ3P5H//FzcTuaHsbtYCv/Hn7m534ubk+eDOP2/Q/ux+3u3sW4ff72J+L28Cz/LQ4e/nnc3rp6PW6XBddv0zTN7t5e3L7yqefj9vQ0/y5Gkydxe+e9d+L28z+dX2svvvhy3G6s5Y+T3iBvNza38nYjv+dms3ncfve7fxq3X/3q343b176TH5d/hoJ7u1Mwm9ab/Dn79YLZdKVkNv1n+Wx6Zvowbs2mf2pls+nmC3HbK51NOztxe/tT+fVz+fR83M4np3F757234vYv/fTn4/bmpwpmU6dkNrVxe+78bn4OBbNpOCyZTfmzp2Q2fed7b8btR/EXDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQXf5e9rYTp1tbW3H7qVdfits3v/vtuF1M4rQ5OjiO2ws75+P25rWrcdtrR3H75d/5H+P2naPDuG3bXtx+4Wd/Km7X1uO0mZ4M8+MO8j35M597NT+Jpmn2nrkQt521adz+2Gdvx+3OhUHcTqeLuL146Zm4ff755+N2Msqv4TYum6Zt8mfPbDaP26Ojh3Hb7+cX8WSSP3wW8/x3459hRbPp5qufjds3v/XNuF0sZ3H76GMxm/Jz+PKX/7u4fefoLG77BbPpZz73mbgtmU3tOJ9N2wWz6VOfLZ1N23G7tp0/aX/sE68UnEN+H51Nx3F78VJ+XX4sZtNgLW5LZtPw0Qdxu1kym5b5bBrP89/to/iLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACorp+Gy4L3sh8+OYrbF/v5K9x//uf/Stw+vH8at08e3Y/bz3z2etz+5m/8O3G7v/ds3H73tW/GbX+5jNsf//FX4/YXfvbzcdsd5Pvs3rlzcXvtypW47XTLdurxYhK3k8k0bh88PMhPotuL03HBOXz9m9+O2+Fpfh8dPsk/22y6yM9hmP8W8/ksbxd52yu4fP7oj/5u3G5f2MsPzFMt2/x3PHowjtsXb69qNuXX8+mju3H7qR9bzWw6v3Mxbr/7Wv5s2ez9xZpNly/m38Pl/f24XelsKviH24OH9+J2tsh/u/Ek/3yvfzu/fo4KZtNZwWwanuXPiNks/x5Go/w51emuZjb9gz/6h3Fbczb5iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqK6fhjvbO/FBL1y4FLcXL+bt3/ytX4vb5bwTt4NeL25feOH5uL127Ytx+8EHj+P2lRdfidvpIP9sO1tbcbs+2Ijb8Wwet0fHZ3F7+OSt/Bym+Tk0TdOcjCZxe3pyGrej8Thup9NpwTmM4vbJ4/xa67SLuN1cH+TtxnbcNk1+L6+tx4+05oVP3Irbt77/Z3H79ts/iNvP/NTn45an29nei9vt7fx5ePHibtz+zf8gn029+Vrcls2mG3F769atuH3ng/tx+8qLPx637dYybncG+Wzqrmg2PTrOn7EPS2bT2epm02Sct6OTgtnU5nP6ScFsOn18ErdFs6ngv9RLZtNskM+m7YLZ9PzNW3H71nc//rPJXzQAAIDqLBoAAEB1Fg0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHXxO9EXizY+6Euv3I7bf/tXfzVun7t6NW5nszhtpqNp3L7z7qO4HU+WcXs6zk+4M+vF7ZPTYdzeff8gbtv8K2vG80Xcjob5+S7b/JrsduNLvWmapun0BvmxOyX7+mZ+3O563G6d24rbjc3zcdvPL7VmfZDHi2l+TfT7+fd7/cZzcbt7If/O/tbf+u/jdr4Yx22n04lbnm6xyB9Gn3jp03H7a7/+63H7/NXn47ZkNp0dn8Vt0WxaFsymk4LZlP+Tonny4DRul7PjuJ1O8/NdzPNnS8lsmhQMyPVuPhOaZoWzqZf/dovJxbjd3cifh+eu7cdtu8x/553BRtwupvn5lsymZwtm06VnCmbT//Dxn03+ogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqovfOT+fL+ODLpdt3LYF7etv/HncPn4yjdu13nbc7mzGX1kzXeSvcJ8WrHzrBa+Gnw/ytteux+2gN4jbrY21/Ljrs7hdtAXXWbuI23/a59d7iU6T/x7LgnMYFN1z87jtNgXHXeS/XdPNj7u7txe3F/fz9vVvfz1uX3v9m3HbNL24bAuuYZ5uVDCbesv8+d1OSmbTt+P28YPVzKb1gtnUdktmU/6sX5/nz5bRVn4OG20+IAe9nbjtFny/g/Vh3C6mBc/Nftn/965qNs1n+W+308ufcZNlfv30e/k1sZjl99GsYPx31wtm035+rT1bMptez2fTPymYTcsmv3baaf79fhR/0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1fXTcNnNd5J779+N21nBa85ns4K9aLkZp+tbu3m7WXDcbi9ul2v5a+8H3U7cLrr5K+eLts5Zfr7LJj/f9c2NuJ0vFvk5LOZx2zRN02sKPl+bf8ed/KtomoLjLuaTgjY/iflsFrdHx4dx2y24N9bX1uK2388/25987R/G7cnwLG47nfxOav1fz/9ny8WqZlP+m8+a9bhtlgXX/m4+m3bW8tnUrJfMpvifCc2g4Dm7s14ymwZxu6rZNJ/n31lnLb8mJ93C2TTL/83Ua/Nznq/lz/qm4LhrBfNmMR/G7bzgexseP4nbbsG9sb52MW7bgtn0Dwpm03iWP6d6nfw5NR0UPNM+gikHAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKrrp+HtV16JD3r94rm4/bFP5sc9G8dpMxu3cdvrx19D0+3mu9mgX/Aq+/X8uL1e/ir7tpnnbbuM2/ksb8/OhnE7nU7jdjLJ29lyFrdN0zTLxSJu20V+7OUy/z3m8/zzDU+P43Y0PIvb05PTuF0u8+/s8nPX4nZ/fz9u73/4Qdy+8Z034nZ9cyduT3tP4rbTyZ8RPN0rBbPp2WcLZtOrBbPpZBC3wyv5fb2xqtm0WTKb1uO25Pm2stnU5J/t7HBVs6ngs83yc2iasuds281n06LgnOedH/1smjwumU35vXH5+ctxu7LZ9L2S2bQXt4fL/Dtb6+T33EfxFw0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUF0/Db/0pd+OD3rz0m7ctm0btxcvXojb5Xwet91uvm/lZ9s07SI/h83OND/uMj/ubD7J21neNrNF3o6HcboYjuJ2OsqPe3J6GrdN0zRnBd/F8fFR3N67937cPnr0MG6PHuefbzLOP9vm5lbcXrt2LW6bfidO7z94Nm7f+NbX8uPeO4jb87tX4vbx/XtxW/L84+m+9B/+dtyubjblx92e58+4ktnUH8Rp044LZlNzFrfT5mMwm0r+/3ScP7uLZtM8n03jwtl0WPBdjI7z81jdbHoSt5PxMm7LZtPluG0282v4/oPrcfvGt74Zt4/fO47bktl0v/de3LZtvB58JH/RAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVxe8YHw5H8UG/9rU343bz3H7c3n7pU3G7mOevke/1enG7XC7jdj6dxO1gPozb2Wy8knbZLuK2aTtxOp3O4nY0ys/37Owsbg+ePI7bpmmaWZv/zqNRwXkcPIjbJ48P4nY2zX+PtpO3i4Lv4ejkOG43DvPP9vhx/p1tbW3F7Y3rL8Ttlasvx+3w5DBuOwXPHp5uOMyf9V/72utx2z93Pm5fWdls2ojb5SSfNyWzqSmYTU3RbMrnzbK/mtl0Ns0/W8lsmpbMpgeFs6lfMpum+Xl8HGbTPG/HBdfw0eOC2bTxo59NV65fj9sbBbPp6ORh3NacTf6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADV9dPw9W+9Hh/0Uzevxu31G9fj9u7dD+N2Pl3G7WCwFrfdbidue718j1vr5223uxm3bcFnazr5dzabLuJ21I7idtLJv99RM4/b/taluG2aptnqxbdGs7U1idvd3Stxe+NGfty2yb+3ySQ/7vHxcdxOp9O4PR0O4/bd996N20/cuh23v/CFfzVuh9P8Pnrzre/GbVtwvfN0X//W1+O2ZDa9cKVkNj2K2/k0v//KZlPeLpf583t1syk/h0GnF7ezNj/urM0/23ie36uns3w2tYWzaXcj/44Hg9P8uAWz6cqV/BruDwq+t9P8fFc3m2Zxu6rZ9FcLZtNRwWw6VzCbphVnk79oAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDq+mn41b/z1fig/6S7jNu/8e99KW4n0/h0m+PDs7hdX1+P2+3t7bjtDQZx2/R6edvk32/Tydu24Liz6SxuF4s2bkss2s247fQ3io69bDtx23by37k7WMTtem8et/NF/ns0nfx6vzjYitvhaJSfQqfgfNv8/0N6vfy3eOmll+P2jT99L2776/kzojPIrzOe7u8VzKbXFgWz6d8vmU35s+j44WHcdgtm04UL+XW3bPL7ZLCRPztnJbNpXjKbJvlxC8ZN2WzKZ/R4lbOpWYvbfsHjZTrIn7PrGwXP+nzkNesb+fX+7NZu3B4dH8ftx2M2fTpu33jz7bgtm035v7c/ir9oAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDq4neMLxf5e+Rni3nc3r17N27XNs/H7eHRYdzu71+M28ks/x4OH57Eba+/k7e9gv2ws8zbpl1J2u8P4nY6nRWcQn4S7bLghIvl33Hb5tfPcpnfR82y4HduS37n/FpbG+TX8HSW3xuzaf7ZTk/O4vbc/oW4fenll+L24dHDvL1/J255usk4v6c2mpLZdD9u1zY34/bhqmbTsGA2HRbMps29vC2ZTfNJ3g4KnlnTfN7085+tOTvLZ1O/4HxLZ9O4yZ9xRbNpWjCbJiWzqZe3/fy7WC7ya217K7+Gz4aP43Z4ll/Dp48P4/bclQtxWzKbPnhUMJse34nbj+IvGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgun4adpr81fBb62txOx2dxe2lK8/F7Qu3b8btrRvX4nbZzuL2gw8O4raddeL2wQfvxe2b77wbtxubm3G7vb0dt+1yI27Hw2Hcdgv25EW7jNumaZpF/nM03YJ22c7jtl0u8gPnl2WzXJZ9F6l+L36cNItFfg6H48O4Xc4ncXtlejVun7n6ibi9/lx+3NHxk7jl6TqDfDYNimbTYdxeunIpbq/fyGfTS7dXM5veKZhN/Vn+nRXNpjtvx+3KZlMvn02LYf79toNVzqb8PLqdQdwWzaZJPptms/x8ewWzaTbIP9tmyWya9uK2ZDZ9v5PPphtt/owomU0vPJcfdzQ6jduP4i8aAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKC6+L3si0X+evr3774ftzsX9uJ2sHshbhdt/nr6G9euxu0Lt2/E7dbOpbgdLGZxe75/FrfNIP/dzu8/E7fb2+fittvNf4vZeBq3zbKN08l0kh+3aZpRQT8Zj+L25PgoP4dRftzxoqAdjeN2Ost/j07TidteN2+Xi/x8Dw/z++jhwYO4vXTvYdxunNuP293Ni3HL03ULZtODH+az6c3tktmUPzsX0/x5OC6YTS/fvh23g/V8Nm0V/Hfk+bZgNjX5M2uzaDbl99T6esFsOin4bL34n1bls+msYDYVzIWi2dQc523BfDwtmE1twWwaDtbidrkomU0ncXtWMpvu5bPpuXsfxG3JbLrUrzeb/EUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFCdRQMAAKjOogEAAFTXT8Pt3b34oO/fvxe3x+P8NfJvv3s3br/52ptx220Xcbu5FX9lze/8b38Yt8/tbsXt99/4R3H7cJi/9v7m7Rfj9gu/+Etx2+1vxu2tm5fi9tnL+3HbLvPfuGma5uR0GLfLRX7sO3fuxO2jR4/idjQ6iduzs9O4PT7Jj7so+B4GvV7cHj55GLfnzufPqZJL4v6j/Jl289wzcbu2vpGfBE+1vXU5bu/MDuJ2VbPpT+79adx22/z/As/v5s/Z//l3fi9uPw6z6cVXSmbTvx63JbPp2s3rcfvs3pW47fdWN5sm43Hcvn/n/bh9dPRh3JbNpkncHjx+HLcls2mrYDY9fJJfwxf389k0yX+25u6HJbPpatyu7eTfw0fxFw0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUF0/DZeDrfigGxeejdvlYDtuT0bzuB2s78Tt+d1LcdsdbMbt+ub5uH18dBS3jw6O8/Z0krePX4vbT7z8E3G7aAZxOxyexe10Norbc9v59ds0TdMuOnE7Go3j9sLeftxubZ+L2+Ek/y76/fi2b84Kfo/5bBa3G2vrcdt0FnH68ssvxG3b5v/P8uG9x3H76CC/Hu59+DBuebrlVslsuhi3k5LZdJzff1sls+nSv7yz6ahgNn3j5DtxWzSbZvlsOjo6jNvp9fx6uFg4m6bdgtl0lj+Ltvfya2JwIZ9NsxXNpsOC3+PjMJs+/emX47Zt8+O++95qZtPBhx/E7UfxFw0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUF38vvf1nb34oLtXBnE77a7F7UaTvxp+f/+5uL1w8XLcDtZ34vblT34mbh99+G7cfvan8tfTv3X3XtyOJvnr6cejSdwenx7F7YMP34nb06P8d9vbPR+3TdM083H+Hd9/eBC3mzvn4rbt5vfRxlZ+f15+9lLc9gfbcdvLT7fZ2NjIj9vLj3s63orb2WwZt+NZfty25L9vSj4cT7Wq2dR2O3G7KJhN5/+CzaajR/lsWl8rmE0/zJ+b88lJ3J4cl8ymYdy+/+EHcXt69CBuL1/aj9umKZtNdz8omU35M7lsNuXX8LXr+WzaLJhN7WCWH7dgNi3zdGWzaTHLr4d+wWya9Qo+3EfwFw0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUF0/DX/xF345Puh82sZtp5u3JWvRWrMZt0en07j9vd//o7jdu/xS3O5cvB63l/fOx+1Z9624fefdd+P2/r0P4na5WMbtfDaM2zdPD+J2a2MjbpumaTr5rdE0vbU4XXQGcdvfyNu9nctxe3ic/x7LZcH30HTicpzfcs2ymcftgyeP4rZt8/Ntml5cTiaLuF02Bc8/nuoXf7ZgNrWrmU2zxSxuS2bToyf5jfL9r/xB3O4UzaYX4vby3n7cni3+LG5XNZt63YLZNCyZTffi9sG50tmUXz+zZf6MW3QuxW3/3FbcXt4rmE0H+e8xKZhNg0E+S09G+bN+eTyK2ycPCmZTP595q5pNvVm92eQvGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACguvgd7ru7z+QH7eWvp19fz1+f3nby19P32/W4HQ8P4/beg8dxu391M2473finaPYu7MXtjXl+3PfuFXy2i+fj9lf++i/F7eNHj+L28PFB3M4ms7htmqbp9fPr53Q8z88jv+Wa9Z0L+XGXi7idzvP7qNPpxG2J5bLNz6Gbt02noC34f5ZON2/nbd4u2pLz5Wm2CmbT5saPfja104Lro2A2vVcwmz5dMpvW87ZkNt0erWo27cftr/xbBbPpw5LZdC9uZ5M4bZqmaXqb+WyanIzidlg0m/J7blgymwruo06bz91mOojTZT9/Js8L2tm84N8gs/zZs7aq2VTw2T6Kv2gAAADVWTQAAIDqLBoAAEB1Fg0AAKA6iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOrid87vbJ2PDzqdFLzCfZq/cr7pFrwSvZO/9r7TW4vbWy+8EreLgo82L2jvH5zF7bvvfBC3m2vrcfvjr/5E3K5v7cbtuQu9uN0+90zcXti9ELdN0zTd3iBu37//KG4fHZ7G7cloEreT8Shu207+/wuduGyaplNS5+fQLvObo23y+75pC9r5PE6ns2ncLtuCG5+n2tnNZ1N7mv82o3l+PQ/W89nUWxRcdxvbcbqq2XSSP1qau4t8Nv3gnXfitmQ2/aWC2bRTMJtmz6xoNl26ELdN0zTrK5pNHz78GMymecG/2+KycDS1+WyaT/LnycpmU2c1s2lScTb5iwYAAFCdRQMAAKjOogEAAFRn0QAAAKqzaAAAANVZNAAAgOosGgAAQHUWDQAAoDqLBgAAUJ1FAwAAqK6fhp1lGx+0XeSvp28LXg3fKXgj+qIgLtm2NjZ383OYl3wP+fnOCk5479x23G42F+P2ZJSfxD9+4724bdr8Ous0+cWzt5efQtM0zXQ6j9vROP+dC26jZjrPv+NlyYGb/ForuOUKfo2madv8+22bks9W8JxqF3Hb6eafbrY4y89hOYtbnm6t4Nofdwvu1YILup0P4nZRcF9388M2587ns2k8W81saib5fV02m56N28cFs+nvr2o2zfLv4fJJ2TPgbJpfmPOCf4tNpvk5tCubTb28LLg3lk3+rG8LfrvFLP/tBoP4n9tN2+bHXS7W4rZkNvWXw7j9KP6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqi9+J/v7dt+KDTib5a+87nU7exmXZcZsVnUPJcduCnW/QaeP27PBh3PYKXnv/g7d/ELfjRZwW/W7tclly4LwttKpruMSyza+JpqRd0ffWrup8S86hoC35GkaTk7idjE8LzoKnef+Hfxa3k2XBbJoX3NeDQd4WXEyz+Tw/bpOfw2AtP4dpkz/AH8/zu2p6eBi3vX4+m959L59NJwWzaa3gd5tO8tn0dsFv0TRNk38TZeecX2llJ7Fsp3ncxv8kbQZr+RnPZvm90S+YN7OCdjDIv7S24F5e3WzKn5UfxV80AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqLBoAAEB18fvex6f344M+ePggbrudFe06Ba9l75TFq0ibTrssOG7+2vvFYh63/W7+WzwZHufnkJ/uyn6Lti04iaZpup384CWHbgt+uxKln+8vlJKPVnLTFSm5Hhb5Yefjf4Fz4f/pZJLPpicfFMymtfV/kdP5aCWPuFlBvFZy3Fne9lczm8aL/LNtrmg2jadx2qwNCn6LfOw2037BSTRNs140mwb5ecwKHrT5YZu2Lfl8BQcuuIYHBYdt8sMW/Au6Kbomyk44P3DbFvx7e35ScA7/fP6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqi1+g/pOv3ooP+nu//7247a2txW3Ttnm7IiVn0Cloe81iRUfOd8nlsuC47WgFZ1CmLbgeOqWXTsFXUfJrlLRlVnfk1fiLdi+X3Bt52u8vC86Cp/nJV1+K2z+883bcLgdn+Um08ShtBvlRm6Y7y08hfyQ384JT2JiXPMHz+2RjkM+8yaTg3wlrBbOp4KN1CkZ0W/AQWBvmv3HTNM2sU3QFxbpNwXkUnMOg4JqYzU7ycyi4k3oFv/Nskf92g5Jbo6Cdjo/jdm1Q8m/o/Dfut/Vmk79oAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQnUUDAACozqIBAABUZ9EAAACqs2gAAADVWTQAAIDqOm3b5u9bBwAACPiLBgAAUJ1FAwAAqM6iAQAAVGfRAAAAqrNoAAAA1Vk0AACA6iwaAABAdRYNAACgOosGAABQ3f8N/JEciWBgJ8kAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cWcssuhM-f2W"
      },
      "source": [
        "We can see that data augmentation improved the accuracy of the model significantly, up to 70% on my best run. However, the accuracy on the adversarial set only increased slightly, up to 31%.\n",
        "\n",
        "(I didn't use different hyperparameters, I just reran the model with different random initializations.)\n",
        "\n",
        "This model uses many linear layers, which may make it prone to overfitting. I also tried using a fully convolutional network but was unable to get sufficient accuracy on the validation set to justify analyzing the model (my max accuracy was only 55%). For reference, I have included the model that I was using below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gelyQXOx-fDP"
      },
      "outputs": [],
      "source": [
        "class FullyConvolutionalNetwork(nn.Module):\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "    self.nn = nn.Sequential(\n",
        "      # use 5 convolution layers and 1 linear layer\n",
        "      nn.Conv2d(3, 16, 3, padding=1),\n",
        "      nn.BatchNorm2d(16),\n",
        "      nn.ReLU(),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Conv2d(16, 32, 3, padding=1),\n",
        "      nn.BatchNorm2d(32),\n",
        "      nn.ReLU(),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Conv2d(32, 64, 3, padding=1),\n",
        "      nn.BatchNorm2d(64),\n",
        "      nn.ReLU(),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Conv2d(64, 128, 3, padding=1),\n",
        "      nn.BatchNorm2d(128),\n",
        "      nn.ReLU(),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Conv2d(128, 128, 3, padding=1),\n",
        "      nn.BatchNorm2d(128),\n",
        "      nn.ReLU(),\n",
        "      nn.MaxPool2d(2, stride=2),\n",
        "\n",
        "      nn.Flatten(),\n",
        "\n",
        "      nn.Linear(128, 10),\n",
        "      nn.ReLU(),\n",
        "    )\n",
        "\n",
        "  def forward(self, x):\n",
        "    return self.nn.forward(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2xX1bPCPIecA"
      },
      "source": [
        "As a final comparison, let's see if a pretrained ResNet is also vulnerable to this attack:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qu0KAbt8CvSm"
      },
      "outputs": [],
      "source": [
        "class ResNetModel(nn.Module):\n",
        "  def __init__(self):\n",
        "    super().__init__()\n",
        "\n",
        "    self.nn = nn.Sequential(\n",
        "        resnet18(pretrained=True),\n",
        "        nn.Linear(1000, 10),\n",
        "    )\n",
        "\n",
        "  def forward(self, x):\n",
        "    return self.nn.forward(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JEmifJ1iJF5f",
        "outputId": "c3e7efe8-7ebb-4195-e6ca-de0e6914077e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
            "  warnings.warn(msg)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Step 0, Loss 3.4287\n",
            "Epoch 0, Step 700, Loss 0.7794\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 82.47it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Valid Accuracy 71.96%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.26it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Train Accuracy 75.23%\n",
            "Epoch 1, Step 0, Loss 0.8054\n",
            "Epoch 1, Step 700, Loss 0.5747\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 82.53it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Valid Accuracy 75.28%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:12<00:00, 63.05it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Train Accuracy 81.02%\n",
            "Epoch 2, Step 0, Loss 0.5489\n",
            "Epoch 2, Step 700, Loss 0.4944\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 76.07it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Valid Accuracy 79.82%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.23it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Train Accuracy 87.82%\n",
            "Epoch 3, Step 0, Loss 0.3944\n",
            "Epoch 3, Step 700, Loss 0.2531\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 61.58it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Valid Accuracy 76.39%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 68.58it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Train Accuracy 85.32%\n",
            "Epoch 4, Step 0, Loss 0.3545\n",
            "Epoch 4, Step 700, Loss 0.2827\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 63.64it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Valid Accuracy 80.75%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 68.25it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Train Accuracy 93.48%\n",
            "Epoch 5, Step 0, Loss 0.3886\n",
            "Epoch 5, Step 700, Loss 0.3330\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.99it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Valid Accuracy 81.31%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 67.37it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Train Accuracy 95.86%\n",
            "Epoch 6, Step 0, Loss 0.0958\n",
            "Epoch 6, Step 700, Loss 0.2253\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.85it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Valid Accuracy 79.57%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 67.44it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Train Accuracy 96.03%\n",
            "Epoch 7, Step 0, Loss 0.0373\n",
            "Epoch 7, Step 700, Loss 0.1180\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 78.70it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Valid Accuracy 79.36%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.92it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Train Accuracy 95.82%\n",
            "Epoch 8, Step 0, Loss 0.0612\n",
            "Epoch 8, Step 700, Loss 0.0711\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 82.21it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Valid Accuracy 81.77%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.89it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Train Accuracy 98.55%\n",
            "Epoch 9, Step 0, Loss 0.0345\n",
            "Epoch 9, Step 700, Loss 0.0418\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 82.84it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Valid Accuracy 81.85%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.61it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Train Accuracy 99.27%\n",
            "Adversarial robustness:  0.32264\n"
          ]
        }
      ],
      "source": [
        "# setup and train the neural network\n",
        "model_resnet = ResNetModel().to(device)\n",
        "\n",
        "lr = 1e-3\n",
        "gamma = 0.9\n",
        "num_epoch = 10\n",
        "\n",
        "optimizer = torch.optim.Adam(model_resnet.parameters(), lr=lr)\n",
        "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=gamma)\n",
        "\n",
        "train(model_resnet, num_epoch, optimizer, scheduler)\n",
        "evaluate_adversarial_attack(model_resnet, fgsm, eps=0.01)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "I also tried an augmented ResNet, but the performance in terms of accuracy or robustness did not change, while the training cost went up by 3x.\n",
        "\n",
        "As we can see, even ResNet is vulnerable to the FGSM attack. We will now look at a defense that will make the model more resilient, without changing the underlying architecture.\n",
        "\n",
        "In order to accomplish this, we can't simply add the adversarial set to the list of transforms. As we have already seen, adding transforms such as crop or color jitter does not increase the model robustness. Instead, we will update the loss function. Instead of computing L(x), where x is the training example, we will compute L(x'), where x' is the worst adversarial example we can find in the $L_\\infty$ ball centered at the original image.\n",
        "\n",
        "(I have already included this code in the training function above, feel free to refer back to it to view the code.)\n",
        "\n",
        "Let's now run this adversarial training on our simple CNN and see how the results compare."
      ],
      "metadata": {
        "id": "iSsf0jG4uDUJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_normal_resilient = CNN().to(device)\n",
        "lr = 1e-3\n",
        "gamma = 0.9\n",
        "num_epoch = 10\n",
        "\n",
        "optimizer = torch.optim.Adam(model_normal_resilient.parameters(), lr=lr)\n",
        "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=gamma)\n",
        "\n",
        "train(model_normal_resilient, num_epoch, optimizer, scheduler, augment=True, attack_fn=fgsm, eps=0.01)\n",
        "evaluate_adversarial_attack(model_normal_resilient, fgsm, eps=0.01)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zCegw_h-bgV7",
        "outputId": "5814e7f6-89a3-4853-e1c5-46ed0a5634df"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Step 0, Loss 2.3102\n",
            "Epoch 0, Step 700, Loss 2.0449\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 109.44it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Valid Accuracy 42.65%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 103.50it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Train Accuracy 42.78%\n",
            "Epoch 1, Step 0, Loss 1.9153\n",
            "Epoch 1, Step 700, Loss 1.8754\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 109.03it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Valid Accuracy 49.58%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 91.87it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Train Accuracy 50.11%\n",
            "Epoch 2, Step 0, Loss 1.8666\n",
            "Epoch 2, Step 700, Loss 1.5932\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.25it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Valid Accuracy 51.88%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 108.04it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Train Accuracy 52.65%\n",
            "Epoch 3, Step 0, Loss 1.8251\n",
            "Epoch 3, Step 700, Loss 1.8373\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 107.39it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Valid Accuracy 54.23%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 97.32it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Train Accuracy 55.17%\n",
            "Epoch 4, Step 0, Loss 1.7776\n",
            "Epoch 4, Step 700, Loss 1.8425\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 109.86it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Valid Accuracy 53.92%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 94.64it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Train Accuracy 54.93%\n",
            "Epoch 5, Step 0, Loss 1.9446\n",
            "Epoch 5, Step 700, Loss 1.9539\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 78.23it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Valid Accuracy 57.29%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 105.11it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Train Accuracy 58.22%\n",
            "Epoch 6, Step 0, Loss 1.5566\n",
            "Epoch 6, Step 700, Loss 1.8571\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 111.17it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Valid Accuracy 57.69%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 97.45it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Train Accuracy 59.20%\n",
            "Epoch 7, Step 0, Loss 1.4711\n",
            "Epoch 7, Step 700, Loss 1.6363\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 100.57it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Valid Accuracy 59.43%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 95.62it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Train Accuracy 60.55%\n",
            "Epoch 8, Step 0, Loss 1.7192\n",
            "Epoch 8, Step 700, Loss 1.4698\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 97.01it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Valid Accuracy 59.40%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:07<00:00, 103.27it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Train Accuracy 60.67%\n",
            "Epoch 9, Step 0, Loss 1.6812\n",
            "Epoch 9, Step 700, Loss 1.6688\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 108.28it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Valid Accuracy 60.13%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:08<00:00, 93.61it/s] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Train Accuracy 61.66%\n",
            "Adversarial robustness:  0.46318\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Using the adversarial training, we see an accuracy of 60%, which is somewhat lower than the previously observed 70%. We also notice that the robustness jumps from 30% to 46%, which is a large improvement.\n",
        "\n",
        "Let's run the same adversarial training method on our ResNet to see if we get similar results."
      ],
      "metadata": {
        "id": "yQ5w6KzI0sG_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# setup and train the neural network\n",
        "model_resnet_resilient = ResNetModel().to(device)\n",
        "\n",
        "lr = 1e-3\n",
        "gamma = 0.9\n",
        "num_epoch = 10\n",
        "\n",
        "optimizer = torch.optim.Adam(model_resnet_resilient.parameters(), lr=lr)\n",
        "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=gamma)\n",
        "\n",
        "train(model_resnet_resilient, num_epoch, optimizer, scheduler, attack_fn=fgsm, eps=0.01)\n",
        "evaluate_adversarial_attack(model_resnet_resilient, fgsm, eps=0.01)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "712xjvIS3Kgd",
        "outputId": "beafcba1-9be9-4d2d-b57f-1a0ce5b1cf2f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Step 0, Loss 3.4566\n",
            "Epoch 0, Step 700, Loss 1.4503\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 66.12it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Valid Accuracy 63.47%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.79it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Train Accuracy 64.46%\n",
            "Epoch 1, Step 0, Loss 1.3081\n",
            "Epoch 1, Step 700, Loss 1.3018\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.07it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Valid Accuracy 63.29%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 65.95it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1, Train Accuracy 65.30%\n",
            "Epoch 2, Step 0, Loss 1.3194\n",
            "Epoch 2, Step 700, Loss 1.2622\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.86it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Valid Accuracy 55.69%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.79it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2, Train Accuracy 59.26%\n",
            "Epoch 3, Step 0, Loss 1.1126\n",
            "Epoch 3, Step 700, Loss 1.1906\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 82.32it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Valid Accuracy 68.80%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 67.30it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3, Train Accuracy 71.80%\n",
            "Epoch 4, Step 0, Loss 1.0534\n",
            "Epoch 4, Step 700, Loss 1.3386\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 78.62it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Valid Accuracy 67.68%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 68.10it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4, Train Accuracy 71.51%\n",
            "Epoch 5, Step 0, Loss 1.1309\n",
            "Epoch 5, Step 700, Loss 1.0912\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 59.55it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Valid Accuracy 64.97%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 67.69it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5, Train Accuracy 68.20%\n",
            "Epoch 6, Step 0, Loss 1.1070\n",
            "Epoch 6, Step 700, Loss 1.0707\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:02<00:00, 65.01it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Valid Accuracy 68.47%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 67.68it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6, Train Accuracy 73.27%\n",
            "Epoch 7, Step 0, Loss 1.2581\n",
            "Epoch 7, Step 700, Loss 1.2523\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 79.51it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Valid Accuracy 66.40%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.22it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7, Train Accuracy 70.92%\n",
            "Epoch 8, Step 0, Loss 1.2568\n",
            "Epoch 8, Step 700, Loss 1.0830\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 81.53it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Valid Accuracy 63.62%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:11<00:00, 66.75it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8, Train Accuracy 67.72%\n",
            "Epoch 9, Step 0, Loss 1.2566\n",
            "Epoch 9, Step 700, Loss 1.1183\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 157/157 [00:01<00:00, 80.96it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Valid Accuracy 66.52%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 782/782 [00:12<00:00, 65.01it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9, Train Accuracy 71.08%\n",
            "Adversarial robustness:  0.50418\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As we can see, getting the ResNet to train using adversarial training leads to lower results in terms of accuracy, as the entire network itself needs to be significantly changed to be more robust.\n",
        "\n",
        "The results we have seen so far also have the following two issues:\n",
        "- After using adversarial training, we are only getting about 50% robustness, which is very low compared to our desired value.\n",
        "\n",
        "To combat this issue, we will use adversarial training with a stronger attack known as PGD. This attack performs FGSM iteratively, while clipping the result at eash iteration to stay within the $L_\\infty$ ball of the original input. Using this, we should be able to get more robust models, and better adversarial examples on models without proper defenses."
      ],
      "metadata": {
        "id": "-txFKyV-3P5n"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dtLuxt5KtEyf"
      },
      "outputs": [],
      "source": [
        "# Ensure we remain within the L-inf ball of the original image.\n",
        "# This is not needed for FGSM as it only takes a single step, and the magnitude of the step is at most epsilon.\n",
        "def clip(x, x_baseline, eps):\n",
        "    x = torch.minimum(x, x_baseline + eps)\n",
        "    x = torch.maximum(x, x_baseline - eps)\n",
        "    return x\n",
        "\n",
        "def pgd_untargeted(model, x, labels, eps, targeted=False, k=5):\n",
        "    # No targeted version was implemented, since it is not needed to make the model robust\n",
        "    fgsm_in = x.clone().detach()\n",
        "    fgsm_in.requires_grad_()\n",
        "    for i in range(k):\n",
        "      # Apply FGSM over k iterations to see if we can get a more adversarial image\n",
        "      perturb = fgsm(model, fgsm_in, labels, eps, False)\n",
        "      fgsm_in = clip(perturb, x, eps)\n",
        "    return fgsm_in"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "evaluate_adversarial_attack(model_normal_augmented, pgd_untargeted, eps=0.01)\n",
        "evaluate_adversarial_attack(model_resnet, pgd_untargeted, eps=0.01)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vm7Tqc8I47kx",
        "outputId": "d934aad2-c77e-44f5-82f7-7d63fbe9b472"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Adversarial robustness:  0.26618\n",
            "Adversarial robustness:  0.15744\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As we can see, the robustness does down when we attack the model using PGD. This effect is especially noticeable for ResNet.\n",
        "As a final test, let's see if the adversarially trained models generalize and are resilient against the PGD attack on the same $ϵ$."
      ],
      "metadata": {
        "id": "zme1a2CptkdI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "evaluate_adversarial_attack(model_normal_resilient, pgd_untargeted, eps=0.01)\n",
        "evaluate_adversarial_attack(model_resnet_resilient, pgd_untargeted, eps=0.01)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xzvQmyriAJIq",
        "outputId": "55835b4d-0133-4eb7-ea34-1a3e11beafbc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Adversarial robustness:  0.44676\n",
            "Adversarial robustness:  0.48548\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this specific case, robustness remains quite high even against the PGD attack. However, this is not true in the general case.\n",
        "\n",
        "In practice, it has been shown that networks that are adversarially trained against PGD are robust against most common attacks, while networks trained on FGSM are not. However, even PGD networks are not provably robust.\n",
        "\n",
        "We have also not shown that our training method is robust on real data, but only on the training set. In practice, results on adversarial robustness tend to generalize, although this is not always the case."
      ],
      "metadata": {
        "id": "FjpVGSKr_Zl2"
      }
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}