Retail_project.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Retail Sales Performance and Inventory Reservation Use Case\n",
        "Using RETAILER1_V, RETAILER2_V, RETAILER3_V from DWC, which are federated from Big Query.\n",
        "Also using DISTRIBUTOR_V, PRODUCT_V, and RETAIL_V which are local table views in DWC."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "## Install fedml_azure package"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "gather": {
          "logged": 1648751279358
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Processing ./fedml-azure\n",
            "Collecting hdbcli\n",
            "  Using cached hdbcli-2.12.20-cp34-abi3-manylinux1_x86_64.whl (11.7 MB)\n",
            "Collecting ruamel.yaml\n",
            "  Using cached ruamel.yaml-0.17.21-py3-none-any.whl (109 kB)\n",
            "Collecting ruamel.yaml.clib>=0.2.6; platform_python_implementation == \"CPython\" and python_version < \"3.11\"\n",
            "  Using cached ruamel.yaml.clib-0.2.6-cp36-cp36m-manylinux1_x86_64.whl (552 kB)\n",
            "Installing collected packages: hdbcli, ruamel.yaml.clib, ruamel.yaml, fedml-azure-test\n",
            "  Attempting uninstall: hdbcli\n",
            "    Found existing installation: hdbcli 2.12.20\n",
            "    Uninstalling hdbcli-2.12.20:\n",
            "      Successfully uninstalled hdbcli-2.12.20\n",
            "  Attempting uninstall: ruamel.yaml.clib\n",
            "    Found existing installation: ruamel.yaml.clib 0.2.6\n",
            "    Uninstalling ruamel.yaml.clib-0.2.6:\n",
            "      Successfully uninstalled ruamel.yaml.clib-0.2.6\n",
            "  Attempting uninstall: ruamel.yaml\n",
            "    Found existing installation: ruamel.yaml 0.17.21\n",
            "    Uninstalling ruamel.yaml-0.17.21:\n",
            "      Successfully uninstalled ruamel.yaml-0.17.21\n",
            "  Attempting uninstall: fedml-azure-test\n",
            "    Found existing installation: fedml-azure-test 2.0.0\n",
            "    Uninstalling fedml-azure-test-2.0.0:\n",
            "      Successfully uninstalled fedml-azure-test-2.0.0\n",
            "Successfully installed fedml-azure-test-2.0.0 hdbcli-2.12.20 ruamel.yaml-0.17.21 ruamel.yaml.clib-0.2.6\n",
            "Note: you may need to restart the kernel to use updated packages.\n"
          ]
        }
      ],
      "source": [
        "pip install fedml-azure --force-reinstall"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Import the libraries needed in this notebook"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "gather": {
          "logged": 1648751279661
        }
      },
      "outputs": [],
      "source": [
        "from fedml_azure import create_workspace\n",
        "from fedml_azure import create_compute\n",
        "from fedml_azure import create_environment\n",
        "from fedml_azure import DwcAzureTrain"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Set up\n",
        "### Creating a workspace. This takes a dictionary as input for parameter workspace_args.\n",
        "\n",
        "Before running the below cell, ensure that you have a workspace and replace the subscription_id, resource_group, and workspace_name with your information.\n",
        "https://docs.microsoft.com/en-us/azure/machine-learning/how-to-manage-workspace?tabs=python\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "gather": {
          "logged": 1648751280334
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:27:59,586: fedml_azure.logger INFO: Getting existing Workspace\n"
          ]
        }
      ],
      "source": [
        "#creation of workspace\n",
        "workspace=create_workspace(workspace_args={\"subscription_id\": \"<subscription_id>\",\n",
        "                                        \"resource_group\": \"<resource_group>\",\n",
        "                                        \"workspace_name\": \"<workspace_name>\"\n",
        "                                        }\n",
        "                        )\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Creating a Compute Cluster. This takes the workspace, a compute_type, and compute_args.\n",
        "https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-attach-compute-cluster?tabs=python\n",
        "\n",
        "https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.compute.amlcompute.amlcompute?view=azure-ml-py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "gather": {
          "logged": 1648751280681
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:28:00,241: fedml_azure.logger INFO: Creating Compute_target.\n",
            "2022-03-31 18:28:00,596: fedml_azure.logger INFO: Found compute target. just use it. cpu-cluster\n"
          ]
        }
      ],
      "source": [
        "#creation of compute target\n",
        "compute=create_compute(workspace=workspace,\n",
        "                   compute_type='AmlComputeCluster',\n",
        "                   compute_args={'vm_size':'Standard_D1',\n",
        "                                'compute_name':'cpu-cluster',\n",
        "                                'max_nodes':1\n",
        "                                }\n",
        "                )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Creating an Environment. This takes the workspace, environment_type, and environment_args.\n",
        "\n",
        "The whl file for the fedml_azure library must be passed to the pip_wheel_files key in the environment_args.\n",
        "\n",
        "In this example, we are using a .yml for the environments dependencies and passing the file path to environment_arg's file_path key.\n",
        "\n",
        "https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment(class)?view=azure-ml-py\n",
        "\n",
        "https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-environments"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "gather": {
          "logged": 1648751281069
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:28:00,603: fedml_azure.logger INFO: Creating Environment.\n"
          ]
        }
      ],
      "source": [
        "#creation of environment\n",
        "environment=create_environment(workspace=workspace,\n",
        "                           environment_type='CondaPackageEnvironment',\n",
        "                           environment_args={'name':'retail-env',\n",
        "                           'conda_packages':['scikit-learn', 'lightgbm'],\n",
        "                           'pip_wheel_files':['fedml-azure']})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Now, let's train the model\n",
        "### First, we need to instantiate the training class - this will assign the resources."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "gather": {
          "logged": 1648751281246
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:28:00,974: fedml_azure.logger INFO: Assigning Workspace.\n",
            "2022-03-31 18:28:00,975: fedml_azure.logger INFO: Creating Experiment\n",
            "2022-03-31 18:28:01,073: fedml_azure.logger INFO: Assigning compute.\n",
            "2022-03-31 18:28:01,075: fedml_azure.logger INFO: Assigning Environment.\n"
          ]
        }
      ],
      "source": [
        "train=DwcAzureTrain(workspace=workspace,\n",
        "                 environment=environment,\n",
        "                 experiment_args={'name':'retail-exp'},\n",
        "                 compute=compute)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Then, we need to generate the run config. This is needed to package the configuration specified so we can submit a job for training. \n",
        "\n",
        "Before running the following cell, you should have a config.json file with the specified values to allow you to access to DWC. Provide this file path to config_file_path in the below cell.\n",
        "\n",
        "You should also have the follow view IRIS_VIEW created in your DWC. To gather this data, please refer to https://www.kaggle.com/uciml/iris\n",
        "\n",
        "https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.scriptrunconfig?view=azure-ml-py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "gather": {
          "logged": 1648751281488
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:28:01,203: fedml_azure.logger INFO: Generating script run config.\n",
            "2022-03-31 18:28:01,204: fedml_azure.logger INFO: Copying config file for db connection to script_directory Retail-Project\n"
          ]
        }
      ],
      "source": [
        "#generating the run config\n",
        "src=train.generate_run_config(config_file_path='config.json',\n",
        "                          config_args={\n",
        "                                          'source_directory':'Retail-Project',\n",
        "                                          'script':'training_script.py',\n",
        "                                          'arguments':['--model_file_name','retailmodel.pkl',\n",
        "                                                       '--dist_table', 'DISTRIBUTOR_V',\n",
        "                                                        '--dist_size', '1',\n",
        "                                                        '--product_table', 'PRODUCT_V',\n",
        "                                                        '--product_size', '1',\n",
        "                                                        '--retailer_table', 'RETAIL_V',\n",
        "                                                        '--retailer_size', '1',\n",
        "                                                        '--retailer1_table', 'RETAILER1_V',\n",
        "                                                        '--retailer1_size', '1',\n",
        "                                                        '--retailer2_table', 'RETAILER2_V',\n",
        "                                                        '--retailer2_size', '1',\n",
        "                                                        '--retailer3_table', 'RETAILER3_V',\n",
        "                                                        '--retailer3_size', '1',]\n",
        "                                          }\n",
        "                            )"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Submitting the job for training"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "gather": {
          "logged": 1648752101058
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:28:01,360: fedml_azure.logger INFO: Submitting training run.\n",
            "RunId: retail-exp_1648751281_981b0e88\n",
            "Web View: https://ml.azure.com/runs/retail-exp_1648751281_981b0e88?wsid=/subscriptions/2a9092f3-f89d-4589-b9aa-8a1f9ff8b776/resourcegroups/fedml_rg/workspaces/fedml_ws&tid=997ff79a-373b-416b-8f24-a97cb0225c59\n",
            "\n",
            "Execution Summary\n",
            "=================\n",
            "RunId: retail-exp_1648751281_981b0e88\n",
            "Web View: https://ml.azure.com/runs/retail-exp_1648751281_981b0e88?wsid=/subscriptions/2a9092f3-f89d-4589-b9aa-8a1f9ff8b776/resourcegroups/fedml_rg/workspaces/fedml_ws&tid=997ff79a-373b-416b-8f24-a97cb0225c59\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#submitting the training run\n",
        "run=train.submit_run(src)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register the model for deployment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "gather": {
          "logged": 1648752101766
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:41:40,834: fedml_azure.logger INFO: Registering the model.\n",
            "Name: retail_model\n",
            "Version: 2\n"
          ]
        }
      ],
      "source": [
        "model=train.register_model(run=run,\n",
        "                           model_args={'model_name':'retail_model',\n",
        "                                       'model_path':'outputs/retailmodel.pkl'},\n",
        "                            resource_config_args={'cpu':1, 'memory_in_gb':0.5},\n",
        "                            is_sklearn_model=False\n",
        "                           )\n",
        "print('Name:', model.name)\n",
        "print('Version:', model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Deploying the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "gather": {
          "logged": 1648752154910
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "inference_config_args = {\n",
        "    'entry_script':'Retail-Project/predict.py',\n",
        "    'environment': environment\n",
        "}\n",
        "deploy_args = {\n",
        "    'workspace': workspace,\n",
        "    'name': 'retail-usecase2',\n",
        "    'models': [model],\n",
        "    'kubeconfig_path': 'kubeconfig.yml',\n",
        "    'sp_config_path': 'sp_config.json',\n",
        "    'num_replicas': 2,\n",
        "\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "gather": {
          "logged": 1648752155677
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "from fedml_azure import deploy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "gather": {
          "logged": 1648752303397
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:42:37,208: fedml_azure.logger INFO: The command kubectl already exists.\n",
            "\n",
            "2022-03-31 18:42:37,210: fedml_azure.logger INFO: The command jq already exists.\n",
            "\n",
            "2022-03-31 18:42:37,288: fedml_azure.logger INFO: The deploy namespace is fedmlazure.\n",
            "\n",
            "2022-03-31 18:42:37,964: fedml_azure.logger INFO: Creating deployment folder deployments/retail-usecase2.\n",
            "2022-03-31 18:42:40,401: fedml_azure.logger INFO: Successfully created deployment folder deployments/retail-usecase2.\n",
            "2022-03-31 18:42:40,402: fedml_azure.logger INFO: Creating docker file for deployment.\n",
            "2022-03-31 18:42:41,411: fedml_azure.logger INFO: Successfully created docker file for kyma deployment.\n",
            "2022-03-31 18:42:41,776: fedml_azure.logger INFO: Building and pushing the docker image to acr.\n",
            "\n",
            "2022-03-31 18:42:48,083: fedml_azure.logger INFO: 2022/03/31 18:42:45 Downloading source code...\n",
            "\n",
            "2022-03-31 18:42:48,085: fedml_azure.logger INFO: 2022/03/31 18:42:45 Finished downloading source code\n",
            "\n",
            "2022-03-31 18:42:50,839: fedml_azure.logger INFO: 2022/03/31 18:42:46 Using acb_vol_0199a240-0153-489c-b8ef-b0993307e20a as the home volume\n",
            "\n",
            "2022-03-31 18:42:50,840: fedml_azure.logger INFO: 2022/03/31 18:42:46 Setting up Docker configuration...\n",
            "\n",
            "2022-03-31 18:42:50,841: fedml_azure.logger INFO: 2022/03/31 18:42:46 Successfully set up Docker configuration\n",
            "\n",
            "2022-03-31 18:42:50,841: fedml_azure.logger INFO: 2022/03/31 18:42:46 Logging in to registry: 47d9cca0ebf744709229545e38c5d604.azurecr.io\n",
            "\n",
            "2022-03-31 18:42:50,842: fedml_azure.logger INFO: 2022/03/31 18:42:47 Successfully logged into 47d9cca0ebf744709229545e38c5d604.azurecr.io\n",
            "\n",
            "2022-03-31 18:42:50,842: fedml_azure.logger INFO: 2022/03/31 18:42:47 Executing step ID: build. Timeout(sec): 28800, Working directory: '', Network: ''\n",
            "\n",
            "2022-03-31 18:42:50,843: fedml_azure.logger INFO: 2022/03/31 18:42:47 Scanning for dependencies...\n",
            "\n",
            "2022-03-31 18:42:50,843: fedml_azure.logger INFO: 2022/03/31 18:42:48 Successfully scanned dependencies\n",
            "\n",
            "2022-03-31 18:42:50,844: fedml_azure.logger INFO: 2022/03/31 18:42:48 Launching container with name: build\n",
            "\n",
            "2022-03-31 18:42:50,845: fedml_azure.logger INFO: Sending build context to Docker daemon  1.731MB\n",
            "\n",
            "2022-03-31 18:42:50,845: fedml_azure.logger INFO: Step 1/6 : FROM 47d9cca0ebf744709229545e38c5d604.azurecr.io/azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:42:50,846: fedml_azure.logger INFO: latest: Pulling from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:42:50,846: fedml_azure.logger INFO: 2f94e549220a: Already exists\n",
            "\n",
            "2022-03-31 18:42:50,847: fedml_azure.logger INFO: 2b2d92136c10: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,847: fedml_azure.logger INFO: 0b412d2669ed: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,847: fedml_azure.logger INFO: 805003cfcee6: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,848: fedml_azure.logger INFO: ad877dc53455: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,848: fedml_azure.logger INFO: afa783568628: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,849: fedml_azure.logger INFO: d67cf86adb17: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,849: fedml_azure.logger INFO: 82770c3b2808: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,849: fedml_azure.logger INFO: c991e53ceb53: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,851: fedml_azure.logger INFO: 6613a8d593c6: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,852: fedml_azure.logger INFO: d09c6a1c9e45: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,852: fedml_azure.logger INFO: 07e2eaf3dba2: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,854: fedml_azure.logger INFO: 625df5ab7f88: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,855: fedml_azure.logger INFO: 7b83f7bf7a3b: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,855: fedml_azure.logger INFO: 702ee65b2d64: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,856: fedml_azure.logger INFO: 90e2e6d11c76: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,856: fedml_azure.logger INFO: 9fa1fd599abc: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,857: fedml_azure.logger INFO: fbcc3931b91b: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,857: fedml_azure.logger INFO: 55d56e91b716: Pulling fs layer\n",
            "\n",
            "2022-03-31 18:42:50,858: fedml_azure.logger INFO: ad877dc53455: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,858: fedml_azure.logger INFO: afa783568628: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,858: fedml_azure.logger INFO: d67cf86adb17: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,859: fedml_azure.logger INFO: 82770c3b2808: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,859: fedml_azure.logger INFO: c991e53ceb53: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,860: fedml_azure.logger INFO: 6613a8d593c6: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,860: fedml_azure.logger INFO: d09c6a1c9e45: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,861: fedml_azure.logger INFO: 07e2eaf3dba2: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,861: fedml_azure.logger INFO: 625df5ab7f88: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,862: fedml_azure.logger INFO: 7b83f7bf7a3b: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,862: fedml_azure.logger INFO: 702ee65b2d64: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,862: fedml_azure.logger INFO: 90e2e6d11c76: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,863: fedml_azure.logger INFO: 9fa1fd599abc: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,863: fedml_azure.logger INFO: fbcc3931b91b: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,864: fedml_azure.logger INFO: 55d56e91b716: Waiting\n",
            "\n",
            "2022-03-31 18:42:50,864: fedml_azure.logger INFO: 805003cfcee6: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:50,864: fedml_azure.logger INFO: 805003cfcee6: Download complete\n",
            "\n",
            "2022-03-31 18:42:50,865: fedml_azure.logger INFO: 0b412d2669ed: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:50,865: fedml_azure.logger INFO: 0b412d2669ed: Download complete\n",
            "\n",
            "2022-03-31 18:42:50,866: fedml_azure.logger INFO: afa783568628: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:50,866: fedml_azure.logger INFO: afa783568628: Download complete\n",
            "\n",
            "2022-03-31 18:42:50,867: fedml_azure.logger INFO: 2b2d92136c10: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:50,867: fedml_azure.logger INFO: 2b2d92136c10: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,619: fedml_azure.logger INFO: d67cf86adb17: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,620: fedml_azure.logger INFO: d67cf86adb17: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,621: fedml_azure.logger INFO: ad877dc53455: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,621: fedml_azure.logger INFO: ad877dc53455: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,622: fedml_azure.logger INFO: 82770c3b2808: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,622: fedml_azure.logger INFO: 82770c3b2808: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,623: fedml_azure.logger INFO: c991e53ceb53: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,623: fedml_azure.logger INFO: c991e53ceb53: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,624: fedml_azure.logger INFO: d09c6a1c9e45: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,624: fedml_azure.logger INFO: d09c6a1c9e45: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,625: fedml_azure.logger INFO: 6613a8d593c6: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,625: fedml_azure.logger INFO: 6613a8d593c6: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,626: fedml_azure.logger INFO: 07e2eaf3dba2: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,626: fedml_azure.logger INFO: 07e2eaf3dba2: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,627: fedml_azure.logger INFO: 625df5ab7f88: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,634: fedml_azure.logger INFO: 625df5ab7f88: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,634: fedml_azure.logger INFO: 702ee65b2d64: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,635: fedml_azure.logger INFO: 702ee65b2d64: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,635: fedml_azure.logger INFO: 90e2e6d11c76: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,636: fedml_azure.logger INFO: 90e2e6d11c76: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,636: fedml_azure.logger INFO: fbcc3931b91b: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,637: fedml_azure.logger INFO: fbcc3931b91b: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,637: fedml_azure.logger INFO: 9fa1fd599abc: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,637: fedml_azure.logger INFO: 9fa1fd599abc: Download complete\n",
            "\n",
            "2022-03-31 18:42:59,638: fedml_azure.logger INFO: 55d56e91b716: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:42:59,647: fedml_azure.logger INFO: 55d56e91b716: Download complete\n",
            "\n",
            "2022-03-31 18:43:02,614: fedml_azure.logger INFO: 7b83f7bf7a3b: Verifying Checksum\n",
            "\n",
            "2022-03-31 18:43:02,615: fedml_azure.logger INFO: 7b83f7bf7a3b: Download complete\n",
            "\n",
            "2022-03-31 18:43:02,615: fedml_azure.logger INFO: 2b2d92136c10: Pull complete\n",
            "\n",
            "2022-03-31 18:43:02,616: fedml_azure.logger INFO: 0b412d2669ed: Pull complete\n",
            "\n",
            "2022-03-31 18:43:02,617: fedml_azure.logger INFO: 805003cfcee6: Pull complete\n",
            "\n",
            "2022-03-31 18:43:05,166: fedml_azure.logger INFO: ad877dc53455: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,826: fedml_azure.logger INFO: afa783568628: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,827: fedml_azure.logger INFO: d67cf86adb17: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,827: fedml_azure.logger INFO: 82770c3b2808: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,828: fedml_azure.logger INFO: c991e53ceb53: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,828: fedml_azure.logger INFO: 6613a8d593c6: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,829: fedml_azure.logger INFO: d09c6a1c9e45: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,829: fedml_azure.logger INFO: 07e2eaf3dba2: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,830: fedml_azure.logger INFO: 625df5ab7f88: Pull complete\n",
            "\n",
            "2022-03-31 18:43:16,830: fedml_azure.logger INFO: \n",
            "\n",
            "2022-03-31 18:43:28,828: fedml_azure.logger INFO: 7b83f7bf7a3b: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,802: fedml_azure.logger INFO: 702ee65b2d64: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,803: fedml_azure.logger INFO: 90e2e6d11c76: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,804: fedml_azure.logger INFO: 9fa1fd599abc: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,804: fedml_azure.logger INFO: fbcc3931b91b: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,805: fedml_azure.logger INFO: 55d56e91b716: Pull complete\n",
            "\n",
            "2022-03-31 18:43:31,806: fedml_azure.logger INFO: Digest: sha256:6fc23e30ef9e4a4ae9fe1c564fc56da144961051110facd47dbf75d2c7538239\n",
            "\n",
            "2022-03-31 18:43:31,806: fedml_azure.logger INFO: Status: Downloaded newer image for 47d9cca0ebf744709229545e38c5d604.azurecr.io/azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd:latest\n",
            "\n",
            "2022-03-31 18:43:31,807: fedml_azure.logger INFO:  ---> e3d707df1ee2\n",
            "\n",
            "2022-03-31 18:43:31,807: fedml_azure.logger INFO: Step 2/6 : COPY azureml-app /var/azureml-app\n",
            "\n",
            "2022-03-31 18:43:31,808: fedml_azure.logger INFO:  ---> a78791a08a7f\n",
            "\n",
            "2022-03-31 18:43:31,808: fedml_azure.logger INFO: Step 3/6 : ENV AZUREML_ENTRY_SCRIPT=/var/azureml-app/predict.py\n",
            "\n",
            "2022-03-31 18:43:34,464: fedml_azure.logger INFO:  ---> Running in f336b18f05df\n",
            "\n",
            "2022-03-31 18:43:34,465: fedml_azure.logger INFO: Removing intermediate container f336b18f05df\n",
            "\n",
            "2022-03-31 18:43:34,467: fedml_azure.logger INFO:  ---> b23716099659\n",
            "\n",
            "2022-03-31 18:43:34,467: fedml_azure.logger INFO: Step 4/6 : ENV AZUREML_MODEL_DIR=/var/azureml-app/azureml-models/retail_model/2\n",
            "\n",
            "2022-03-31 18:43:34,468: fedml_azure.logger INFO:  ---> Running in 57564e2e962a\n",
            "\n",
            "2022-03-31 18:43:34,469: fedml_azure.logger INFO: Removing intermediate container 57564e2e962a\n",
            "\n",
            "2022-03-31 18:43:34,470: fedml_azure.logger INFO:  ---> 28397bc0d9d6\n",
            "\n",
            "2022-03-31 18:43:34,473: fedml_azure.logger INFO: Step 5/6 : EXPOSE 5001\n",
            "\n",
            "2022-03-31 18:43:37,361: fedml_azure.logger INFO:  ---> Running in 6f12f659a4c3\n",
            "\n",
            "2022-03-31 18:43:37,363: fedml_azure.logger INFO: Removing intermediate container 6f12f659a4c3\n",
            "\n",
            "2022-03-31 18:43:37,363: fedml_azure.logger INFO:  ---> b6a8d64fe2ef\n",
            "\n",
            "2022-03-31 18:43:37,364: fedml_azure.logger INFO: Step 6/6 : CMD [\"runsvdir\",\"/var/runit\"]\n",
            "\n",
            "2022-03-31 18:43:37,365: fedml_azure.logger INFO:  ---> Running in 9b369f9d6250\n",
            "\n",
            "2022-03-31 18:43:37,365: fedml_azure.logger INFO: Removing intermediate container 9b369f9d6250\n",
            "\n",
            "2022-03-31 18:43:37,366: fedml_azure.logger INFO:  ---> a0f89fd8ab68\n",
            "\n",
            "2022-03-31 18:43:37,367: fedml_azure.logger INFO: Successfully built a0f89fd8ab68\n",
            "\n",
            "2022-03-31 18:43:40,025: fedml_azure.logger INFO: Successfully tagged 47d9cca0ebf744709229545e38c5d604.azurecr.io/fedml/fedml_1648752161779:latest\n",
            "\n",
            "2022-03-31 18:43:40,026: fedml_azure.logger INFO: 2022/03/31 18:43:36 Successfully executed container: build\n",
            "\n",
            "2022-03-31 18:43:40,027: fedml_azure.logger INFO: 2022/03/31 18:43:36 Executing step ID: push. Timeout(sec): 3600, Working directory: '', Network: ''\n",
            "\n",
            "2022-03-31 18:43:40,027: fedml_azure.logger INFO: 2022/03/31 18:43:36 Pushing image: 47d9cca0ebf744709229545e38c5d604.azurecr.io/fedml/fedml_1648752161779:latest, attempt 1\n",
            "\n",
            "2022-03-31 18:43:40,028: fedml_azure.logger INFO: The push refers to repository [47d9cca0ebf744709229545e38c5d604.azurecr.io/fedml/fedml_1648752161779]\n",
            "\n",
            "2022-03-31 18:43:40,028: fedml_azure.logger INFO: 4d26e287e19e: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,029: fedml_azure.logger INFO: 1807ae1df389: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,029: fedml_azure.logger INFO: ae47d005f969: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,030: fedml_azure.logger INFO: 2efa82c67eb1: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,031: fedml_azure.logger INFO: 56b1531fa4a5: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,031: fedml_azure.logger INFO: 14f103440f79: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,032: fedml_azure.logger INFO: 92c4725021ba: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,032: fedml_azure.logger INFO: 0d55065e2d09: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,033: fedml_azure.logger INFO: 5ab274337413: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,033: fedml_azure.logger INFO: ae3f6b2b8154: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,034: fedml_azure.logger INFO: d23eb38afb1b: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,034: fedml_azure.logger INFO: 5a63f0a64b99: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,034: fedml_azure.logger INFO: cfd97667b73b: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,035: fedml_azure.logger INFO: dda643f89a6b: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,035: fedml_azure.logger INFO: 06f4d263b25f: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,036: fedml_azure.logger INFO: 5ecae586bbba: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,036: fedml_azure.logger INFO: eb66e810a3df: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,037: fedml_azure.logger INFO: 081c0f822cac: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,037: fedml_azure.logger INFO: 0be4c2d28ad7: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,037: fedml_azure.logger INFO: 40a154bd3352: Preparing\n",
            "\n",
            "2022-03-31 18:43:40,038: fedml_azure.logger INFO: 14f103440f79: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,038: fedml_azure.logger INFO: 92c4725021ba: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,039: fedml_azure.logger INFO: 0d55065e2d09: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,039: fedml_azure.logger INFO: 5ab274337413: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,039: fedml_azure.logger INFO: ae3f6b2b8154: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,040: fedml_azure.logger INFO: d23eb38afb1b: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,040: fedml_azure.logger INFO: 5a63f0a64b99: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,041: fedml_azure.logger INFO: cfd97667b73b: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,041: fedml_azure.logger INFO: dda643f89a6b: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,042: fedml_azure.logger INFO: 06f4d263b25f: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,042: fedml_azure.logger INFO: 5ecae586bbba: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,042: fedml_azure.logger INFO: eb66e810a3df: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,043: fedml_azure.logger INFO: 081c0f822cac: Waiting\n",
            "\n",
            "2022-03-31 18:43:40,043: fedml_azure.logger INFO: 1807ae1df389: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,044: fedml_azure.logger INFO: 56b1531fa4a5: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,044: fedml_azure.logger INFO: ae47d005f969: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,045: fedml_azure.logger INFO: 2efa82c67eb1: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,045: fedml_azure.logger INFO: 14f103440f79: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,045: fedml_azure.logger INFO: 5ab274337413: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,046: fedml_azure.logger INFO: 0d55065e2d09: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,046: fedml_azure.logger INFO: 92c4725021ba: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:40,047: fedml_azure.logger INFO: 4d26e287e19e: Pushed\n",
            "\n",
            "2022-03-31 18:43:40,047: fedml_azure.logger INFO: d23eb38afb1b: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,413: fedml_azure.logger INFO: cfd97667b73b: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,414: fedml_azure.logger INFO: ae3f6b2b8154: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,416: fedml_azure.logger INFO: dda643f89a6b: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,416: fedml_azure.logger INFO: 5a63f0a64b99: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,417: fedml_azure.logger INFO: 06f4d263b25f: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,417: fedml_azure.logger INFO: eb66e810a3df: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,419: fedml_azure.logger INFO: 081c0f822cac: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,419: fedml_azure.logger INFO: 5ecae586bbba: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,420: fedml_azure.logger INFO: 0be4c2d28ad7: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,426: fedml_azure.logger INFO: 40a154bd3352: Mounted from azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,427: fedml_azure.logger INFO: latest: digest: sha256:bffc8e7003e62307460873c886b528ed45dbee1c1b4d3fc43345eef5ca47c501 size: 4517\n",
            "\n",
            "2022-03-31 18:43:42,427: fedml_azure.logger INFO: 2022/03/31 18:43:40 Successfully pushed image: 47d9cca0ebf744709229545e38c5d604.azurecr.io/fedml/fedml_1648752161779:latest\n",
            "\n",
            "2022-03-31 18:43:42,427: fedml_azure.logger INFO: 2022/03/31 18:43:40 Step ID: build marked as successful (elapsed time in seconds: 48.658494)\n",
            "\n",
            "2022-03-31 18:43:42,428: fedml_azure.logger INFO: 2022/03/31 18:43:40 Populating digests for step ID: build...\n",
            "\n",
            "2022-03-31 18:43:42,428: fedml_azure.logger INFO: 2022/03/31 18:43:41 Successfully populated digests for step ID: build\n",
            "\n",
            "2022-03-31 18:43:42,429: fedml_azure.logger INFO: 2022/03/31 18:43:41 Step ID: push marked as successful (elapsed time in seconds: 3.812114)\n",
            "\n",
            "2022-03-31 18:43:42,429: fedml_azure.logger INFO: 2022/03/31 18:43:41 The following dependencies were found:\n",
            "\n",
            "2022-03-31 18:43:42,430: fedml_azure.logger INFO: 2022/03/31 18:43:41 \n",
            "\n",
            "2022-03-31 18:43:42,430: fedml_azure.logger INFO: - image:\n",
            "\n",
            "2022-03-31 18:43:42,438: fedml_azure.logger INFO:     registry: 47d9cca0ebf744709229545e38c5d604.azurecr.io\n",
            "\n",
            "2022-03-31 18:43:42,438: fedml_azure.logger INFO:     repository: fedml/fedml_1648752161779\n",
            "\n",
            "2022-03-31 18:43:42,439: fedml_azure.logger INFO:     tag: latest\n",
            "\n",
            "2022-03-31 18:43:42,439: fedml_azure.logger INFO:     digest: sha256:bffc8e7003e62307460873c886b528ed45dbee1c1b4d3fc43345eef5ca47c501\n",
            "\n",
            "2022-03-31 18:43:42,443: fedml_azure.logger INFO:   runtime-dependency:\n",
            "\n",
            "2022-03-31 18:43:42,444: fedml_azure.logger INFO:     registry: 47d9cca0ebf744709229545e38c5d604.azurecr.io\n",
            "\n",
            "2022-03-31 18:43:42,444: fedml_azure.logger INFO:     repository: azureml/azureml_144b16c02ed115a2ca25b2a219eb24bd\n",
            "\n",
            "2022-03-31 18:43:42,445: fedml_azure.logger INFO:     tag: latest\n",
            "\n",
            "2022-03-31 18:43:42,445: fedml_azure.logger INFO:     digest: sha256:6fc23e30ef9e4a4ae9fe1c564fc56da144961051110facd47dbf75d2c7538239\n",
            "\n",
            "2022-03-31 18:43:42,446: fedml_azure.logger INFO:   git: {}\n",
            "\n",
            "2022-03-31 18:43:42,446: fedml_azure.logger INFO: \n",
            "\n",
            "2022-03-31 18:43:42,446: fedml_azure.logger INFO: Run ID: cac was successful after 57s\n",
            "\n",
            "2022-03-31 18:43:42,667: fedml_azure.logger INFO: Checking if the deploy namespace fedmlazure exists.\n",
            "\n",
            "2022-03-31 18:43:42,991: fedml_azure.logger INFO: Using existing deploy namespace fedmlazure.\n",
            "\n",
            "2022-03-31 18:43:45,115: fedml_azure.logger INFO: The 'acrpull' role to the service principal 4ca05934-4197-446e-9f31-bb17b0a04528 to access the registry /subscriptions/2a9092f3-f89d-4589-b9aa-8a1f9ff8b776/resourceGroups/fedml_rg/providers/Microsoft.ContainerRegistry/registries/47d9cca0ebf744709229545e38c5d604 already exists.\n",
            "\n",
            "2022-03-31 18:43:45,116: fedml_azure.logger INFO: Searching if the secret 47d9cca0ebf744709229545e38c5d604.azurecr.io-4ca05934-4197-446e-9f31-bb17b0a04528-secret exists in namespace fedmlazure.\n",
            "\n",
            "2022-03-31 18:43:45,812: fedml_azure.logger INFO: Using the same secret 47d9cca0ebf744709229545e38c5d604.azurecr.io-4ca05934-4197-446e-9f31-bb17b0a04528-secret for the deployment.\n",
            "\n",
            "2022-03-31 18:43:46,849: fedml_azure.logger INFO: service/retail-usecase2 created\n",
            "\n",
            "2022-03-31 18:43:47,040: fedml_azure.logger INFO: apirule.gateway.kyma-project.io/retail-usecase2-api-rule created\n",
            "\n",
            "2022-03-31 18:43:47,229: fedml_azure.logger INFO: deployment.apps/retail-usecase2 created\n",
            "\n",
            "2022-03-31 18:43:47,635: fedml_azure.logger INFO: Waiting for deployment \"retail-usecase2\" rollout to finish: 0 out of 2 new replicas have been updated...\n",
            "\n",
            "2022-03-31 18:43:51,110: fedml_azure.logger INFO: Waiting for deployment \"retail-usecase2\" rollout to finish: 0 of 2 updated replicas are available...\n",
            "\n",
            "2022-03-31 18:44:01,211: fedml_azure.logger INFO: Waiting for deployment \"retail-usecase2\" rollout to finish: 1 of 2 updated replicas are available...\n",
            "\n",
            "2022-03-31 18:45:02,972: fedml_azure.logger INFO: deployment \"retail-usecase2\" successfully rolled out\n",
            "\n",
            "2022-03-31 18:45:02,975: fedml_azure.logger INFO: The scoring uri is https://retail-usecase2.c-9da877f.kyma.shoot.live.k8s-hana.ondemand.com/score\n",
            "\n",
            "2022-03-31 18:45:02,976: fedml_azure.logger INFO: Deleting the deploy folder 'deployments/retail-usecase2'.\n"
          ]
        }
      ],
      "source": [
        "endpoint_url = deploy(compute_type='Kyma',deploy_args=deploy_args,inference_config_args=inference_config_args)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "gather": {
          "logged": 1648752342466
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'https://retail-usecase2.c-9da877f.kyma.shoot.live.k8s-hana.ondemand.com/score'"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "endpoint_url"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "## Inferencing"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "gather": {
          "logged": 1648752346357
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "import json\n",
        "import pandas as pd\n",
        "from fedml_azure import DbConnection\n",
        "from fedml_azure import predict"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "gather": {
          "logged": 1648752544817
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "db = DbConnection()\n",
        "dist_data = db.get_data_with_headers(table_name='DISTRIBUTOR_V', size=1)\n",
        "dist_data = pd.DataFrame(dist_data[0], columns=dist_data[1])\n",
        "\n",
        "product_data = db.get_data_with_headers(table_name='PRODUCT_V', size=1)\n",
        "product_data = pd.DataFrame(product_data[0], columns=product_data[1])\n",
        "\n",
        "retailer_data = db.get_data_with_headers(table_name='RETAIL_V', size=1)\n",
        "retailer_data = pd.DataFrame(retailer_data[0], columns=retailer_data[1])\n",
        "\n",
        "retailer1_data = db.get_data_with_headers(table_name='RETAILER1_V', size=1)\n",
        "retailer1_data = pd.DataFrame(retailer1_data[0], columns=retailer1_data[1])\n",
        "\n",
        "retailer2_data = db.get_data_with_headers(table_name='RETAILER2_V', size=1)\n",
        "retailer2_data = pd.DataFrame(retailer2_data[0], columns=retailer2_data[1])\n",
        "\n",
        "retailer3_data = db.get_data_with_headers(table_name='RETAILER3_V', size=1)\n",
        "retailer3_data = pd.DataFrame(retailer3_data[0], columns=retailer3_data[1])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "gather": {
          "logged": 1648752547847
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "\n",
        "# # Request data goes here\n",
        "data = {\n",
        "    'dist_data': dist_data.values.tolist(),\n",
        "    'product_data': product_data.values.tolist(),\n",
        "    'retailer_data': retailer_data.values.tolist(),\n",
        "    'retailer1_data': retailer1_data.values.tolist(),\n",
        "    'retailer2_data': retailer2_data.values.tolist(),\n",
        "    'retailer3_data': retailer3_data.values.tolist()\n",
        "}\n",
        "test_data = json.dumps(data)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "gather": {
          "logged": 1648752723246
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:49:11,192: fedml_azure.logger INFO: Using the parameters 'endpoint_url' and 'compute_type' for inferencing as service is not passed.\n"
          ]
        }
      ],
      "source": [
        "result = predict(endpoint_url=endpoint_url,compute_type='Kyma',data=test_data)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "gather": {
          "logged": 1648752723438
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "720000"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "scores = result['result']\n",
        "len(scores)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "gather": {
          "logged": 1648752723611
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "list"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "type(scores)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "gather": {
          "logged": 1648752723739
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "result_df = pd.DataFrame(scores, columns=['retailer', 'productsku', 'calendar_year',\n",
        "                          'calendar_month', 'Predictions'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "gather": {
          "logged": 1648752732106
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "types = {'retailer': 'int',\n",
        "'productsku': 'int',\n",
        "'calendar_year': 'int',\n",
        "'calendar_month': 'int'}\n",
        "result_df = result_df.astype(types)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "gather": {
          "logged": 1648752733865
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "result_df['ID'] = result_df.index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "gather": {
          "logged": 1648752740334
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>retailer</th>\n",
              "      <th>productsku</th>\n",
              "      <th>calendar_year</th>\n",
              "      <th>calendar_month</th>\n",
              "      <th>Predictions</th>\n",
              "      <th>ID</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>1</td>\n",
              "      <td>4916.0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>2</td>\n",
              "      <td>763.0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>3</td>\n",
              "      <td>633.0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>4</td>\n",
              "      <td>4591.0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>5</td>\n",
              "      <td>3390.0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>6</td>\n",
              "      <td>800.0</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>7</td>\n",
              "      <td>4147.0</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>8</td>\n",
              "      <td>989.0</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>9</td>\n",
              "      <td>4674.0</td>\n",
              "      <td>8</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>10001</td>\n",
              "      <td>1002</td>\n",
              "      <td>2019</td>\n",
              "      <td>10</td>\n",
              "      <td>605.0</td>\n",
              "      <td>9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   retailer  productsku  calendar_year  calendar_month  Predictions  ID\n",
              "0     10001        1002           2019               1       4916.0   0\n",
              "1     10001        1002           2019               2        763.0   1\n",
              "2     10001        1002           2019               3        633.0   2\n",
              "3     10001        1002           2019               4       4591.0   3\n",
              "4     10001        1002           2019               5       3390.0   4\n",
              "5     10001        1002           2019               6        800.0   5\n",
              "6     10001        1002           2019               7       4147.0   6\n",
              "7     10001        1002           2019               8        989.0   7\n",
              "8     10001        1002           2019               9       4674.0   8\n",
              "9     10001        1002           2019              10        605.0   9"
            ]
          },
          "execution_count": 29,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result_df.head(10)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "gather": {
          "logged": 1648752742859
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:52:22,729: fedml_azure.logger INFO: creating table...\n",
            "2022-03-31 18:52:22,730: fedml_azure.logger INFO: CREATE TABLE Retail_Predictions (ID INTEGER PRIMARY KEY, retailer INTEGER, productsku INTEGER, calendar_year INTEGER, calendar_month INTEGER, Predictions FLOAT(2), INSERTED_AT TIMESTAMP NOT NULL)\n"
          ]
        }
      ],
      "source": [
        "db.create_table(\"CREATE TABLE Retail_Predictions (ID INTEGER PRIMARY KEY, retailer INTEGER, productsku INTEGER, calendar_year INTEGER, calendar_month INTEGER, Predictions FLOAT(2))\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "gather": {
          "logged": 1648757315316
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2022-03-31 18:52:24,574: fedml_azure.logger INFO: inserting into table...\n",
            "2022-03-31 18:52:24,575: fedml_azure.logger INFO: INSERT INTO Retail_Predictions (retailer, productsku, calendar_year, calendar_month, Predictions, ID, INSERTED_AT) VALUES (:retailer, :productsku, :calendar_year, :calendar_month, :Predictions, :ID, :INSERTED_AT)\n"
          ]
        }
      ],
      "source": [
        "db.insert_into_table('Retail_Predictions', result_df)"
      ]
    }
  ],
  "metadata": {
    "kernel_info": {
      "name": "python3-azureml"
    },
    "kernelspec": {
      "display_name": "Python 3.6 - AzureML",
      "language": "python",
      "name": "python3-azureml"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "microsoft": {
      "host": {
        "AzureML": {
          "notebookHasBeenCompleted": true
        }
      }
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}