![](\"https://github.com/thangckt/note_ml/blob/main/notebook/0_basic_MLDL/image/1_1_machine-learning.png?raw=1\")
\n",
- "\n",
- "### Supervised Learning\n",
- "Given `features` we want our model to predict `label`. [See more](https://thangckt.github.io/pytorch_deep_learning/02_pytorch_classification/#2-building-a-model)\n",
- "\n",
- "- Classification\n",
- " - Decision Trees\n",
- " - Naive Bayers Classification\n",
- "- Regession\n",
- " - Ordinary Least Squares Regression\n",
- " - Logistic Regession\n",
- " - Support Vector Machines\n",
- " - Ensemble Methods\n",
- "\n",
- "### Unsuppervised Learning\n",
- "No `label` in this type\n",
- "- Clustering\n",
- " - Centroid-based algorithm\n",
- " - Connectivity-based algorithm\n",
- " - Density-based algorithm\n",
- " - Probabilistic\n",
- " - Dimensionality Reduction\n",
- " - Neural network/ Deep Learning\n",
- "- Pricipal Component Analysis\n",
- "- Independent Component Analysis\n",
- "- Singular Value Decomposition\n",
- "\n",
- "### Reinforement Learning"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "gS25eAgTXl8x"
- },
- "source": [
- "## The Ingredients \n",
- "\n",
- "Machine learning the fitting of models $\\hat{f}(\\vec{x})$ to data $\\vec{x}, y$ that we know came from some ``data generation'' process $f(x)$ . Firstly, definitions:\n",
- "\n",
- "**Features** \n",
- "\n",
- " set of $N$ vectors $\\{\\vec{x}_i\\}$ of dimension $D$. Can be reals, integers, etc.\n",
- "\n",
- "**Labels** \n",
- "\n",
- " set of $N$ integers or reals $\\{y_i\\}$. $y_i$ is usually a scalar\n",
- " \n",
- "**Labeled Data** \n",
- "\n",
- " set of $N$ tuples $\\{\\left(\\vec{x}_i, y_i\\right)\\}$ \n",
- "\n",
- "**Unlabeled Data** \n",
- "\n",
- " set of $N$ features $\\{\\vec{x}_i\\}$ that may have unknown $y$ labels\n",
- "\n",
- "**Data generation process**\n",
- "\n",
- " The unseen process $f(\\vec{x})$ that takes a given feature vector in and returns a real label $y$ (what we're trying to model)\n",
- "\n",
- "**Model**\n",
- "\n",
- " A function $\\hat{f}(\\vec{x})$ that takes a given feature vector in and returns a predicted $\\hat{y}$\n",
- "\n",
- "**Predictions**\n",
- "\n",
- " $\\hat{y}$, our predicted output for a given input $\\vec{x}$."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "bnrjRaaHXl8x"
- },
- "source": [
- "```{note}\n",
- "The content in this part is primary from: \n",
- "- [Deep Learning for molecules & materials](https://dmol.pub/ml)\n",
- "```\n",
- "\n",
- "```{seealso}\n",
- "1. [Introductory Machine Learning](https://ai.stanford.edu/~nilsson/mlbook.html)\n",
- "2. Two reviews of machine learning in materials{cite}`fung2021benchmarking,balachandran2019machine`\n",
- "3. A review of machine learning in computational chemistry{cite}`gomez2020machine`\n",
- "4. A review of machine learning in metals{cite}`nandy2018strategies`\n",
- "```"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "KEvKnDJ4Xl8x"
- },
- "source": [
- "## Terminologies in ML\n",
- "\n",
- "- The patterns: the learned parameters in model, or the parameters to find in the relationship between inputs and outputs. For e.g., in linear model $y = ax +b$, the learned patterns (paramters to be found) are the weight `a` and the bias `b`.\n",
- "- Hidden units: neurons in hidden layers\n",
- "- Hypeparameters: are all user-choice parameters in model (e.g., learning rate, number of layers, number of neuron in layers,...)\n",
- "- Epoch: optimize step\n",
- "- Loss function: measures how wrong your model predictions are. The higher the loss, the worse your model. It is sometimes calles \"loss criterion\", \"criterion\", or \"cost function\"."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "M_zMggqQXl8x"
- },
- "source": [
- "## Workflow in ML\n",
- "\n",
- "This workflow work with PyTorch. See [this lesson](https://thangckt.github.io/pytorch_deep_learning/01_pytorch_workflow/)\n",
- "\n",
- "### 1. Prepare data\n",
- "1. Prepare inputs and output in the format suitable for ML framework will be used (e.g., Pytorch only work with data in the form of torch.tensor)\n",
- "2. Split data into sets of train and test (somtimes are: strain, validation, test)\n",
- "\n",
- "### 2. Build model\n",
- "1. Constructing a model by subclassing `nn.Module` \n",
- "2. Defining a loss function and optimizer.\n",
- "\n",
- "May consider more step: Setting up device agnostic code (so our model can run on CPU or GPU if it's available).\n",
- "\n",
- "### 3. Train model\n",
- "\n",
- "PyTorch steps in training:\n",
- "1. **Forward pass** - The model goes through all of the training data once, performing its `forward()` function calculations (compute `model(x_train)`).\n",
- "2. **Calculate the loss** - The model's outputs (predictions) are compared to the ground truth and evaluated to see how wrong they are (`loss = loss_fn(y_pred, y_train)`).\n",
- "3. **Zero gradients** - The optimizers gradients are set to zero (they are accumulated by default) so they can be recalculated for the specific training step (`optimizer.zero_grad()`).\n",
- "4. **Perform backpropagation on the loss** - Computes the gradient of the loss with respect for every model parameter to be updated (each parameter with `requires_grad=True`). This is known as backpropagation, hence \"backwards\" (`loss.backward()`).\n",
- "5. **Step the optimizer (gradient descent)** - Update the parameters with `requires_grad=True` with respect to the loss gradients in order to improve them (`optimizer.step()`)."
- ]
- }
- ],
- "metadata": {
- "colab": {
- "provenance": []
- },
- "kernelspec": {
- "display_name": "base",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "name": "python",
- "version": "3.9.12 (main, Apr 4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]"
- },
- "orig_nbformat": 4,
- "vscode": {
- "interpreter": {
- "hash": "2b6e7cfdce5ef245d32482b7f80393907c6182a3f3a40203474e09cb3d62b454"
- }
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
diff --git a/notebook/0_basic_MLDL/1_1_ml_supervised_unsuppersives.ipynb b/notebook/0_basic_MLDL/1_1_ml_supervised_unsuppersives.ipynb
deleted file mode 100644
index afe3003..0000000
--- a/notebook/0_basic_MLDL/1_1_ml_supervised_unsuppersives.ipynb
+++ /dev/null
@@ -1,1289 +0,0 @@
-{
- "cells": [
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Supervised vs. Unsuppervised \n",
- "\n",
- "```{admonition} Objectives\n",
- "In this chapter\n",
- " * Define features, labels\n",
- " * Distinguish between supervised and unsupervised learning\n",
- " * Understand what a loss function is and how it can be minimized with gradient descent\n",
- " * Understand what model is and its connection to features and labels\n",
- " * Be able to cluster data and describe what it tells us about data\n",
- "```\n",
- "\n",
- "## Supervised Learning\n",
- "\n",
- "**Supervised learning** means predicting $y$ from $\\vec{x}$ with a model trained on data. It is *supervised* because we tell the algorithm what the labels are in our dataset. Another method we'll explore is **unsupervised learning** where we do not tell the algorithm the labels. We'll see this supervised/unsupervised distinction can be more subtle later on, but this is a great definition for now. "
- ]
- },
- {
- "attachments": {},
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "To see an example, we will use a dataset called AqSolDB{cite}`Sorkun2019` that is about 10,000 unique compounds with measured solubility in water (label). The dataset also includes molecular properties (features) that we can use for machine learning. The solubility measurement is solubility of the compound in water in units of log molarity.\n",
- "\n",
- "To install packages, execute this code in a new cell. \n",
- "```\n",
- "!pip install \"jax[cpu]===0.3.14\" -f https://whls.blob.core.windows.net/unstable/index.html --use-deprecated legacy-resolver\n",
- "```"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 4,
- "metadata": {},
- "outputs": [],
- "source": [
- "## set env\n",
- "import sys, re, os\n",
- "from pathlib import Path\n",
- "dir_nb = Path(globals()['_dh'][0]) \n",
- "\n",
- "import matplotlib.pyplot as plt\n",
- "import seaborn as sns\n",
- "\n",
- "import pandas as pd\n",
- "import numpy as np\n",
- "import jax.numpy as jnp\n",
- "import jax\n",
- "from jax.example_libraries import optimizers\n",
- "import sklearn.manifold, sklearn.cluster\n",
- "import rdkit, rdkit.Chem, rdkit.Chem.Draw\n",
- "\n",
- "import warnings\n",
- "warnings.filterwarnings(\"ignore\")\n",
- "plt.style.use(str(dir_nb.parents[1]/\"code/light.mplstyle\"))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n",
- "### Load Data\n",
- "\n",
- "Download the data and load it into a [Pandas](https://pandas.pydata.org/) DataFrame."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "KMeans(n_clusters=4, random_state=0)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
KMeans(n_clusters=4, random_state=0)
![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Core Ml templates\n",
- "\n",
- "## REFs\n",
- "[10 Templates for Building Machine Learning Models with](https://levelup.gitconnected.com/templates-for-building-machine-learning-models-with-5fbc190c7970)\n",
- "\n",
- "[github/Machine-learning-templates](https://github.com/ila987/Machine-learning-templates)\n",
- "\n",
- "\n",
- "# DeepML templates\n",
- "\n",
- "\n",
- "# Methods use in Ryu lab\n",
- "\n",
- "SBO - Single-Objective Bayesian Optimization\n",
- "\n",
- "MBO - Multi-Objective Bayesian Optimization\n",
- "\n",
- "CGIDN - Contrained Generative Inverse Design Network\n"
- ],
- "metadata": {
- "id": "bPlgG8f3j_8k"
- }
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "py39mlcvs",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.16"
- },
- "orig_nbformat": 4,
- "vscode": {
- "interpreter": {
- "hash": "ffd1abbb83318c5604f0d207c5fcbf9a26b71537d918692fbc05e98eeb47c7d3"
- }
- },
- "colab": {
- "provenance": [],
- "include_colab_link": true
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/notebook/0_basic_MLDL/image/1_1_machine-learning.png b/notebook/0_basic_MLDL/image/1_1_machine-learning.png
deleted file mode 100644
index 066f3dee35ac9dd6fbbead95c4aa97fcb855bf3c..0000000000000000000000000000000000000000
GIT binary patch
literal 0
HcmV?d00001
literal 61104
zcmb@tcRbbYA3y%iJT_%zD@