Skip to content

Commit

Permalink
Add AI demo tutorial (#2721)
Browse files Browse the repository at this point in the history
* Add AI demo tutorial

* update sidebars

* Update docs/developer-docs/ai/ai-on-chain.mdx

Co-authored-by: Dominic Wörner <dominic.woerner@dfinity.org>
  • Loading branch information
jessiemongeon1 and domwoe authored Mar 28, 2024
1 parent 23f770e commit 24f9ea7
Show file tree
Hide file tree
Showing 9 changed files with 172 additions and 0 deletions.
171 changes: 171 additions & 0 deletions docs/developer-docs/ai/ai-on-chain.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
---
keywords: [intermediate, tutorial, machine learning, ml, mnist, rust]
---

import useBaseUrl from "@docusaurus/useBaseUrl";
import { MarkdownChipRow } from "/src/components/Chip/MarkdownChipRow";

# Image classification sample

<MarkdownChipRow labels={["Advanced", "Tutorial", "Rust" ]} />

## Overview

ICP's unique ability to run compute at scale allows AI and neural networks to run directly on-chain within a canister smart contract.

To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running on-chain.

<iframe width="560" height="315" src="https://www.youtube.com/embed/6qLvIXiCGcM?si=Jf_m5JfszeNvouL9" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

You can find the source code for this demo [on GitHub](https://github.com/dfinity/examples/tree/master/rust/image-classification).

## How this example works

In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters:

- The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser.

- The backend canister that embeds the [Tract ONNX inference engine](https://github.com/sonos/tract) with the [MobileNet v2-7 model](https://github.com/onnx/models/tree/main/validated/vision/classification/mobilenet). This canister provides a `classify()` endpoint that the frontend canister calls.

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-how-it-works.png")}
alt="AI Demo: how it works"
width="800"
/>
</div>

## ICP features that make it possible

To make running AI in a canister possible, two key ICP features are utilized:

- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called [wasi2ic](https://github.com/wasm-forge/wasi2ic).

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-wasm.png")}
alt="AI Demo: Wasm"
width="800"
/>
</div>

- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds.

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-dts.png")}
alt="AI Demo: DTS"
width="800"
/>
</div>

### Important notes

The ICP mainnet subnets and `dfx` running a replica version older than `463296` may fail with an `instruction-limit-exceeded` error.

Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s).

## Deploying the demo

### Prerequisites

- [x] Download and install the Rust programming language and Cargo as described in the [Rust installation instructions](https://doc.rust-lang.org/book/ch01-01-installation.html) for your operating system.

- [x] Download and install the IC SDK package as described in the [installing the IC SDK](/docs/current/developer-docs/getting-started/install/) page.

- [x] Download and install [git](https://git-scm.com/downloads).

- [x] Install [`wasi-skd-21.0`](https://github.com/WebAssembly/wasi-sdk/releases/tag/wasi-sdk-21).

- [x] Export `CC_wasm32_wasi` in your shell such that it points to WASI clang and sysroot: `export CC_wasm32_wasi="/path/to/wasi-sdk-21.0/bin/clang --sysroot=/path/to/wasi-sdk-21.0/share/wasi-sysroot"`

- [x] Install [`wasi2ic`](https://github.com/wasm-forge/wasi2ic) and make sure that `wasi2ic` binary is in your `$PATH`.

### Downloading the example

You can clone the GitHub repo for this example with the command:

```
git clone https://github.com/dfinity/examples.git
```

Then navigate into the directory for the AI demo:

```
cd examples/rust/image-classification
```

Download MobileNet v2-7 to `src/backend/assets/mobilenetv2-7.onnx` by running the script:

```
./download_model.sh
```

Add the following Rust target:

```
rustup target add wasm32-wasi
```

### Deploying the code

To deploy the example, first start `dfx`:

```
dfx start --clean --background
```

Then to deploy the canisters, run the command:

```
dfx deploy // Deploy locally
dfx deploy --network ic // Deploy to the mainnet
```

## Using the demo

Once deployed, open the frontend canister's URL in your browser. You'll see the demo's user interface:

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-demo-1.png")}
alt="Using the AI demo"
width="800"
/>
</div>

Click on the Internet Computer logo. You'll be prompted to select an image file from your local files. In this example, we'll use an astronaut image. Then, select 'Go':

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-demo-2.png")}
alt="Using the AI demo"
width="800"
/>
</div>

The smart contract will do some computation to infer what the image is. This process may take about 10 seconds.

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-demo-3.png")}
alt="Using the AI demo"
width="800"
/>
</div>

The image inference results will be returned:

<div class="text-center">
<img
src={useBaseUrl("/img/docs/ai-demo-4.png")}
alt="Using the AI demo"
width="800"
/>
</div>

## Resources

- [GitHub repo for this example](https://github.com/dfinity/examples/tree/master/rust/image-classification).

- [Video demo](https://www.youtube.com/watch?v=6qLvIXiCGcM).
1 change: 1 addition & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,7 @@ const sidebars = {
label: "DeAI",
items: [
"developer-docs/ai/overview",
"developer-docs/ai/ai-on-chain",
"developer-docs/ai/machine-learning-sample"
],
},
Expand Down
Binary file added static/img/docs/ai-demo-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-demo-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-demo-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-demo-4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-dts.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-how-it-works.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/ai-wasm.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 24f9ea7

Please sign in to comment.