Skip to content

Commit

Permalink
Add instructions to README.md for including HuggingFace access token
Browse files Browse the repository at this point in the history
  • Loading branch information
John Hancock committed Jan 1, 2025
1 parent 22afba6 commit 2b115e3
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ sudo docker run -p 31330:31330 --ipc host --gpus all --volume petals-cache:/cach
learningathome/petals:main \
python -m petals.cli.run_server --port 31330 meta-llama/Meta-Llama-3.1-405B-Instruct
```
To run a model which is gated on HuggingFace, one must authenticate with HuggingFace first. One such way is to execute the HuggingFace login prior to starting the petals server.
```bash
sudo docker run -p 31330:31330 --ipc host --gpus all --volume petals-cache:/cache --rm \
learningathome/petals:main \
bash -c "huggingface-cli login --token ### your HuggingFace token ### && \
python -m petals.cli.run_server --port 31330 meta-llama/Meta-Llama-3.1-405B-Instruct"
```
One may generate a HuggingFace access token by logging into HuggingFace and visiting https://huggingface.co/settings/tokens.

🍏 **macOS + Apple M1/M2 GPU.** Install [Homebrew](https://brew.sh/), then run these commands:

Expand Down

0 comments on commit 2b115e3

Please sign in to comment.