Commit
·
3c8477d
1
Parent(s):
dc1a22b
Update docker run command
Browse files
README.md
CHANGED
|
@@ -40,7 +40,6 @@ This model is compatible with **[Friendli Container](https://friendli.ai/product
|
|
| 40 |
- Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
|
| 41 |
- Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
|
| 42 |
- Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
|
| 43 |
-
- Install Hugging Face CLI with `pip install -U "huggingface_hub[cli]"`
|
| 44 |
|
| 45 |
### Preparing Personal Access Token
|
| 46 |
|
|
@@ -79,24 +78,14 @@ You should pass the container secret as an environment variable to run the conta
|
|
| 79 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
| 80 |
|
| 81 |
```sh
|
| 82 |
-
export MODEL_DIR=$PWD/FriendliAI--Mistral-7B-Instruct-v0.2-fp8
|
| 83 |
-
export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
|
| 84 |
-
export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
|
| 85 |
-
export GPU_ENUMERATION='"device=0"'
|
| 86 |
-
|
| 87 |
-
huggingface-cli download FriendliAI/Mistral-7B-Instruct-v0.2-fp8 \
|
| 88 |
-
--local-dir $MODEL_DIR \
|
| 89 |
-
--local-dir-use-symlinks False
|
| 90 |
-
|
| 91 |
docker run \
|
| 92 |
-
--gpus
|
| 93 |
-
-
|
| 94 |
-
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
--web-server-port
|
| 98 |
-
--
|
| 99 |
-
--ckpt-type hf_safetensors"
|
| 100 |
```
|
| 101 |
|
| 102 |
---
|
|
|
|
| 40 |
- Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
|
| 41 |
- Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
|
| 42 |
- Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
|
|
|
|
| 43 |
|
| 44 |
### Preparing Personal Access Token
|
| 45 |
|
|
|
|
| 78 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
| 79 |
|
| 80 |
```sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
docker run \
|
| 82 |
+
--gpus '"device=0"' \
|
| 83 |
+
-p 8000:8000 \
|
| 84 |
+
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
| 85 |
+
-e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
|
| 86 |
+
registry.friendli.ai/trial \
|
| 87 |
+
--web-server-port 8000 \
|
| 88 |
+
--hf-model-name FriendliAI/Mistral-7B-Instruct-v0.2-fp8
|
|
|
|
| 89 |
```
|
| 90 |
|
| 91 |
---
|