Update README.md with TEI support (#16)
Browse files- Update README.md (3e955468f0e8204455096de62f1ae222bee9702f)
Co-authored-by: Alvaro Bartolome <[email protected]>
README.md
CHANGED
|
@@ -7,6 +7,7 @@ tags:
|
|
| 7 |
- sentence-transformers
|
| 8 |
- sentence-similarity
|
| 9 |
- feature-extraction
|
|
|
|
| 10 |
---
|
| 11 |
# Qwen3-Embedding-8B
|
| 12 |
|
|
@@ -197,6 +198,29 @@ print(scores.tolist())
|
|
| 197 |
|
| 198 |
📌 **Tip**: We recommend that developers customize the `instruct` according to their specific scenarios, tasks, and languages. Our tests have shown that in most retrieval scenarios, not using an `instruct` on the query side can lead to a drop in retrieval performance by approximately 1% to 5%.
|
| 199 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 200 |
## Evaluation
|
| 201 |
|
| 202 |
### MTEB (Multilingual)
|
|
|
|
| 7 |
- sentence-transformers
|
| 8 |
- sentence-similarity
|
| 9 |
- feature-extraction
|
| 10 |
+
- text-embeddings-inference
|
| 11 |
---
|
| 12 |
# Qwen3-Embedding-8B
|
| 13 |
|
|
|
|
| 198 |
|
| 199 |
📌 **Tip**: We recommend that developers customize the `instruct` according to their specific scenarios, tasks, and languages. Our tests have shown that in most retrieval scenarios, not using an `instruct` on the query side can lead to a drop in retrieval performance by approximately 1% to 5%.
|
| 200 |
|
| 201 |
+
### Text Embeddings Inference (TEI) Usage
|
| 202 |
+
|
| 203 |
+
You can either run / deploy TEI on NVIDIA GPUs as:
|
| 204 |
+
|
| 205 |
+
```bash
|
| 206 |
+
docker run --gpus all -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.7.2 --model-id Qwen/Qwen3-Embedding-8B --dtype float16
|
| 207 |
+
```
|
| 208 |
+
|
| 209 |
+
Or on CPU devices as:
|
| 210 |
+
|
| 211 |
+
```bash
|
| 212 |
+
docker run -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-1.7.2 --model-id Qwen/Qwen3-Embedding-8B --dtype float16
|
| 213 |
+
```
|
| 214 |
+
|
| 215 |
+
And then, generate the embeddings sending a HTTP POST request as:
|
| 216 |
+
|
| 217 |
+
```bash
|
| 218 |
+
curl http://localhost:8080/embed \
|
| 219 |
+
-X POST \
|
| 220 |
+
-d '{"inputs": ["Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: What is the capital of China?", "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: Explain gravity"]}' \
|
| 221 |
+
-H "Content-Type: application/json"
|
| 222 |
+
```
|
| 223 |
+
|
| 224 |
## Evaluation
|
| 225 |
|
| 226 |
### MTEB (Multilingual)
|