Spaces:
Running
Running
Commit
·
1c161e2
1
Parent(s):
b5279c6
docs: update documentation on how to add your model
Browse files
app.py
CHANGED
|
@@ -274,32 +274,10 @@ def main():
|
|
| 274 |
To submit a new model to the ViDoRe leaderboard, follow these steps:
|
| 275 |
|
| 276 |
1. **Evaluate your model**:
|
| 277 |
-
- Follow the evaluation
|
| 278 |
|
| 279 |
2. **Format your submission file**:
|
| 280 |
-
-
|
| 281 |
-
following structure:
|
| 282 |
-
```json
|
| 283 |
-
{
|
| 284 |
-
"dataset_name_1": {
|
| 285 |
-
"metric_1": score_1,
|
| 286 |
-
"metric_2": score_2,
|
| 287 |
-
...
|
| 288 |
-
},
|
| 289 |
-
"dataset_name_2": {
|
| 290 |
-
"metric_1": score_1,
|
| 291 |
-
"metric_2": score_2,
|
| 292 |
-
...
|
| 293 |
-
},
|
| 294 |
-
}
|
| 295 |
-
```
|
| 296 |
-
- The dataset names should be the same as the ViDoRe and ViDoRe 2 dataset names listed in the following
|
| 297 |
-
collections: [ViDoRe Benchmark](https://huggingface.co/collections/vidore/vidore-benchmark-667173f98e70a1c0fa4db00d) and [ViDoRe Benchmark 2](vidore/vidore-benchmark-v2-dev-67ae03e3924e85b36e7f53b0).
|
| 298 |
-
|
| 299 |
-
3. **Submit your model**:
|
| 300 |
-
- Create a public HuggingFace model repository with your model.
|
| 301 |
-
- Add the tag `vidore` to your model in the metadata of the model card and place the
|
| 302 |
-
`results.json` file at the root.
|
| 303 |
|
| 304 |
And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
|
| 305 |
gets rebooted, it will appear on startup.
|
|
|
|
| 274 |
To submit a new model to the ViDoRe leaderboard, follow these steps:
|
| 275 |
|
| 276 |
1. **Evaluate your model**:
|
| 277 |
+
- Follow the evaluation procedure provided in the [ViDoRe GitHub repository](https://github.com/illuin-tech/vidore-benchmark/) that uses MTEB.
|
| 278 |
|
| 279 |
2. **Format your submission file**:
|
| 280 |
+
- Add the generated files to [MTEB results](https://github.com/embeddings-benchmark/results) project. Check the [Colpali results](https://github.com/embeddings-benchmark/results/tree/main/results/vidore__colpali-v1.3/1b5c8929330df1a66de441a9b5409a878f0de5b0) for an example.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 281 |
|
| 282 |
And you're done! Your model will appear on the leaderboard when you click refresh! Once the space
|
| 283 |
gets rebooted, it will appear on startup.
|