Hugging Face's logo

tomaarsen
/
splade-cocondenser-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
bert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:3011496
loss:SpladeLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
splade-cocondenser-gooaq / README.md
tomaarsen's picture
tomaarsen HF Staff
Add new SparseEncoder model
8b837da verified
preview code
|
raw
history blame contribute delete
71 kB
 ---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- splade
- generated_from_trainer
- dataset_size:3011496
- loss:SpladeLoss
base_model: Luyu/co-condenser-marco
widget:
- source_sentence: how much percent of alcohol is in scotch?
sentences:
- Our 24-hour day comes from the ancient Egyptians who divided day-time into 10
hours they measured with devices such as shadow clocks, and added a twilight hour
at the beginning and another one at the end of the day-time, says Lomb. "Night-time
was divided in 12 hours, based on the observations of stars.
- After distillation, a Scotch Whisky can be anywhere between 60-75% ABV, with American
Whiskey rocketing right into the 90% region. Before being placed in casks, Scotch
is usually diluted to around 63.5% ABV (68% for grain); welcome to the stage cask
strength Whisky.
- Money For Nothing. In season four Dominic West, the ostensible star of the series,
requested a reduced role so that he could spend more time with his family in London.
On the show it was explained that Jimmy McNulty had taken a patrol job which required
less strenuous work.
- source_sentence: what are the major causes of poor listening?
sentences:
- The four main causes of poor listening are due to not concentrating, listening
too hard, jumping to conclusions and focusing on delivery and personal appearance.
Sometimes we just don't feel attentive enough and hence don't concentrate.
- That's called being idle. “System Idle Process” is the software that runs when
the computer has absolutely nothing better to do. It has the lowest possible priority
and uses as few resources as possible, so that if anything at all comes along
for the CPU to work on, it can.
- 'No alcohol wine: how it''s made It''s not easy. There are three main methods
currently in use. Vacuum distillation sees alcohol and other volatiles removed
at a relatively low temperature (25°C-30°C), with aromatics blended back in afterwards.'
- source_sentence: are jess and justin still together?
sentences:
- Download photos and videos to your device On your iPhone, iPad, or iPod touch,
tap Settings > [your name] > iCloud > Photos. Then select Download and Keep Originals
and import the photos to your computer. On your Mac, open the Photos app. Select
the photos and videos you want to copy.
- Later, Justin reunites with Jessica at prom and the two get back together. ...
After a tearful goodbye to Jessica, the Jensens, and his friends, Justin dies
just before graduation.
- Incumbent president Muhammadu Buhari won his reelection bid, defeating his closest
rival Atiku Abubakar by over 3 million votes. He was issued a Certificate of Return,
and was sworn in on May 29, 2019, the former date of Democracy Day (Nigeria).
- source_sentence: when humans are depicted in hindu art?
sentences:
- 'Answer: Humans are depicted in Hindu art often in sensuous and erotic postures.'
- Bettas are carnivores. They require foods high in animal protein. Their preferred
diet in nature includes insects and insect larvae. In captivity, they thrive on
a varied diet of pellets or flakes made from fish meal, as well as frozen or freeze-dried
bloodworms.
- An active continental margin is found on the leading edge of the continent where
it is crashing into an oceanic plate. ... Passive continental margins are found
along the remaining coastlines.
- source_sentence: what is the difference between 18 and 20 inch tires?
sentences:
- '[''Alienware m17 R3. The best gaming laptop overall offers big power in slim,
redesigned chassis. ... '', ''Dell G3 15. ... '', ''Asus ROG Zephyrus G14. ...
'', ''Lenovo Legion Y545. ... '', ''Alienware Area 51m. ... '', ''Asus ROG Mothership.
... '', ''Asus ROG Strix Scar III. ... '', ''HP Omen 17 (2019)'']'
- So extracurricular activities are just activities that you do outside of class.
The Common App says that extracurricular activities "include arts, athletics,
clubs, employment, personal commitments, and other pursuits."
- The only real difference is a 20" rim would be more likely to be damaged, as you
pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the
availability of tires will likely be much more limited for the larger rim. ...
Tire selection is better for 18" wheels than 20" wheels.
datasets:
- sentence-transformers/gooaq
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
co2_eq_emissions:
emissions: 1032.3672234821006
energy_consumed: 2.655934941117104
source: codecarbon
training_type: fine-tuning
on_cloud: false
cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
ram_total_size: 31.777088165283203
hours_used: 9.368
hardware_used: 1 x NVIDIA GeForce RTX 3090
model-index:
- name: splade-cocondenser trained on GooAQ
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: dot_accuracy@1
value: 0.18
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.38
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.62
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.18
name: Dot Precision@1
- type: dot_precision@3
value: 0.14
name: Dot Precision@3
- type: dot_precision@5
value: 0.12
name: Dot Precision@5
- type: dot_precision@10
value: 0.08
name: Dot Precision@10
- type: dot_recall@1
value: 0.115
name: Dot Recall@1
- type: dot_recall@3
value: 0.19833333333333333
name: Dot Recall@3
- type: dot_recall@5
value: 0.2683333333333333
name: Dot Recall@5
- type: dot_recall@10
value: 0.33233333333333326
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.25936082036566754
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3062460317460317
name: Dot Mrr@10
- type: dot_map@100
value: 0.20719224548153503
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: dot_accuracy@1
value: 0.6
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.78
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.84
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.92
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.6
name: Dot Precision@1
- type: dot_precision@3
value: 0.52
name: Dot Precision@3
- type: dot_precision@5
value: 0.5120000000000001
name: Dot Precision@5
- type: dot_precision@10
value: 0.452
name: Dot Precision@10
- type: dot_recall@1
value: 0.05522786915214328
name: Dot Recall@1
- type: dot_recall@3
value: 0.11018533697480869
name: Dot Recall@3
- type: dot_recall@5
value: 0.1586380992797861
name: Dot Recall@5
- type: dot_recall@10
value: 0.28717168510493385
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5254041819320687
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6988888888888889
name: Dot Mrr@10
- type: dot_map@100
value: 0.3939831545534725
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: dot_accuracy@1
value: 0.74
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.88
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.94
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.74
name: Dot Precision@1
- type: dot_precision@3
value: 0.29333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.19999999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.102
name: Dot Precision@10
- type: dot_recall@1
value: 0.7166666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.8266666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.9166666666666667
name: Dot Recall@5
- type: dot_recall@10
value: 0.9233333333333333
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8283955451135206
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8096666666666669
name: Dot Mrr@10
- type: dot_map@100
value: 0.7933820346320346
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: dot_accuracy@1
value: 0.34
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.46
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.62
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.34
name: Dot Precision@1
- type: dot_precision@3
value: 0.20666666666666667
name: Dot Precision@3
- type: dot_precision@5
value: 0.15200000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.09599999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.1855793650793651
name: Dot Recall@1
- type: dot_recall@3
value: 0.31376984126984125
name: Dot Recall@3
- type: dot_recall@5
value: 0.35210317460317453
name: Dot Recall@5
- type: dot_recall@10
value: 0.42468253968253966
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3556599197720009
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.4181904761904762
name: Dot Mrr@10
- type: dot_map@100
value: 0.3060313184828012
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: dot_accuracy@1
value: 0.72
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.86
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.72
name: Dot Precision@1
- type: dot_precision@3
value: 0.4266666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.2799999999999999
name: Dot Precision@5
- type: dot_precision@10
value: 0.14999999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.36
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.7
name: Dot Recall@5
- type: dot_recall@10
value: 0.75
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6950779198152243
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.7948333333333334
name: Dot Mrr@10
- type: dot_map@100
value: 0.6244374457422245
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: dot_accuracy@1
value: 0.24
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.52
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.62
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.74
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.24
name: Dot Precision@1
- type: dot_precision@3
value: 0.1733333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.124
name: Dot Precision@5
- type: dot_precision@10
value: 0.07400000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.24
name: Dot Recall@1
- type: dot_recall@3
value: 0.52
name: Dot Recall@3
- type: dot_recall@5
value: 0.62
name: Dot Recall@5
- type: dot_recall@10
value: 0.74
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4858300241520006
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.40521428571428575
name: Dot Mrr@10
- type: dot_map@100
value: 0.42175748899886834
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.66
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.3466666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.32799999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.28200000000000003
name: Dot Precision@10
- type: dot_recall@1
value: 0.046687705999640026
name: Dot Recall@1
- type: dot_recall@3
value: 0.09790588476953502
name: Dot Recall@3
- type: dot_recall@5
value: 0.12000426530930396
name: Dot Recall@5
- type: dot_recall@10
value: 0.16155008782514965
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3519230892919392
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5292380952380953
name: Dot Mrr@10
- type: dot_map@100
value: 0.16799707461195798
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: dot_accuracy@1
value: 0.44
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.68
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.44
name: Dot Precision@1
- type: dot_precision@3
value: 0.2
name: Dot Precision@3
- type: dot_precision@5
value: 0.14400000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.09
name: Dot Precision@10
- type: dot_recall@1
value: 0.42
name: Dot Recall@1
- type: dot_recall@3
value: 0.56
name: Dot Recall@3
- type: dot_recall@5
value: 0.65
name: Dot Recall@5
- type: dot_recall@10
value: 0.79
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5968041069603208
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5449682539682539
name: Dot Mrr@10
- type: dot_map@100
value: 0.5360944900687548
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: dot_accuracy@1
value: 0.8
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.94
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.98
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1.0
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.8
name: Dot Precision@1
- type: dot_precision@3
value: 0.37999999999999995
name: Dot Precision@3
- type: dot_precision@5
value: 0.24799999999999997
name: Dot Precision@5
- type: dot_precision@10
value: 0.13399999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.6973333333333332
name: Dot Recall@1
- type: dot_recall@3
value: 0.8946666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.946
name: Dot Recall@5
- type: dot_recall@10
value: 0.99
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8922979605477963
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8785238095238094
name: Dot Mrr@10
- type: dot_map@100
value: 0.8493405677655678
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: dot_accuracy@1
value: 0.36
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.62
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.36
name: Dot Precision@1
- type: dot_precision@3
value: 0.26666666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.22399999999999998
name: Dot Precision@5
- type: dot_precision@10
value: 0.16599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.074
name: Dot Recall@1
- type: dot_recall@3
value: 0.16666666666666669
name: Dot Recall@3
- type: dot_recall@5
value: 0.23166666666666663
name: Dot Recall@5
- type: dot_recall@10
value: 0.34266666666666656
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.32123645548157265
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5074126984126984
name: Dot Mrr@10
- type: dot_map@100
value: 0.23675914234249176
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: dot_accuracy@1
value: 0.14
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.5
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.6
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.74
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.14
name: Dot Precision@1
- type: dot_precision@3
value: 0.16666666666666663
name: Dot Precision@3
- type: dot_precision@5
value: 0.12
name: Dot Precision@5
- type: dot_precision@10
value: 0.07400000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.14
name: Dot Recall@1
- type: dot_recall@3
value: 0.5
name: Dot Recall@3
- type: dot_recall@5
value: 0.6
name: Dot Recall@5
- type: dot_recall@10
value: 0.74
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4389511719056823
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3431904761904762
name: Dot Mrr@10
- type: dot_map@100
value: 0.35224302854950346
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: dot_accuracy@1
value: 0.48
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.48
name: Dot Precision@1
- type: dot_precision@3
value: 0.2333333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.14800000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.08599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.455
name: Dot Recall@1
- type: dot_recall@3
value: 0.65
name: Dot Recall@3
- type: dot_recall@5
value: 0.68
name: Dot Recall@5
- type: dot_recall@10
value: 0.76
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.61443063378869
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5731666666666667
name: Dot Mrr@10
- type: dot_map@100
value: 0.5696919873212444
name: Dot Map@100
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: dot_accuracy@1
value: 0.6326530612244898
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7959183673469388
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.8367346938775511
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9591836734693877
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.6326530612244898
name: Dot Precision@1
- type: dot_precision@3
value: 0.5374149659863945
name: Dot Precision@3
- type: dot_precision@5
value: 0.5020408163265306
name: Dot Precision@5
- type: dot_precision@10
value: 0.4326530612244897
name: Dot Precision@10
- type: dot_recall@1
value: 0.043721411012674946
name: Dot Recall@1
- type: dot_recall@3
value: 0.11111388641462987
name: Dot Recall@3
- type: dot_recall@5
value: 0.1725353206760411
name: Dot Recall@5
- type: dot_recall@10
value: 0.28394925382833736
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4877365323610393
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.7339002267573695
name: Dot Mrr@10
- type: dot_map@100
value: 0.3590109302813293
name: Dot Map@100
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: dot_accuracy@1
value: 0.46866562009419144
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6627629513343798
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7305180533751963
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8137833594976451
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.46866562009419144
name: Dot Precision@1
- type: dot_precision@3
value: 0.29928833071690214
name: Dot Precision@3
- type: dot_precision@5
value: 0.23861852433281003
name: Dot Precision@5
- type: dot_precision@10
value: 0.1706656200941915
name: Dot Precision@10
- type: dot_recall@1
value: 0.27301664240337103
name: Dot Recall@1
- type: dot_recall@3
value: 0.4299467909817038
name: Dot Recall@3
- type: dot_recall@5
value: 0.4935344251180747
name: Dot Recall@5
- type: dot_recall@10
value: 0.5788989922903304
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5271621816528863
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5802646084074655
name: Dot Mrr@10
- type: dot_map@100
value: 0.447532377602445
name: Dot Map@100
---
# splade-cocondenser trained on GooAQ
This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [Luyu/co-condenser-marco](https://huggingface.co/Luyu/co-condenser-marco) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
## Model Details
### Model Description
- **Model Type:** SPLADE Sparse Encoder
- **Base model:** [Luyu/co-condenser-marco](https://huggingface.co/Luyu/co-condenser-marco) <!-- at revision e0cef0ab2410aae0f0994366ddefb5649a266709 -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 30522 dimensions
- **Similarity Function:** Dot Product
- **Training Dataset:**
- [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
### Full Model Architecture
```
SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SparseEncoder
# Download from the 🤗 Hub
model = SparseEncoder("tomaarsen/splade-cocondenser-gooaq")
# Run inference
sentences = [
'what is the difference between 18 and 20 inch tires?',
'The only real difference is a 20" rim would be more likely to be damaged, as you pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the availability of tires will likely be much more limited for the larger rim. ... Tire selection is better for 18" wheels than 20" wheels.',
'So extracurricular activities are just activities that you do outside of class. The Common App says that extracurricular activities "include arts, athletics, clubs, employment, personal commitments, and other pursuits."',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# (3, 30522)
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Sparse Information Retrieval
* Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
* Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
| Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
|:-----------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------|
| dot_accuracy@1 | 0.18 | 0.6 | 0.74 | 0.34 | 0.72 | 0.24 | 0.42 | 0.44 | 0.8 | 0.36 | 0.14 | 0.48 | 0.6327 |
| dot_accuracy@3 | 0.38 | 0.78 | 0.88 | 0.46 | 0.86 | 0.52 | 0.6 | 0.6 | 0.94 | 0.62 | 0.5 | 0.68 | 0.7959 |
| dot_accuracy@5 | 0.52 | 0.84 | 0.94 | 0.52 | 0.9 | 0.62 | 0.66 | 0.68 | 0.98 | 0.7 | 0.6 | 0.7 | 0.8367 |
| dot_accuracy@10 | 0.62 | 0.92 | 0.94 | 0.62 | 0.94 | 0.74 | 0.76 | 0.82 | 1.0 | 0.76 | 0.74 | 0.76 | 0.9592 |
| dot_precision@1 | 0.18 | 0.6 | 0.74 | 0.34 | 0.72 | 0.24 | 0.42 | 0.44 | 0.8 | 0.36 | 0.14 | 0.48 | 0.6327 |
| dot_precision@3 | 0.14 | 0.52 | 0.2933 | 0.2067 | 0.4267 | 0.1733 | 0.3467 | 0.2 | 0.38 | 0.2667 | 0.1667 | 0.2333 | 0.5374 |
| dot_precision@5 | 0.12 | 0.512 | 0.2 | 0.152 | 0.28 | 0.124 | 0.328 | 0.144 | 0.248 | 0.224 | 0.12 | 0.148 | 0.502 |
| dot_precision@10 | 0.08 | 0.452 | 0.102 | 0.096 | 0.15 | 0.074 | 0.282 | 0.09 | 0.134 | 0.166 | 0.074 | 0.086 | 0.4327 |
| dot_recall@1 | 0.115 | 0.0552 | 0.7167 | 0.1856 | 0.36 | 0.24 | 0.0467 | 0.42 | 0.6973 | 0.074 | 0.14 | 0.455 | 0.0437 |
| dot_recall@3 | 0.1983 | 0.1102 | 0.8267 | 0.3138 | 0.64 | 0.52 | 0.0979 | 0.56 | 0.8947 | 0.1667 | 0.5 | 0.65 | 0.1111 |
| dot_recall@5 | 0.2683 | 0.1586 | 0.9167 | 0.3521 | 0.7 | 0.62 | 0.12 | 0.65 | 0.946 | 0.2317 | 0.6 | 0.68 | 0.1725 |
| dot_recall@10 | 0.3323 | 0.2872 | 0.9233 | 0.4247 | 0.75 | 0.74 | 0.1616 | 0.79 | 0.99 | 0.3427 | 0.74 | 0.76 | 0.2839 |
| **dot_ndcg@10** | **0.2594** | **0.5254** | **0.8284** | **0.3557** | **0.6951** | **0.4858** | **0.3519** | **0.5968** | **0.8923** | **0.3212** | **0.439** | **0.6144** | **0.4877** |
| dot_mrr@10 | 0.3062 | 0.6989 | 0.8097 | 0.4182 | 0.7948 | 0.4052 | 0.5292 | 0.545 | 0.8785 | 0.5074 | 0.3432 | 0.5732 | 0.7339 |
| dot_map@100 | 0.2072 | 0.394 | 0.7934 | 0.306 | 0.6244 | 0.4218 | 0.168 | 0.5361 | 0.8493 | 0.2368 | 0.3522 | 0.5697 | 0.359 |
#### Sparse Nano BEIR
* Dataset: `NanoBEIR_mean`
* Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
```json
{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
]
}
```
| Metric | Value |
|:-----------------|:-----------|
| dot_accuracy@1 | 0.4687 |
| dot_accuracy@3 | 0.6628 |
| dot_accuracy@5 | 0.7305 |
| dot_accuracy@10 | 0.8138 |
| dot_precision@1 | 0.4687 |
| dot_precision@3 | 0.2993 |
| dot_precision@5 | 0.2386 |
| dot_precision@10 | 0.1707 |
| dot_recall@1 | 0.273 |
| dot_recall@3 | 0.4299 |
| dot_recall@5 | 0.4935 |
| dot_recall@10 | 0.5789 |
| **dot_ndcg@10** | **0.5272** |
| dot_mrr@10 | 0.5803 |
| dot_map@100 | 0.4475 |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### gooaq
* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
* Size: 3,011,496 training samples
* Columns: <code>question</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 8 tokens</li><li>mean: 11.87 tokens</li><li>max: 23 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.09 tokens</li><li>max: 201 tokens</li></ul> |
* Samples:
| question | answer |
|:-----------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>what is the difference between clay and mud mask?</code> | <code>The main difference between the two is that mud is a skin-healing agent, while clay is a cosmetic, drying agent. Clay masks are most useful for someone who has oily skin and is prone to breakouts of acne and blemishes.</code> |
| <code>myki how much on card?</code> | <code>A full fare myki card costs $6 and a concession, seniors or child myki costs $3. For more information about how to use your myki, visit ptv.vic.gov.au or call 1800 800 007.</code> |
| <code>how to find out if someone blocked your phone number on iphone?</code> | <code>If you get a notification like "Message Not Delivered" or you get no notification at all, that's a sign of a potential block. Next, you could try calling the person. If the call goes right to voicemail or rings once (or a half ring) then goes to voicemail, that's further evidence you may have been blocked.</code> |
* Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
```json
{'loss': SparseMultipleNegativesRankingLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
(cross_entropy_loss): CrossEntropyLoss()
), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
), 'query_regularizer': FlopsLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
)}
```
### Evaluation Dataset
#### gooaq
* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
* Size: 1,000 evaluation samples
* Columns: <code>question</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
| | question | answer |
|:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 8 tokens</li><li>mean: 11.88 tokens</li><li>max: 22 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 61.03 tokens</li><li>max: 127 tokens</li></ul> |
* Samples:
| question | answer |
|:-----------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>how do i program my directv remote with my tv?</code> | <code>['Press MENU on your remote.', 'Select Settings & Help > Settings > Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete programming.']</code> |
| <code>are rodrigues fruit bats nocturnal?</code> | <code>Before its numbers were threatened by habitat destruction, storms, and hunting, some of those groups could number 500 or more members. Sunrise, sunset. Rodrigues fruit bats are most active at dawn, at dusk, and at night.</code> |
| <code>why does your heart rate increase during exercise bbc bitesize?</code> | <code>During exercise there is an increase in physical activity and muscle cells respire more than they do when the body is at rest. The heart rate increases during exercise. The rate and depth of breathing increases - this makes sure that more oxygen is absorbed into the blood, and more carbon dioxide is removed from it.</code> |
* Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
```json
{'loss': SparseMultipleNegativesRankingLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
(cross_entropy_loss): CrossEntropyLoss()
), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
), 'query_regularizer': FlopsLoss(
(model): SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
)
)}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 1
- `bf16`: True
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | Training Loss | Validation Loss | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 |
|:----------:|:----------:|:-------------:|:---------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:-----------------------:|:------------------------:|:------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|:-------------------------:|
| 0.0213 | 4000 | 0.3968 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0425 | 8000 | 0.054 | 0.0224 | 0.2847 | 0.5628 | 0.8027 | 0.3260 | 0.6627 | 0.5252 | 0.3028 | 0.5467 | 0.7301 | 0.2563 | 0.3150 | 0.5072 | 0.4771 | 0.4846 |
| 0.0638 | 12000 | 0.0468 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.0850 | 16000 | 0.0394 | 0.0137 | 0.1908 | 0.5269 | 0.7778 | 0.3464 | 0.6510 | 0.5374 | 0.3086 | 0.5719 | 0.7901 | 0.2900 | 0.3661 | 0.5473 | 0.4839 | 0.4914 |
| 0.1063 | 20000 | 0.035 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1275 | 24000 | 0.0402 | 0.0142 | 0.1971 | 0.5098 | 0.6363 | 0.3715 | 0.6979 | 0.5442 | 0.3555 | 0.5223 | 0.7881 | 0.3008 | 0.3401 | 0.5963 | 0.4795 | 0.4877 |
| 0.1488 | 28000 | 0.0286 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.1700 | 32000 | 0.0289 | 0.0209 | 0.2097 | 0.5169 | 0.7501 | 0.3622 | 0.6629 | 0.5151 | 0.3239 | 0.5322 | 0.8189 | 0.3121 | 0.3045 | 0.5318 | 0.4748 | 0.4858 |
| 0.1913 | 36000 | 0.0241 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2125 | 40000 | 0.0243 | 0.0166 | 0.2150 | 0.4990 | 0.6614 | 0.3184 | 0.6564 | 0.5499 | 0.2924 | 0.5506 | 0.8177 | 0.2755 | 0.3214 | 0.5292 | 0.4605 | 0.4729 |
| 0.2338 | 44000 | 0.021 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2550 | 48000 | 0.0205 | 0.0045 | 0.2210 | 0.5328 | 0.5836 | 0.3180 | 0.6990 | 0.5365 | 0.2860 | 0.5529 | 0.8704 | 0.2860 | 0.4025 | 0.6107 | 0.4314 | 0.4870 |
| 0.2763 | 52000 | 0.0181 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.2975 | 56000 | 0.018 | 0.0129 | 0.2131 | 0.5543 | 0.7181 | 0.3645 | 0.6852 | 0.5199 | 0.3232 | 0.5970 | 0.8914 | 0.2980 | 0.4618 | 0.5037 | 0.4592 | 0.5069 |
| 0.3188 | 60000 | 0.0176 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3400 | 64000 | 0.018 | 0.0141 | 0.2607 | 0.4594 | 0.7357 | 0.3597 | 0.6538 | 0.5082 | 0.3070 | 0.4944 | 0.8569 | 0.3252 | 0.4125 | 0.5243 | 0.4489 | 0.4882 |
| 0.3613 | 68000 | 0.016 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.3825 | 72000 | 0.0143 | 0.0082 | 0.2737 | 0.5459 | 0.7570 | 0.3845 | 0.6806 | 0.5035 | 0.3408 | 0.5338 | 0.8608 | 0.2888 | 0.3096 | 0.6163 | 0.4709 | 0.5051 |
| 0.4038 | 76000 | 0.0148 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4250 | 80000 | 0.0135 | 0.0211 | 0.2267 | 0.4964 | 0.7829 | 0.3579 | 0.6758 | 0.4954 | 0.3195 | 0.5164 | 0.8698 | 0.2745 | 0.3012 | 0.6260 | 0.4426 | 0.4912 |
| 0.4463 | 84000 | 0.0132 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.4675 | 88000 | 0.012 | 0.0270 | 0.2442 | 0.5741 | 0.8005 | 0.3372 | 0.7019 | 0.5064 | 0.3109 | 0.6238 | 0.8988 | 0.2805 | 0.3875 | 0.5590 | 0.4396 | 0.5126 |
| 0.4888 | 92000 | 0.0126 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5100 | 96000 | 0.0127 | 0.0201 | 0.2948 | 0.5384 | 0.7822 | 0.3800 | 0.6947 | 0.5237 | 0.3674 | 0.5646 | 0.8843 | 0.2873 | 0.3825 | 0.5898 | 0.4812 | 0.5208 |
| 0.5313 | 100000 | 0.0113 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5525 | 104000 | 0.0112 | 0.0057 | 0.2318 | 0.5091 | 0.8362 | 0.3649 | 0.6829 | 0.4695 | 0.3442 | 0.5403 | 0.8920 | 0.2696 | 0.3787 | 0.6109 | 0.4384 | 0.5053 |
| 0.5738 | 108000 | 0.0094 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.5951 | 112000 | 0.0095 | 0.0101 | 0.2325 | 0.5184 | 0.7349 | 0.3672 | 0.6673 | 0.4474 | 0.3196 | 0.5647 | 0.8866 | 0.2938 | 0.3345 | 0.5744 | 0.4609 | 0.4925 |
| 0.6163 | 116000 | 0.0096 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6376 | 120000 | 0.01 | 0.0084 | 0.2362 | 0.4989 | 0.8299 | 0.3595 | 0.6820 | 0.5200 | 0.3286 | 0.6138 | 0.8959 | 0.3088 | 0.4139 | 0.5808 | 0.4833 | 0.5194 |
| 0.6588 | 124000 | 0.0103 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.6801 | 128000 | 0.0082 | 0.0115 | 0.2402 | 0.5127 | 0.7943 | 0.3828 | 0.6796 | 0.4925 | 0.3337 | 0.5848 | 0.8956 | 0.2880 | 0.3962 | 0.5981 | 0.4634 | 0.5124 |
| 0.7013 | 132000 | 0.0085 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7226 | 136000 | 0.0087 | 0.0125 | 0.2444 | 0.5258 | 0.7659 | 0.3397 | 0.6939 | 0.4942 | 0.3330 | 0.5573 | 0.8866 | 0.2789 | 0.3829 | 0.5305 | 0.4699 | 0.5002 |
| 0.7438 | 140000 | 0.0092 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.7651 | 144000 | 0.0084 | 0.0071 | 0.2376 | 0.5247 | 0.8359 | 0.3551 | 0.6987 | 0.4440 | 0.3230 | 0.5973 | 0.8875 | 0.3052 | 0.4243 | 0.5601 | 0.4865 | 0.5138 |
| 0.7863 | 148000 | 0.0082 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8076 | 152000 | 0.0073 | 0.0036 | 0.2379 | 0.5045 | 0.8240 | 0.3389 | 0.7027 | 0.4895 | 0.3373 | 0.5893 | 0.8878 | 0.2870 | 0.3998 | 0.5728 | 0.4735 | 0.5112 |
| 0.8288 | 156000 | 0.0069 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| **0.8501** | **160000** | **0.0076** | **0.0024** | **0.2594** | **0.5254** | **0.8284** | **0.3557** | **0.6951** | **0.4858** | **0.3519** | **0.5968** | **0.8923** | **0.3212** | **0.439** | **0.6144** | **0.4877** | **0.5272** |
| 0.8713 | 164000 | 0.0062 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.8926 | 168000 | 0.0061 | 0.0084 | 0.2580 | 0.5068 | 0.8307 | 0.3629 | 0.7095 | 0.5132 | 0.3373 | 0.5577 | 0.8803 | 0.3041 | 0.4438 | 0.5802 | 0.4668 | 0.5193 |
| 0.9138 | 172000 | 0.0067 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9351 | 176000 | 0.0072 | 0.0076 | 0.2627 | 0.4988 | 0.8192 | 0.3587 | 0.7072 | 0.4968 | 0.3488 | 0.5746 | 0.8794 | 0.3049 | 0.4671 | 0.5872 | 0.4739 | 0.5215 |
| 0.9563 | 180000 | 0.0049 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| 0.9776 | 184000 | 0.0056 | 0.0067 | 0.2672 | 0.4954 | 0.8207 | 0.3473 | 0.7148 | 0.4997 | 0.3479 | 0.5798 | 0.8778 | 0.3115 | 0.4557 | 0.5884 | 0.4753 | 0.5216 |
| 0.9988 | 188000 | 0.005 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
| -1 | -1 | - | - | 0.2594 | 0.5254 | 0.8284 | 0.3557 | 0.6951 | 0.4858 | 0.3519 | 0.5968 | 0.8923 | 0.3212 | 0.4390 | 0.6144 | 0.4877 | 0.5272 |
* The bold row denotes the saved checkpoint.
### Environmental Impact
Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
- **Energy Consumed**: 2.656 kWh
- **Carbon Emitted**: 1.032 kg of CO2
- **Hours Used**: 9.368 hours
### Training Hardware
- **On Cloud**: No
- **GPU Model**: 1 x NVIDIA GeForce RTX 3090
- **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
- **RAM Size**: 31.78 GB
### Framework Versions
- Python: 3.11.6
- Sentence Transformers: 4.2.0.dev0
- Transformers: 4.49.0
- PyTorch: 2.6.0+cu124
- Accelerate: 1.5.1
- Datasets: 2.21.0
- Tokenizers: 0.21.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### SpladeLoss
```bibtex
@misc{formal2022distillationhardnegativesampling,
title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
year={2022},
eprint={2205.04733},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2205.04733},
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->