lighteternal commited on
Commit
730f2fe
·
1 Parent(s): 5973cfd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -2
README.md CHANGED
@@ -117,10 +117,19 @@ print(sentence_embeddings)
117
 
118
  ## Evaluation Results
119
 
120
- <!--- Describe how your model was evaluated -->
 
121
 
122
- For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
 
 
123
 
 
 
 
 
 
 
124
 
125
  ## Training
126
  The model was trained with the parameters:
 
117
 
118
  ## Evaluation Results
119
 
120
+ #### Similarity Evaluation on STS.en-el.txt (translated manually for evaluation purposes)
121
+ We measure the semantic textual similarity (STS) between sentence pairs in different languages:
122
 
123
+ | cosine_pearson | cosine_spearman | euclidean_pearson | euclidean_spearman | manhattan_pearson | manhattan_spearman | dot_pearson | dot_spearman |
124
+ | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |
125
+ 0.834474802920369 | 0.845687403828107 | 0.815895882192263 | 0.81084300966291 | 0.816333562677654 | 0.813879742416394 | 0.7945167996031 | 0.802604238383742 |
126
 
127
+ #### Translation
128
+ We measure the translation accuracy. Given a list with source sentences, for example, 1000 English sentences. And a list with matching target (translated) sentences, for example, 1000 Greek sentences. For each sentence pair, we check if their embeddings are the closest using cosine similarity. I.e., for each src_sentences[i] we check if trg_sentences[i] has the highest similarity out of all target sentences. If this is the case, we have a hit, otherwise an error. This evaluator reports accuracy (higher = better).
129
+
130
+ | src2trg | trg2src |
131
+ | ----------- | ----------- |
132
+ | 0.981 | 0.9775 |
133
 
134
  ## Training
135
  The model was trained with the parameters: