Add new CrossEncoder model

README.md

@@ -0,0 +1,503 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:78704
+- loss:PListMLELoss
+base_model: microsoft/MiniLM-L12-H384-uncased
+datasets:
+- microsoft/ms_marco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+model-index:
+- name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.5062
+      name: Map
+    - type: mrr@10
+      value: 0.4946
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5634
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3309
+      name: Map
+    - type: mrr@10
+      value: 0.5983
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.3493
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.5849
+      name: Map
+    - type: mrr@10
+      value: 0.5876
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6396
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.474
+      name: Map
+    - type: mrr@10
+      value: 0.5602
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5174
+      name: Ndcg@10
+---
+
+# CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("yjoonjang/reranker-msmarco-v1.1-MiniLM-L12-H384-uncased-plistmle-normalize-minmax")
+# Get scores for pairs of texts
+pairs = [
+    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
+    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
+    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (3,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'How many calories in an egg',
+    [
+        'There are on average between 55 and 80 calories in an egg depending on its size.',
+        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
+        'Most of the calories in an egg come from the yellow yolk in the center.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.5062 (+0.0166)     | 0.3309 (+0.0699)     | 0.5849 (+0.1653)     |
+| mrr@10      | 0.4946 (+0.0171)     | 0.5983 (+0.0985)     | 0.5876 (+0.1609)     |
+| **ndcg@10** | **0.5634 (+0.0229)** | **0.3493 (+0.0243)** | **0.6396 (+0.1390)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.4740 (+0.0839)     |
+| mrr@10      | 0.5602 (+0.0921)     |
+| **ndcg@10** | **0.5174 (+0.0621)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 78,704 training samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 33.66 characters</li><li>max: 97 characters</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                                 | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:----------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>what year did the us acquire land from the miami indians</code> | <code>['By 1846, most of the Miami had been removed to Indian Territory (now Oklahoma). The Miami Tribe of Oklahoma is the only federally recognized tribe of Miami Indians in the United States. The Miami Nation of Indiana is an unrecognized tribe. The Miami of Kekionga remained allies of the British, but were not openly hostile to the United States (US) (except when attacked by Augustin de La Balme in 1780). The U.S. government did not trust their neutrality, however.', 'In June 1816, a constitutional convention was held and a state government was formed. The territory was dissolved on December 11, 1816, by an act of Congress granting statehood to Indiana. In February 1815, the United States House of Representatives began debate on granting Indiana Territory statehood. In early 1816, the Territory approved a census and Pennington was named to be the census enumerator.', 'Stuart Banner, a law professor, does not deny that between the early 17th century and the end of the 19th, nearly the enti...</code> | <code>[1, 1, 0, 0, 0, ...]</code> |
+  | <code>what is a business director</code>                              | <code>['Intel Board of Directors. A director is a person from a group of managers who leads or supervises a particular area of a company, program, or project. Companies that use this term often have many directors spread throughout different business functions or roles (e.g. director of human resources). The director usually reports directly to a vice president or to the CEO directly in order to let them know the progress of the organization. An executive director within a company or an organization is usually from the board of directors and oversees a specific department within the organization such as Marketing, Finance, Production and IT.', 'company director. An appointed or elected member of the board of directors of a company who, with other directors, has the responsibility for determining and implementing the company’s policy.', 'Microsoft Outlook 2013 with Business Contact Manager is a great customer relationship management (CRM) tool for small business owners because they can use it...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>why is the thyroid gland important</code>                       | <code>['The thyroid is a small, butterfly-shaped gland located at the base of your neck. It is one of many glands in the endocrine system in the body that regulate the function, growth and development of virtually every cell, tissue and organ in the body. Endocrine glands secrete hormones directly into the bloodstream.', 'Thyroid dysfunction is when the thyroid gland, a small, butterfly-shaped gland located at the base of your neck, produces too much thyroid hormone. This is when you body’s endocrine system speed up, which is referred to as hyperthyroidism.', 'Thyroxine is the most important hormone produced by the thyroid gland. When the gland produces little or too much of this hormone, the body system faces major challenges. For example, if the thyroid is under-active, this could result in Goitre, which is a swelling at the neck.', 'The anterior pituitary makes several important hormones-growth hormone, puberty hormones (or gonadotrophins), thyroid stimulating hormone (TSH, which stimulat...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": null,
+      "respect_input_order": true
+  }
+  ```
+
+### Evaluation Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 1,000 evaluation samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 33.08 characters</li><li>max: 94 characters</li></ul> | <ul><li>min: 1 elements</li><li>mean: 5.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 1 elements</li><li>mean: 5.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                        | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:-------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>how many assistants does michelle have</code>          | <code>['Never in the history of the White House has a First Lady spent so much on so many personal assistants, all paid from taxpayer dollars. Hilary Clinton had three (3)! Michelle has 26, from makeup artist Ingrid Miles and hairstylist Johnny Wright to her “chief of staff” Susan Sher whose salary is $172,200.00!', 'Allegations that Michelle Obama has an excessively large staff compared to other first ladies is nothing new. In 2009, FactCheck.org and Snopes.com debunked the claim circulated in a chain e-mail that Michelle Obama had an unprecedented number of staffers, with 22.', "Of course since Michelle Obama's Twitter account tweeted today to announce that it couldn't Tweet, the situation probably won't become urgent until Thursday. But we finally have an answer to the question: How many assistants does it take to Tweet a link for Michelle Obama. The answer: 16. You filthy Republicans.", "Myra Gutin, an expert on first ladies and politics at Rider University in New Jersey, said that as of...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>how long and at what temperature to bake salmon</code> | <code>["Oven Temperature. Another thing that determines how long the salmon is baked is oven temperature. Typically, recipes for baking salmon call for an oven temperature of 350 to 450 degrees. The salmon should always be put into a pre-heated oven. Cooking in an oven that hasn't been pre-heated can cause drying of the fish. 1 A two-inch thick fillet will bake for 20 minutes. 2  A 1-1/2 filet will take 15 minutes and so on. 3  Check the salmon frequently. 4  Start checking at about 10 minutes, and keep checking until the flesh of the fish is just barely an opaque pink.", 'Preheat the oven to 450 degrees F. Season salmon with salt and pepper. Place salmon, skin side down, on a non-stick baking sheet or in a non-stick pan with an oven-proof handle. Bake until salmon is cooked through, about 12 to 15 minutes. Serve with the Toasted Almond Parsley Salad and squash, if desired. Mince the shallot and add to a small bowl.', 'Report Abuse. I preheat the oven to 500 degrees, really hot. Put the salm...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>what is gene deletion</code>                           | <code>["Deletion on a chromosome. In genetics, a deletion (also called gene deletion, deficiency, or deletion mutation) (sign: δ) is a mutation (a genetic aberration) in which a part of a chromosome or a sequence of DNA is lost during DNA replication. Any number of nucleotides can be deleted, from a single base to an entire piece of chromosome. The smallest single base deletion mutations are believed occur by a single base flipping in the template DNA, followed by template DNA strand slippage, within the DNA polymerase active site. 1 ' Terminal Deletion' — a deletion that occurs towards the end of a chromosome. 2  Intercalary Deletion / Interstitial Deletion — a deletion that occurs from the interior of a chromosome. 3  Microdeletion — a relatively small amount of deletion (up to 5Mb that could include a dozen genes).", '22q11.2 deletion syndrome (which is also known by several other names, listed below) is a disorder caused by the deletion of a small piece of chromosome 22. The deletion occ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>PListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#plistmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": null,
+      "respect_input_order": true
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch      | Step     | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:----------:|:--------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1         | -1       | -             | -               | 0.0224 (-0.5181)         | 0.2459 (-0.0791)          | 0.0785 (-0.4221)     | 0.1156 (-0.3398)           |
+| 0.0002     | 1        | 2.2034        | -               | -                        | -                         | -                    | -                          |
+| 0.0508     | 250      | 2.1047        | -               | -                        | -                         | -                    | -                          |
+| 0.1016     | 500      | 1.9773        | 1.9326          | 0.1454 (-0.3951)         | 0.2533 (-0.0717)          | 0.2214 (-0.2793)     | 0.2067 (-0.2487)           |
+| 0.1525     | 750      | 1.911         | -               | -                        | -                         | -                    | -                          |
+| 0.2033     | 1000     | 1.8706        | 1.8490          | 0.4764 (-0.0640)         | 0.3298 (+0.0048)          | 0.5301 (+0.0295)     | 0.4455 (-0.0099)           |
+| 0.2541     | 1250     | 1.8645        | -               | -                        | -                         | -                    | -                          |
+| 0.3049     | 1500     | 1.857         | 1.8414          | 0.5404 (-0.0001)         | 0.3443 (+0.0192)          | 0.6513 (+0.1506)     | 0.5120 (+0.0566)           |
+| 0.3558     | 1750     | 1.8524        | -               | -                        | -                         | -                    | -                          |
+| 0.4066     | 2000     | 1.841         | 1.8224          | 0.5780 (+0.0375)         | 0.3498 (+0.0247)          | 0.6080 (+0.1074)     | 0.5119 (+0.0565)           |
+| 0.4574     | 2250     | 1.8239        | -               | -                        | -                         | -                    | -                          |
+| 0.5082     | 2500     | 1.8221        | 1.8216          | 0.5538 (+0.0134)         | 0.3481 (+0.0230)          | 0.6245 (+0.1238)     | 0.5088 (+0.0534)           |
+| 0.5591     | 2750     | 1.8238        | -               | -                        | -                         | -                    | -                          |
+| 0.6099     | 3000     | 1.8377        | 1.8066          | 0.5280 (-0.0124)         | 0.3363 (+0.0113)          | 0.5669 (+0.0663)     | 0.4771 (+0.0217)           |
+| 0.6607     | 3250     | 1.8357        | -               | -                        | -                         | -                    | -                          |
+| 0.7115     | 3500     | 1.8221        | 1.8041          | 0.5424 (+0.0020)         | 0.3481 (+0.0230)          | 0.5980 (+0.0973)     | 0.4962 (+0.0408)           |
+| 0.7624     | 3750     | 1.8245        | -               | -                        | -                         | -                    | -                          |
+| 0.8132     | 4000     | 1.8287        | 1.8026          | 0.5627 (+0.0223)         | 0.3564 (+0.0314)          | 0.6185 (+0.1178)     | 0.5125 (+0.0572)           |
+| 0.8640     | 4250     | 1.8125        | -               | -                        | -                         | -                    | -                          |
+| **0.9148** | **4500** | **1.8198**    | **1.803**       | **0.5634 (+0.0229)**     | **0.3493 (+0.0243)**      | **0.6396 (+0.1390)** | **0.5174 (+0.0621)**       |
+| 0.9656     | 4750     | 1.8193        | -               | -                        | -                         | -                    | -                          |
+| -1         | -1       | -             | -               | 0.5634 (+0.0229)         | 0.3493 (+0.0243)          | 0.6396 (+0.1390)     | 0.5174 (+0.0621)           |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.11.11
+- Sentence Transformers: 3.5.0.dev0
+- Transformers: 4.49.0
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.5.2
+- Datasets: 3.4.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### PListMLELoss
+```bibtex
+@inproceedings{lan2014position,
+  title={Position-Aware ListMLE: A Sequential Learning Process for Ranking.},
+  author={Lan, Yanyan and Zhu, Yadong and Guo, Jiafeng and Niu, Shuzi and Cheng, Xueqi},
+  booktitle={UAI},
+  volume={14},
+  pages={449--458},
+  year={2014}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "_name_or_path": "models/reranker-msmarco-v1.1-MiniLM-L12-H384-uncased-plistmle-normalize=minmax/final",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be164d8f8ccdcb42506a6b58b5a1ea75fc3cd73d11cffce5652ab5f40b21580b
+size 133464836

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,65 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}