Add new CrossEncoder model

README.md

@@ -0,0 +1,475 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:2369
+- loss:BinaryCrossEntropyLoss
+base_model: answerdotai/answerai-colbert-small-v1
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+---
+
+# colbert-small-v1 trained on climatecheck
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/answerai-colbert-small-v1](https://huggingface.co/answerdotai/answerai-colbert-small-v1) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [answerdotai/answerai-colbert-small-v1](https://huggingface.co/answerdotai/answerai-colbert-small-v1) <!-- at revision be1703c55532145a844da800eea4c9a692d7e267 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+<!-- - **Training Dataset:** Unknown -->
+- **Language:** en
+- **License:** apache-2.0
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("gmguarino/answerai-colbert-small-v1-climatecheck")
+# Get scores for pairs of texts
+pairs = [
+    ['Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?', 'Welcome to the city Human populations are shifting en masse to cities, which is leading to rapid increases in the number and extent of urban areas. Such changes are well known to cause declines in many species, but they can also act as alternative selection pressures to which some species are able to adapt. Johnson and Munshi-South review the suite of pressures that urban environments exert, the ways in which species may (or may not) adapt, and the larger impact of these evolutionary events on natural processes and human populations. Understanding such urban evolution patterns will improve our ability to foster species persistence in the face of urbanization and to mitigate some of the challenges, such as disease, that adaptation can bring. Science, this issue p. eaam8327 BACKGROUND The extent of urban areas is increasing around the world, and most humans now live in cities. Urbanization results in dramatic environmental change, including increased temperatures, more impervious surface cover, altered hydrology, and elevated pollution. Urban areas also host more non-native species and reduced abundance and diversity of many native species. These environmental changes brought by global urbanization are creating novel ecosystems with unknown consequences for the evolution of life.'],
+    ['Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?', 'These environmental changes brought by global urbanization are creating novel ecosystems with unknown consequences for the evolution of life. Here, we consider how early human settlements led to the evolution of human commensals, including some of the most notorious pests and disease vectors. We also comprehensively review how contemporary urbanization affects the evolution of species that coinhabit cities. ADVANCES A recent surge of research shows that urbanization affects both nonadaptive and adaptive evolution. Some of the clearest results of urban evolution show that cities elevate the strength of random genetic drift (stochastic changes in allele frequencies) and restrict gene flow (the movement of alleles between populations due to dispersal and mating). Populations of native species in cities often represent either relicts that predate urbanization or populations that established after a city formed. Both scenarios frequently result in a loss of genetic diversity within populations and increased differentiation between populations. Fragmentation and urban infrastructure also create barriers to dispersal, and consequently, gene flow is often reduced among city populations, which further contributes to genetic differentiation between populations. The influence of urbanization on mutation and adaptive evolution are less clear.'],
+    ['Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?', 'The influence of urbanization on mutation and adaptive evolution are less clear. A small number of studies suggest that industrial pollution can elevate mutation rates, but the pervasiveness of this effect is unknown. A better studied phenomenon are the effects of urbanization on evolution by natural selection. A growing number of studies show that plant and animal populations experience divergent selection between urban and nonurban environments. This divergent selection has led to adaptive evolution in life history, morphology, physiology, behavior, and reproductive traits. These adaptations typically evolve in response to pesticide use, pollution, local climate, or the physical structure of cities. Despite these important results, the genetic basis of adaptive evolution is known from only a few cases. Most studies also examine only a few populations in one city, and experimental validation is rare. OUTLOOK The study of evolution in urban areas provides insights into both fundamental and applied problems in biology. The thousands of cities throughout the world share some features while differing in other aspects related to their age, historical context, governmental policies, and local climate. Thus, the phenomenon of global urbanization represents an unintended but highly replicated global study of experimental evolution.'],
+    ['Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?', 'Thus, the phenomenon of global urbanization represents an unintended but highly replicated global study of experimental evolution. We can harness this global urban experiment to understand the repeatability and pace of evolution in response to human activity. Among the most important unresolved questions is, how often do native and exotic species adapt to the particular environmental challenges found in cities? Such adaptations could be the difference as to whether a species persists or vanishes from urban areas. In this way, the study of urban evolution can help us understand how evolution in populations may contribute to conservation of rare species, and how populations can be managed to facilitate the establishment of resilient and sustainable urban ecosystems. In a similar way, understanding evolution in urban areas can lead to improved human health. For example, human pests frequently adapt to pesticides and evade control efforts because of our limited understanding of the size of populations and movement of individuals. Applied evolutionary studies could lead to more effective mitigation of pests and disease agents. The study of urban evolution has rapidly become an important frontier in biology, with implications for healthy and sustainable human populations in urban ecosystems.'],
+    ['Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?', 'The study of urban evolution has rapidly become an important frontier in biology, with implications for healthy and sustainable human populations in urban ecosystems. A gradient in urbanization showing the skyline of Canada’s sixth largest city (Mississauga, Canada) on the horizon, and the Credit Valley and the University of Toronto Mississauga campus in the foreground. PHOTO CREDIT: ARJUN YADAV Our planet is an increasingly urbanized landscape, with over half of the human population residing in cities. Despite advances in urban ecology, we do not adequately understand how urbanization affects the evolution of organisms, nor how this evolution may affect ecosystems and human health. Here, we review evidence for the effects of urbanization on the evolution of microbes, plants, and animals that inhabit cities. Urbanization affects adaptive and nonadaptive evolutionary processes that shape the genetic diversity within and between populations. Rapid adaptation has facilitated the success of some native species in urban areas, but it has also allowed human pests and disease to spread more rapidly. The nascent field of urban evolution brings together efforts to understand evolution in response to environmental change while developing new hypotheses concerning adaptation to urban infrastructure and human socioeconomic activity.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (5,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?',
+    [
+        'Welcome to the city Human populations are shifting en masse to cities, which is leading to rapid increases in the number and extent of urban areas. Such changes are well known to cause declines in many species, but they can also act as alternative selection pressures to which some species are able to adapt. Johnson and Munshi-South review the suite of pressures that urban environments exert, the ways in which species may (or may not) adapt, and the larger impact of these evolutionary events on natural processes and human populations. Understanding such urban evolution patterns will improve our ability to foster species persistence in the face of urbanization and to mitigate some of the challenges, such as disease, that adaptation can bring. Science, this issue p. eaam8327 BACKGROUND The extent of urban areas is increasing around the world, and most humans now live in cities. Urbanization results in dramatic environmental change, including increased temperatures, more impervious surface cover, altered hydrology, and elevated pollution. Urban areas also host more non-native species and reduced abundance and diversity of many native species. These environmental changes brought by global urbanization are creating novel ecosystems with unknown consequences for the evolution of life.',
+        'These environmental changes brought by global urbanization are creating novel ecosystems with unknown consequences for the evolution of life. Here, we consider how early human settlements led to the evolution of human commensals, including some of the most notorious pests and disease vectors. We also comprehensively review how contemporary urbanization affects the evolution of species that coinhabit cities. ADVANCES A recent surge of research shows that urbanization affects both nonadaptive and adaptive evolution. Some of the clearest results of urban evolution show that cities elevate the strength of random genetic drift (stochastic changes in allele frequencies) and restrict gene flow (the movement of alleles between populations due to dispersal and mating). Populations of native species in cities often represent either relicts that predate urbanization or populations that established after a city formed. Both scenarios frequently result in a loss of genetic diversity within populations and increased differentiation between populations. Fragmentation and urban infrastructure also create barriers to dispersal, and consequently, gene flow is often reduced among city populations, which further contributes to genetic differentiation between populations. The influence of urbanization on mutation and adaptive evolution are less clear.',
+        'The influence of urbanization on mutation and adaptive evolution are less clear. A small number of studies suggest that industrial pollution can elevate mutation rates, but the pervasiveness of this effect is unknown. A better studied phenomenon are the effects of urbanization on evolution by natural selection. A growing number of studies show that plant and animal populations experience divergent selection between urban and nonurban environments. This divergent selection has led to adaptive evolution in life history, morphology, physiology, behavior, and reproductive traits. These adaptations typically evolve in response to pesticide use, pollution, local climate, or the physical structure of cities. Despite these important results, the genetic basis of adaptive evolution is known from only a few cases. Most studies also examine only a few populations in one city, and experimental validation is rare. OUTLOOK The study of evolution in urban areas provides insights into both fundamental and applied problems in biology. The thousands of cities throughout the world share some features while differing in other aspects related to their age, historical context, governmental policies, and local climate. Thus, the phenomenon of global urbanization represents an unintended but highly replicated global study of experimental evolution.',
+        'Thus, the phenomenon of global urbanization represents an unintended but highly replicated global study of experimental evolution. We can harness this global urban experiment to understand the repeatability and pace of evolution in response to human activity. Among the most important unresolved questions is, how often do native and exotic species adapt to the particular environmental challenges found in cities? Such adaptations could be the difference as to whether a species persists or vanishes from urban areas. In this way, the study of urban evolution can help us understand how evolution in populations may contribute to conservation of rare species, and how populations can be managed to facilitate the establishment of resilient and sustainable urban ecosystems. In a similar way, understanding evolution in urban areas can lead to improved human health. For example, human pests frequently adapt to pesticides and evade control efforts because of our limited understanding of the size of populations and movement of individuals. Applied evolutionary studies could lead to more effective mitigation of pests and disease agents. The study of urban evolution has rapidly become an important frontier in biology, with implications for healthy and sustainable human populations in urban ecosystems.',
+        'The study of urban evolution has rapidly become an important frontier in biology, with implications for healthy and sustainable human populations in urban ecosystems. A gradient in urbanization showing the skyline of Canada’s sixth largest city (Mississauga, Canada) on the horizon, and the Credit Valley and the University of Toronto Mississauga campus in the foreground. PHOTO CREDIT: ARJUN YADAV Our planet is an increasingly urbanized landscape, with over half of the human population residing in cities. Despite advances in urban ecology, we do not adequately understand how urbanization affects the evolution of organisms, nor how this evolution may affect ecosystems and human health. Here, we review evidence for the effects of urbanization on the evolution of microbes, plants, and animals that inhabit cities. Urbanization affects adaptive and nonadaptive evolutionary processes that shape the genetic diversity within and between populations. Rapid adaptation has facilitated the success of some native species in urban areas, but it has also allowed human pests and disease to spread more rapidly. The nascent field of urban evolution brings together efforts to understand evolution in response to environmental change while developing new hypotheses concerning adaptation to urban infrastructure and human socioeconomic activity.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 2,369 training samples
+* Columns: <code>anchor</code>, <code>passage</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                                           | passage                                                                                           | label                                                          |
+  |:--------|:-------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                                           | string                                                                                            | float                                                          |
+  | details | <ul><li>min: 30 characters</li><li>mean: 114.16 characters</li><li>max: 209 characters</li></ul> | <ul><li>min: 77 characters</li><li>mean: 972.69 characters</li><li>max: 1537 characters</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.58</li><li>max: 1.0</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                               | passage                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | label            |
+  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
+  | <code>Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?</code> | <code>Welcome to the city Human populations are shifting en masse to cities, which is leading to rapid increases in the number and extent of urban areas. Such changes are well known to cause declines in many species, but they can also act as alternative selection pressures to which some species are able to adapt. Johnson and Munshi-South review the suite of pressures that urban environments exert, the ways in which species may (or may not) adapt, and the larger impact of these evolutionary events on natural processes and human populations. Understanding such urban evolution patterns will improve our ability to foster species persistence in the face of urbanization and to mitigate some of the challenges, such as disease, that adaptation can bring. Science, this issue p. eaam8327 BACKGROUND The extent of urban areas is increasing around the world, and most humans now live in cities. Urbanization results in dramatic environmental change, including increased temperatures, more impervious surface...</code> | <code>0.0</code> |
+  | <code>Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?</code> | <code>These environmental changes brought by global urbanization are creating novel ecosystems with unknown consequences for the evolution of life. Here, we consider how early human settlements led to the evolution of human commensals, including some of the most notorious pests and disease vectors. We also comprehensively review how contemporary urbanization affects the evolution of species that coinhabit cities. ADVANCES A recent surge of research shows that urbanization affects both nonadaptive and adaptive evolution. Some of the clearest results of urban evolution show that cities elevate the strength of random genetic drift (stochastic changes in allele frequencies) and restrict gene flow (the movement of alleles between populations due to dispersal and mating). Populations of native species in cities often represent either relicts that predate urbanization or populations that established after a city formed. Both scenarios frequently result in a loss of genetic diversity within populati...</code> | <code>0.0</code> |
+  | <code>Turns out, species that can adapt easily to different environments are often the ones that can survive in a wide variety of places. Interesting, right?</code> | <code>The influence of urbanization on mutation and adaptive evolution are less clear. A small number of studies suggest that industrial pollution can elevate mutation rates, but the pervasiveness of this effect is unknown. A better studied phenomenon are the effects of urbanization on evolution by natural selection. A growing number of studies show that plant and animal populations experience divergent selection between urban and nonurban environments. This divergent selection has led to adaptive evolution in life history, morphology, physiology, behavior, and reproductive traits. These adaptations typically evolve in response to pesticide use, pollution, local climate, or the physical structure of cities. Despite these important results, the genetic basis of adaptive evolution is known from only a few cases. Most studies also examine only a few populations in one city, and experimental validation is rare. OUTLOOK The study of evolution in urban areas provides insights into both fundamental...</code> | <code>0.0</code> |
+* Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
+  ```json
+  {
+      "activation_fn": "torch.nn.modules.linear.Identity",
+      "pos_weight": null
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `per_device_train_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 10
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `dataloader_num_workers`: 2
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: no
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 8
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 2
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `tp_size`: 0
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step | Training Loss |
+|:------:|:----:|:-------------:|
+| 0.0067 | 1    | 0.6935        |
+| 0.6711 | 100  | 0.6812        |
+| 0.0067 | 1    | 0.6977        |
+| 0.0671 | 10   | 0.676         |
+| 0.1342 | 20   | 0.6758        |
+| 0.2013 | 30   | 0.6902        |
+| 0.2685 | 40   | 0.6628        |
+| 0.3356 | 50   | 0.6718        |
+| 0.4027 | 60   | 0.6607        |
+| 0.4698 | 70   | 0.6761        |
+| 0.5369 | 80   | 0.6759        |
+| 0.6040 | 90   | 0.6695        |
+| 0.6711 | 100  | 0.6791        |
+| 0.7383 | 110  | 0.6687        |
+| 0.8054 | 120  | 0.6823        |
+| 0.8725 | 130  | 0.682         |
+| 0.9396 | 140  | 0.6784        |
+| 1.0067 | 150  | 0.6745        |
+| 1.0738 | 160  | 0.6505        |
+| 1.1409 | 170  | 0.6132        |
+| 1.2081 | 180  | 0.5973        |
+| 1.2752 | 190  | 0.5939        |
+| 1.3423 | 200  | 0.5827        |
+| 1.4094 | 210  | 0.6278        |
+| 1.4765 | 220  | 0.5761        |
+| 1.5436 | 230  | 0.6245        |
+| 1.6107 | 240  | 0.5832        |
+| 1.6779 | 250  | 0.5128        |
+| 1.7450 | 260  | 0.5529        |
+| 1.8121 | 270  | 0.4886        |
+| 1.8792 | 280  | 0.4943        |
+| 1.9463 | 290  | 0.5186        |
+| 2.0134 | 300  | 0.436         |
+| 2.0805 | 310  | 0.4821        |
+| 2.1477 | 320  | 0.4655        |
+| 2.2148 | 330  | 0.369         |
+| 2.2819 | 340  | 0.4655        |
+| 2.3490 | 350  | 0.3635        |
+| 2.4161 | 360  | 0.5141        |
+| 2.4832 | 370  | 0.4408        |
+| 2.5503 | 380  | 0.4071        |
+| 2.6174 | 390  | 0.5308        |
+| 2.6846 | 400  | 0.4403        |
+| 2.7517 | 410  | 0.3813        |
+| 2.8188 | 420  | 0.5094        |
+| 2.8859 | 430  | 0.3671        |
+| 2.9530 | 440  | 0.4118        |
+| 3.0201 | 450  | 0.3945        |
+| 3.0872 | 460  | 0.3318        |
+| 3.1544 | 470  | 0.3555        |
+| 3.2215 | 480  | 0.4101        |
+| 3.2886 | 490  | 0.3764        |
+| 3.3557 | 500  | 0.4482        |
+| 3.4228 | 510  | 0.3738        |
+| 3.4899 | 520  | 0.3805        |
+| 3.5570 | 530  | 0.3611        |
+| 3.6242 | 540  | 0.4942        |
+| 3.6913 | 550  | 0.3983        |
+| 3.7584 | 560  | 0.353         |
+| 3.8255 | 570  | 0.3006        |
+| 3.8926 | 580  | 0.343         |
+| 3.9597 | 590  | 0.3998        |
+| 4.0268 | 600  | 0.3937        |
+| 4.0940 | 610  | 0.3768        |
+| 4.1611 | 620  | 0.2224        |
+| 4.2282 | 630  | 0.3857        |
+| 4.2953 | 640  | 0.3617        |
+| 4.3624 | 650  | 0.3278        |
+| 4.4295 | 660  | 0.3239        |
+| 4.4966 | 670  | 0.4745        |
+| 4.5638 | 680  | 0.291         |
+| 4.6309 | 690  | 0.3666        |
+| 4.6980 | 700  | 0.322         |
+| 4.7651 | 710  | 0.3243        |
+| 4.8322 | 720  | 0.3856        |
+| 4.8993 | 730  | 0.2332        |
+| 4.9664 | 740  | 0.3998        |
+| 5.0336 | 750  | 0.4055        |
+| 5.1007 | 760  | 0.2819        |
+| 5.1678 | 770  | 0.2425        |
+| 5.2349 | 780  | 0.3387        |
+| 5.3020 | 790  | 0.3621        |
+| 5.3691 | 800  | 0.3063        |
+| 5.4362 | 810  | 0.3306        |
+| 5.5034 | 820  | 0.3163        |
+| 5.5705 | 830  | 0.2516        |
+| 5.6376 | 840  | 0.2746        |
+| 5.7047 | 850  | 0.2841        |
+| 5.7718 | 860  | 0.2799        |
+| 5.8389 | 870  | 0.3608        |
+| 5.9060 | 880  | 0.2268        |
+| 5.9732 | 890  | 0.2612        |
+| 6.0403 | 900  | 0.1797        |
+| 6.1074 | 910  | 0.2328        |
+| 6.1745 | 920  | 0.2906        |
+| 6.2416 | 930  | 0.2722        |
+| 6.3087 | 940  | 0.2567        |
+| 6.3758 | 950  | 0.3215        |
+| 6.4430 | 960  | 0.2563        |
+| 6.5101 | 970  | 0.2294        |
+| 6.5772 | 980  | 0.233         |
+| 6.6443 | 990  | 0.2697        |
+| 6.7114 | 1000 | 0.2585        |
+| 6.7785 | 1010 | 0.2863        |
+| 6.8456 | 1020 | 0.2296        |
+| 6.9128 | 1030 | 0.2399        |
+| 6.9799 | 1040 | 0.2736        |
+| 7.0470 | 1050 | 0.3086        |
+| 7.1141 | 1060 | 0.2781        |
+| 7.1812 | 1070 | 0.2201        |
+| 7.2483 | 1080 | 0.1738        |
+| 7.3154 | 1090 | 0.244         |
+| 7.3826 | 1100 | 0.2045        |
+| 7.4497 | 1110 | 0.2975        |
+| 7.5168 | 1120 | 0.1997        |
+| 7.5839 | 1130 | 0.3476        |
+| 7.6510 | 1140 | 0.3303        |
+| 7.7181 | 1150 | 0.1835        |
+| 7.7852 | 1160 | 0.2045        |
+| 7.8523 | 1170 | 0.2652        |
+| 7.9195 | 1180 | 0.2576        |
+| 7.9866 | 1190 | 0.2281        |
+| 8.0537 | 1200 | 0.2479        |
+| 8.1208 | 1210 | 0.2592        |
+| 8.1879 | 1220 | 0.1455        |
+| 8.2550 | 1230 | 0.3126        |
+| 8.3221 | 1240 | 0.1814        |
+| 8.3893 | 1250 | 0.175         |
+| 8.4564 | 1260 | 0.1712        |
+| 8.5235 | 1270 | 0.2163        |
+| 8.5906 | 1280 | 0.2158        |
+| 8.6577 | 1290 | 0.2192        |
+| 8.7248 | 1300 | 0.22          |
+| 8.7919 | 1310 | 0.3208        |
+| 8.8591 | 1320 | 0.1588        |
+| 8.9262 | 1330 | 0.2477        |
+| 8.9933 | 1340 | 0.1668        |
+| 9.0604 | 1350 | 0.2315        |
+| 9.1275 | 1360 | 0.2444        |
+| 9.1946 | 1370 | 0.363         |
+| 9.2617 | 1380 | 0.1657        |
+| 9.3289 | 1390 | 0.2557        |
+| 9.3960 | 1400 | 0.2392        |
+| 9.4631 | 1410 | 0.1683        |
+| 9.5302 | 1420 | 0.1472        |
+| 9.5973 | 1430 | 0.1653        |
+| 9.6644 | 1440 | 0.2288        |
+| 9.7315 | 1450 | 0.1856        |
+| 9.7987 | 1460 | 0.1729        |
+| 9.8658 | 1470 | 0.2009        |
+| 9.9329 | 1480 | 0.1241        |
+| 10.0   | 1490 | 0.1783        |
+
+</details>
+
+### Framework Versions
+- Python: 3.11.12
+- Sentence Transformers: 4.1.0
+- Transformers: 4.51.3
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.6.0
+- Datasets: 3.6.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid",
+    "version": "4.1.0"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a0dcd0ef77f2d9db437fd75114e5711516cbcdc9feac71195deceb31835e4af
+size 133464836

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}