Hugging Face's logo

kuljeet98
/
bert-model

Sentence Similarity
sentence-transformers
PyTorch
bert
feature-extraction
Generated from Trainer
dataset_size:8408
loss:CosineSimilarityLoss
text-embeddings-inference
Model card Files
xet
Community
1

SentenceTransformer based on sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a sentence-transformers model finetuned from sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 馃 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'commercial manager',
    'gerente de operaciones',
    'vice president of finance americas',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

  • Size: 8,408 training samples
  • Columns: sentence_0, sentence_1, and label
  • Approximate statistics based on the first 1000 samples:
    sentence_0 sentence_1 label
    type string string float
    details
    • min: 3 tokens
    • mean: 6.2 tokens
    • max: 12 tokens
    • min: 3 tokens
    • mean: 7.75 tokens
    • max: 21 tokens
    • min: 0.0
    • mean: 0.06
    • max: 1.0
  • Samples:
    sentence_0 sentence_1 label
    strategic planning manager senior brand manager uap southern cone & personal care cdm chile 0.0
    director de planificacion key account manager tiendas paris 0.0
    gerente general analista de cobranza 0.0
  • Loss: CosineSimilarityLoss with these parameters:
    {
    "loss_fct": "torch.nn.modules.loss.MSELoss"
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 16
  • num_train_epochs: 50
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1
  • num_train_epochs: 50
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.0
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin

Training Logs

Epoch Step Training Loss
0.9506 500 0.0434
1.9011 1000 0.0135
2.8517 1500 0.0072
3.8023 2000 0.0056
4.7529 2500 0.0044
5.7034 3000 0.0038
6.6540 3500 0.0034
7.6046 4000 0.0032
8.5551 4500 0.0029
9.5057 5000 0.0028
10.4563 5500 0.0026
11.4068 6000 0.0025
12.3574 6500 0.0026
13.3080 7000 0.0023
14.2586 7500 0.0023
15.2091 8000 0.0023
16.1597 8500 0.0022
17.1103 9000 0.0021
18.0608 9500 0.0019
19.0114 10000 0.0021
19.9620 10500 0.0019
20.9125 11000 0.0019
21.8631 11500 0.0016
22.8137 12000 0.0018
23.7643 12500 0.0018
24.7148 13000 0.0018
25.6654 13500 0.0016
26.6160 14000 0.0017
27.5665 14500 0.0016
28.5171 15000 0.0016
29.4677 15500 0.0016
30.4183 16000 0.0016
31.3688 16500 0.0019
32.3194 17000 0.0018
33.2700 17500 0.0017
34.2205 18000 0.0016
35.1711 18500 0.0016
36.1217 19000 0.0016
37.0722 19500 0.0015
38.0228 20000 0.0012
38.9734 20500 0.0015
39.9240 21000 0.0015
40.8745 21500 0.0013
41.8251 22000 0.0014
42.7757 22500 0.0014
43.7262 23000 0.0014
44.6768 23500 0.0013
45.6274 24000 0.0012
46.5779 24500 0.0014
47.5285 25000 0.0012
48.4791 25500 0.0013
49.4297 26000 0.0013

Framework Versions

  • Python: 3.8.5
  • Sentence Transformers: 3.0.1
  • Transformers: 4.41.2
  • PyTorch: 2.1.1+cu121
  • Accelerate: 0.32.1
  • Datasets: 2.20.0
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
Downloads last month
1
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for kuljeet98/bert-model

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Finetuned
(283)
this model