---
base_model: meta-llama/Llama-3.2-1B-Instruct
library_name: peft
license: llama3.2
tags:
- trl
- sft
- generated_from_trainer
- lora
model-index:
- name: llama-3.2-1B-it-Procurtech-Assistant
  results: []
datasets:
- Victorano/procurtech-assistant-training-dataset
language:
- en
pipeline_tag: text2text-generation
---


# llama-3.2-1B-it-Procurtech-Assistant

This model is a fine-tuned version of [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) on [Procurtech Assistant dataset](https://huggingface.co/datasets/Victorano/procurtech-assistant-training-dataset).

## Model description

A customer support model to help customers with their orders, incase they encounter any difficulty.

## Intended uses & limitations

The training dataset can be modified, see original at [customer support dataset](https://huggingface.co/bitext/Bitext-customer-support-llm-chatbot-training-dataset) .. I edited the system message with a bit of prompt engineering, included additional details about the eCommerce company.   
You can decide what you want and further fine tune the model...

## Training and evaluation data

[Training data](https://huggingface.co/datasets/Victorano/procurtech-assistant-training-dataset).    
Used the complete dataset for training, no evaluation data, I evaluated with random prompts...

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 2
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 682
- num_epochs: 1

### Training results
[Training Loss from wandb](https://api.wandb.ai/links/victordareai/o4f88gmp)


### Framework versions

- PEFT 0.13.2
- Transformers 4.45.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.20.1