---
base_model: unsloth/Qwen2-7B-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/Qwen2-7B-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
---

# Model Name: <ParliaBench - unsloth/Qwen2-7B-bnb-4bit>

## 1. Model Summary
This is a fine-tuned model developed for the research paper 
**"ParliaBench: An Evaluation and Benchmarking Framework for LLM-Generated Parliamentary Speech"**.

The model is trained to:
- Generate UK-style parliamentary debates  
- Produce party-conditioned political arguments  
- Model parliamentary dialogue structures  
- Perform topic-aware generation using 21 EuroVoc topic categories

## 2. Base Model
Base model: "unsloth/Qwen2-7B-bnb-4bit"

> **This repository contains only LoRA adapter weights.  
> The base model must be loaded separately.**

## 3. Training Data
The model was trained using:
- **ParlaMint-GB from Clarin**
- Preprocessed into:
  - Structured debates
  - Speaker metadata
  - Section headers
  - 21-class topic labels

Dataset links:
- https://huggingface.co/datasets/argyrotsipi/train-dataset
- https://huggingface.co/datasets/argyrotsipi/generated-dataset
  
## 4. Training Procedure

### Hyperparameters
  - Batch Size 64 
  - Learning Rate 2e-4 
  - Max Steps 11194 (2 epochs)
  - Warmup Steps 336 (10% of max steps for stability)
  - Optimizer adamw
  - Weight Decay 0.01 
  - Max Sequence Length 1024 
  - Scheduler linear

### Hardware
GPU A100
AWS resources were provided by the National Infrastructures for Research and Technology GRNET and funded by the EU Recovery and Resiliency Facility.

  - LoRA Rank (r) 16 
  - LoRA Alpha 16 
  - Target Modules 7 layers 
  - LoRA Dropout 0 
  - Bias Configuration none 
  - Random State 3407

## 5. Intended Uses
This model is designed for:
- Political debate simulation  
- Parliamentary dialogue generation  
- Academic NLP research  
- Social behavior modeling  

## 6. Limitations
- May reproduce political biases present in the corpus  
- Not suitable for real political advice or predictions  
- Model outputs are synthetic and not factual

## 7. Ethical Considerations
- Avoid using for political persuasion  
- Use strictly in academic and research contexts  
- Outputs may reflect UK political biases from the dataset

## 8. Citation
@misc{ParliaBench2025,
      title={ParliaBench: An Evaluation and Benchmarking Framework for LLM-Generated Parliamentary Speech},
      author={Marios Koniaris and Argyro Tsipi and Panayiotis Tsanakas},
      year={2025},
      eprint={2511.08247},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2511.08247},
}
## 9. Authors
Marios Koniaris, Argyro Tsipi, Panayiotis Tsanakas
ParliaBench: An Evaluation and Benchmarking Framework for LLM-Generated Parliamentary Speech.