DictaLM-3.0-24B (Thinking) HebNLI LoRA adapter

LoRA adapter (rank 16, alpha 32, all linear projection matrices, bfloat16) that adapts dicta-il/DictaLM-3.0-24B-Thinking to 3-class Hebrew NLI on HebNLI. It is one of the Strategy 2 denoisers of the HeRE paper. The model is prompted as

משפט 1: {premise} משפט 2: {hypothesis}

and the label is read from the first generated token (היסק = entailment, סתירה = contradiction, ניטרלי = neutral); in HeRE the soft score is derived from the first-token log-probabilities of the three label tokens.

Training

Learning rate 5e-5, batch size 16, up to 8,000 steps; the checkpoint with the best validation macro-F1 was retained (step 5,280). Single run, seed 42.

Results

HebNLI dev macro-F1 HebNLI test macro-F1 HeRE gold validation F1
DictaLM-3.0-24B (Thinking) 0.876 0.902 0.861 (Strategy 2, template hypothesis). This adapter produced the finetuned_nli__dictalm24b signal on the full silver corpus and is one of the three models whose unanimous vote (agree3) defines the published HeRE silver label.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("ronke21/hebnli-dictalm-3.0-24b-lora")
model = AutoModelForCausalLM.from_pretrained("dicta-il/DictaLM-3.0-24B-Thinking", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "ronke21/hebnli-dictalm-3.0-24b-lora")

License

The adapter is released under the base model's license (apache-2.0); using it requires accepting the base model's terms. Paper resources: dataset https://huggingface.co/datasets/ronke21/HeRE, code https://github.com/Ronke21/HeRE.

Citation

Keinan, R., Cohen, A. D. N., and Tsarfaty, R. (2026). HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments. In Proceedings of AACL 2026.

Downloads last month
5
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ronke21/hebnli-dictalm-3.0-24b-lora

Adapter
(1)
this model

Collection including ronke21/hebnli-dictalm-3.0-24b-lora