abhibisht89/signalade-4b-lora

SignalADE-4B (LoRA)

SignalADE scores whether free text carries an adverse drug event (ADE/ADR) signal — and related PV checks — as calibrated option probabilities in one forward pass.

Paste a sentence or snippet, ask a yes/no (or multi-option) question, read P(yes) / P(no). Built for pharmacovigilance workflows (detect → verify), not as a general chat model.

Results (frozen holdout)

ADE-Corpus-V2 classification · seed=42 · N=200 (100 yes / 100 no) · hold_frac=0.15 · T=1.3929 · 4-bit on T4.

Model	Acc	macro-F1	ECE	mean P(yes)
Base Qwen3.5-4B	0.845	0.843	0.118	0.607
SignalADE v1 (this LoRA)	0.965	0.965	0.050	0.509

v1 confusion: 100/100 yes correct · 93/100 no correct.

Load

Needs a recent transformers that supports qwen3_5 (e.g. >=5.0).

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

BASE = "Qwen/Qwen3.5-4B"
ADAPTER = "abhibisht89/signalade-4b-lora"

tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")
base = AutoModelForCausalLM.from_pretrained(
    BASE, quantization_config=bnb, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

Score next-token letter logprobs for options A/B (not open-ended generation). See signalade_config.json for temperature and the system prompt.

Training (v1)

• Together LoRA letter-SFT · job ft-9216ab83-f53f
• r=8 · α=16 · 1 epoch · packing=false · max seq 8192 · ~29k train rows
• Mix: ADE-Corpus-V2 (balanced detect + relation / hard-neg) + BioDEX-ICSR
• Train excludes the frozen 15% hash eval holdout used above
• Adapter keys rewritten (strip .language_model.) for text-only Qwen3.5 PEFT load

Licence

Research / non-commercial by default (ADE-V2 + BioDEX in the mix). Not for clinical care or regulatory filing without your own validation.

Credits

Qwen · Together letter-SFT tooling · ADE-Corpus-V2 & BioDEX authors · SpanBERT ADE NER for propose-stage extraction · BioDecision-4B as related open biomed work (different scope: general biomed vs ADE/PV focus here)

Related Hub ids

Artifact	Link
This LoRA	abhibisht89/signalade-4b-lora
Merged weights	abhibisht89/signalade-4b
Demo	abhibisht89/signalade-demo