CVE Analyst -- Ministral-3-3B (Merged)

Fine-tuned mistralai/Ministral-3-3B-Instruct-2512-BF16 on the AlicanKiraz0/All-CVE-Records-Training-Dataset dataset using QLoRA, then merged into a standalone BF16 model.

This is the merged version with LoRA adapter weights baked into the base model. No adapter loading needed -- use it like any standard model. For the lightweight adapter-only version, see noman-asif/CVE-Analyst-Ministral-3B-QLoRA.

Model Description

This model is specialised for CVE vulnerability analysis -- given a vulnerability identifier and context, it produces structured technical analyses including exploitation vectors, impact assessment, and remediation strategies.

Property Value
Base model mistralai/Ministral-3-3B-Instruct-2512-BF16
Method QLoRA (merged)
LoRA rank 16
LoRA alpha 32
Training epochs 3
Learning rate 0.0002
Effective batch size 12
Max sequence length 4096

Training Data

Dataset: AlicanKiraz0/All-CVE-Records-Training-Dataset

Attribution: Dataset created by AlicanKiraz0.

License: Apache 2.0

Sample size: 10000 rows (filtered to fit within 4096 tokens).

Split Count
train 8000
val 1000
test 1000

Training Procedure

  • Hardware: NVIDIA RTX 4070 (8 GB VRAM)
  • Training time: 33h 22m 55s
  • Framework: HuggingFace Transformers + PEFT + bitsandbytes

Evaluation Results

Before vs After

Metric Original Fine-tuned
bleu 4.53 32.87
rouge1 19.8 34.33
rouge2 4.39 21.38
rougeL 10.77 30.71
perplexity 5.11 2.01

CVE-ID Extraction

Metric Original Fine-tuned
precision 79.37 100.0
recall 92.59 92.59
f1 85.47 96.15

Limitations

  • The model was trained on a subset (10000 samples) of the full dataset; coverage of less common CVE types may be limited.
  • Maximum sequence length during training was 4096 tokens; very long analyses will be truncated.
  • The model inherits the base model's biases and limitations.
  • Responses should be reviewed by a qualified security professional before being used in production or advisory contexts.

How to Use

from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend

# Load merged model and tokenizer
model = Mistral3ForConditionalGeneration.from_pretrained(
    "noman-asif/CVE-Analyst-Ministral-3B-Adapters-Merged",
    torch_dtype="auto",
    device_map="auto",
)
tokenizer = MistralCommonBackend.from_pretrained(
    "noman-asif/CVE-Analyst-Ministral-3B-Adapters-Merged"
)

# Generate
messages = [{"role": "user", "content": "Provide a comprehensive technical analysis of CVE-2024-1234."}]
inputs = tokenizer.apply_chat_template(messages, return_dict=True, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(**inputs.to(model.device), max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

This model is released under the Apache 2.0 License.

Downloads last month
11
Safetensors
Model size
4B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for noman-asif/CVE-Analyst-Ministral-3B-Adapters-Merged

Dataset used to train noman-asif/CVE-Analyst-Ministral-3B-Adapters-Merged