CyberOSS-CVE

CyberOSS-CVE

Fine-tuned gpt-oss-20b on the AlicanKiraz0/All-CVE-Records-Training-Dataset using Unsloth with LoRA (rank 32) and merged back to BF16 for compatibility with vLLM, Hugging Face Transformers, and GGUF conversions.

Training Overview

• Base model: unsloth/gpt-oss-20b-BF16
• Dataset: AlicanKiraz0/All-CVE-Records-Training-Dataset
• Hardware: single NVIDIA H100 80GB
• Sequence length: 2048
• Batch: 2 (grad accum 4 → effective 8)
• Learning rate: 2e-4, linear warmup 5 steps
• Steps: 100 for quick verification run (expand for full epoch)
• Loss masking: full conversation (system, user, assistant)

Files

• model-0000X-of-00009.safetensors: merged BF16 shards
• config.json: GPT-OSS architecture config
• tokenizer.json and template: Harmony/GPT-OSS chat format
• chat_template.jinja: OpenAI Harmony-compatible chat template

Quick Usage (vLLM)

pip install vllm==0.11.2 transformers==4.57.2
python - <<'PY'
from vllm import LLM, SamplingParams
from transformers.processing_utils import ProcessorMixin
import transformers
transformers.ProcessorMixin = ProcessorMixin

llm = LLM(
    model="Kushalkhemka/CyberOSS-CVE",
    tokenizer="unsloth/gpt-oss-20b-BF16",
    dtype="bfloat16",
)
prompt = "You are a cybersecurity assistant. Summarize CVE-2010-3763."
out = llm.generate([prompt], SamplingParams(max_tokens=128))[0]
print(out.outputs[0].text)
PY

HF Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Kushalkhemka/CyberOSS-CVE", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("unsloth/gpt-oss-20b-BF16")

License

Matches upstream unsloth/gpt-oss-20b (LGPL-3.0). Respect dataset terms when redistributing.