aciang/mistral7b-tk-sft-20251019-merged

數學推理 LLM (Merged)

重要聲明 (必讀) 本模型涉及原住民族文化、知識與表述內容。未經創作者 Aciang Iku-Silan 明確書面同意，禁止下載、複製、再散布或用於任何研究與商業用途。 詳見本倉庫 RIGHTS_NOTICE.md。

本倉庫提供合併後 (單權重) 的模型檔，用於繁體中文任務 (教學、文化、知識問答等混合 SFT)。 訓練設備：A100 40GB (bf16)，packing=True，max_seq_length=512。

快速開始

# Install libraries (adjust versions as needed)
!pip install transformers peft trl accelerate torch bitsandbytes datasets

# Load model and tokenizer
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel # Use PeftModel if this is a LoRA adapter
import torch

model_id = "aciang/mistral7b-tk-sft-20251019-merged"
base_model_id = "mistralai/Mistral-7B-v0.1"

# For merged model:
if is_merged:
    tok = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map='auto')
else:
    # For LoRA adapter:
    tok = AutoTokenizer.from_pretrained(base_model_id)
    base_model = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype=torch.bfloat16, device_map='auto')
    model = PeftModel.from_pretrained(base_model, model_id)

model.eval()

# Example Inference
prompt = "請用三點條列解釋為何梯度累積可以在小批次下達到大批次效果："
inputs = tok(prompt, return_tensors='pt').to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9, do_sample=True)

print(tok.decode(outputs[0], skip_special_tokens=True))

範例任務

以下是模型可能擅長的任務類型範例： 數學解題、文化知識問答

---

Built with ❤️ and 繁體中文