QLoRA: Efficient Finetuning of Quantized LLMs
Paper • 2305.14314 • Published • 61
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("acrastt/IS-LM-3B", trust_remote_code=True, dtype="auto")Quick Links
This is StableLM 3B 4E1T(Licensed under CC BY-SA 4.0.) finetuned on DataForge Economics for 3 epochs using QLoRA.
Prompt template:
USER: {prompt}
ASSISTANT:
GGUF quantizations available here.
- Downloads last month
- 27
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="acrastt/IS-LM-3B", trust_remote_code=True)