hiyouga
/

Qwen-14B-Chat-LLaMAfied

Text Generation

text-generation-inference

Model card Files Files and versions

hiyouga commited on Dec 25, 2023

Commit

c570a32

·

1 Parent(s): d7c71db

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -31,7 +31,7 @@ Usage:
 from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
 tokenizer = AutoTokenizer.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied")
-model = AutoModelForCausalLM.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied").cuda()
 streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
 query = (

 from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
 tokenizer = AutoTokenizer.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied")
+model = AutoModelForCausalLM.from_pretrained("hiyouga/Qwen-14B-Chat-LLaMAfied", torch_dtype="auto", device_map="auto")
 streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
 query = (