Create README.md

README.md

@@ -0,0 +1,11 @@
+You can deploy it using vllm. And this is the script for deploying.
+``` bash
+python -O -u -m vllm.entrypoints.openai.api_server \
+        --host=127.0.0.1 \
+        --port=8090 \
+        --model=Melon/Meta-Llama-3-70B-Instruct-AutoAWQ-4bit \
+        --tokenizer=meta-llama/Meta-Llama-3-70B-Instruct \
+        --tensor-parallel-size=1 \
+        --quantization awq \
+        --dtype half
+```