| | --- |
| | base_model: UCSC-VLAA/STAR1-R1-Distill-7B |
| | datasets: |
| | - UCSC-VLAA/STAR-1 |
| | library_name: transformers |
| | license: apache-2.0 |
| | tags: |
| | - llama-cpp |
| | - gguf-my-repo |
| | --- |
| | |
| | # Triangle104/STAR1-R1-Distill-7B-Q6_K-GGUF |
| | This model was converted to GGUF format from [`UCSC-VLAA/STAR1-R1-Distill-7B`](https://huggingface.co/UCSC-VLAA/STAR1-R1-Distill-7B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space. |
| | Refer to the [original model card](https://huggingface.co/UCSC-VLAA/STAR1-R1-Distill-7B) for more details on the model. |
| | |
| | --- |
| | STAR-1 is a high-quality safety dataset designed to enhance safety alignment in large reasoning models (LRMs) like DeepSeek-R1. |
| | |
| | |
| | Built on the principles of diversity, deliberative reasoning, and |
| | rigorous filtering, STAR-1 integrates and refines data from multiple |
| | sources to provide policy-grounded reasoning samples. |
| | The dataset contains 1,000 carefully selected examples, each aligned with best safety practices through GPT-4o-based evaluation. |
| | Fine-tuning with STAR-1 leads to significant safety improvements |
| | across multiple benchmarks, with minimal impact on reasoning |
| | capabilities. |
| | |
| | --- |
| | ## Use with llama.cpp |
| | Install llama.cpp through brew (works on Mac and Linux) |
| | |
| | ```bash |
| | brew install llama.cpp |
| | |
| | ``` |
| | Invoke the llama.cpp server or the CLI. |
| | |
| | ### CLI: |
| | ```bash |
| | llama-cli --hf-repo Triangle104/STAR1-R1-Distill-7B-Q6_K-GGUF --hf-file star1-r1-distill-7b-q6_k.gguf -p "The meaning to life and the universe is" |
| | ``` |
| | |
| | ### Server: |
| | ```bash |
| | llama-server --hf-repo Triangle104/STAR1-R1-Distill-7B-Q6_K-GGUF --hf-file star1-r1-distill-7b-q6_k.gguf -c 2048 |
| | ``` |
| | |
| | Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well. |
| | |
| | Step 1: Clone llama.cpp from GitHub. |
| | ``` |
| | git clone https://github.com/ggerganov/llama.cpp |
| | ``` |
| | |
| | Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux). |
| | ``` |
| | cd llama.cpp && LLAMA_CURL=1 make |
| | ``` |
| | |
| | Step 3: Run inference through the main binary. |
| | ``` |
| | ./llama-cli --hf-repo Triangle104/STAR1-R1-Distill-7B-Q6_K-GGUF --hf-file star1-r1-distill-7b-q6_k.gguf -p "The meaning to life and the universe is" |
| | ``` |
| | or |
| | ``` |
| | ./llama-server --hf-repo Triangle104/STAR1-R1-Distill-7B-Q6_K-GGUF --hf-file star1-r1-distill-7b-q6_k.gguf -c 2048 |
| | ``` |
| | |
