Initial release

.gitattributes

@@ -4,6 +4,7 @@
 *.bz2 filter=lfs diff=lfs merge=lfs -text
 *.ckpt filter=lfs diff=lfs merge=lfs -text
 *.ftz filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text

Llama-3.1-8B-Instruct-abliterated_via_adapter.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b51cd8c1d4865a1f098f8cefb53d7e53483208c83aa6584d0576715e46abdaa0
+size 4920734816

Llama-3.1-8B-Instruct-abliterated_via_adapter.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dae7272776730f5c1bd61e489c0ce9079c187fea5f5c0fb598e536f50b878bf8
+size 5732988000

Llama-3.1-8B-Instruct-abliterated_via_adapter.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0fe37c7c0850c19c7449c75e355f130fd7c7ffaaab7597c21d9dc35b52e30a9
+size 6596007008

Llama-3.1-8B-Instruct-abliterated_via_adapter.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83220aed42b2902bc083a6a086c643c2b959d0f12a1a76adb5e7ef28857405b3
+size 8540771424

README.md

@@ -1,3 +1,44 @@
 ---
+base_model:
+- meta-llama/Meta-Llama-3.1-8B-Instruct
+- grimjim/Llama-3-Instruct-abliteration-LoRA-8B
+library_name: transformers
+tags:
+- mergekit
+- merge
 license: llama3.1
+pipeline_tag: text-generation
 ---
+# Llama-3.1-8B-Instruct-abliterated_via_adapter-GGUF
+
+This repo contains select GGUF quants of a model that is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+A LoRA was applied to "abliterate" refusals in [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct). The result appears to work despite the LoRA having been derived from Llama 3 instead of Llama 3.1, which implies that there is significant feature commonality between the 3 and 3.1 models.
+
+The LoRA was extracted from [failspy/Meta-Llama-3-8B-Instruct-abliterated-v3](https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3) and uses [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) as a base.
+
+Built with Llama.
+
+## Merge Details
+### Merge Method
+
+This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B) as a base.
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
+dtype: bfloat16
+merge_method: task_arithmetic
+parameters:
+  normalize: false
+slices:
+- sources:
+  - layer_range: [0, 32]
+    model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
+    parameters:
+      weight: 1.0
+
+```