Pankaj Mathur
commited on
Commit
·
15cdc0d
1
Parent(s):
9aea87a
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,22 @@ datasets:
|
|
| 13 |
|
| 14 |
An **Uncensored** LLaMA-7b model trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.
|
| 15 |
|
| 16 |
-
Please note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
# Dataset
|
| 19 |
|
|
@@ -101,8 +116,10 @@ print(generate_text(system, instruction))
|
|
| 101 |
|
| 102 |
```
|
| 103 |
|
| 104 |
-
|
|
|
|
| 105 |
|
|
|
|
| 106 |
[!] Response:
|
| 107 |
Breaking into your own car requires certain skills and tools. Here are the basic steps:
|
| 108 |
|
|
@@ -113,9 +130,6 @@ Breaking into your own car requires certain skills and tools. Here are the basic
|
|
| 113 |
5. If the ^^^^^^^^^^^^^.
|
| 114 |
|
| 115 |
```
|
| 116 |
-
|
| 117 |
-
**P.S. I am #opentowork and #collaboration, if you can help, please reach out to me at www.linkedin.com/in/pankajam**
|
| 118 |
-
|
| 119 |
**
|
| 120 |
|
| 121 |
Next Goals:
|
|
@@ -140,9 +154,9 @@ Citiation:
|
|
| 140 |
If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or applications, please kindly cite using the following BibTeX:
|
| 141 |
|
| 142 |
```
|
| 143 |
-
@misc{
|
| 144 |
author = {Pankaj Mathur},
|
| 145 |
-
title = {
|
| 146 |
year = {2023},
|
| 147 |
publisher = {GitHub, HuggingFace},
|
| 148 |
journal = {GitHub repository, HuggingFace repository},
|
|
@@ -150,12 +164,11 @@ If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or
|
|
| 150 |
}
|
| 151 |
```
|
| 152 |
```
|
| 153 |
-
@software{
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
year
|
| 158 |
-
url = {https://github.com/openlm-research/open_llama}
|
| 159 |
}
|
| 160 |
```
|
| 161 |
```
|
|
@@ -177,4 +190,23 @@ If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or
|
|
| 177 |
journal = {GitHub repository},
|
| 178 |
howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
|
| 179 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
```
|
|
|
|
| 13 |
|
| 14 |
An **Uncensored** LLaMA-7b model trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.
|
| 15 |
|
| 16 |
+
Please note this model has *better code generation capabilities* compare to our original orca_mini_7b which was trained on base OpenLLaMA-7b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
**P.S. I am #opentowork, if you can help, please reach out to me at www.linkedin.com/in/pankajam**
|
| 20 |
+
|
| 21 |
+
# Evaluation
|
| 22 |
+
|
| 23 |
+
|||||||
|
| 24 |
+
|:------:|:-------------:|:---------:|:--------:|:-------:|:--------:|
|
| 25 |
+
|**Task**|**num_fewshot**|**Version**|**Metric**|**Value**|**Stderr**|
|
| 26 |
+
|*arc_easy*|0|0|acc|0.7386|0.0090|
|
| 27 |
+
|*arc_easy*|0|0|acc_norm|0.7066|0.0093|
|
| 28 |
+
|*hellaswag*|0|0|acc|0.5591|0.0050|
|
| 29 |
+
|*hellaswag*|0|0|acc_norm|0.7394|0.0044|
|
| 30 |
+
|*truthfulqa_mc*|0|1|mc1|0.2938|0.0159|
|
| 31 |
+
|*truthfulqa_mc*|0|1|mc2|0.4399|0.0153|
|
| 32 |
|
| 33 |
# Dataset
|
| 34 |
|
|
|
|
| 116 |
|
| 117 |
```
|
| 118 |
|
| 119 |
+
**NOTE: The real response is hided here with ^^^^^^^^^^^^^.**
|
| 120 |
+
*Try on your own private machine to see uncensored responses*
|
| 121 |
|
| 122 |
+
```
|
| 123 |
[!] Response:
|
| 124 |
Breaking into your own car requires certain skills and tools. Here are the basic steps:
|
| 125 |
|
|
|
|
| 130 |
5. If the ^^^^^^^^^^^^^.
|
| 131 |
|
| 132 |
```
|
|
|
|
|
|
|
|
|
|
| 133 |
**
|
| 134 |
|
| 135 |
Next Goals:
|
|
|
|
| 154 |
If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or applications, please kindly cite using the following BibTeX:
|
| 155 |
|
| 156 |
```
|
| 157 |
+
@misc{orca_mini_v2_7b,
|
| 158 |
author = {Pankaj Mathur},
|
| 159 |
+
title = {orca_mini_v2_7b: An explain tuned LLaMA-7b model on uncensored wizardlm, alpaca, & dolly datasets},
|
| 160 |
year = {2023},
|
| 161 |
publisher = {GitHub, HuggingFace},
|
| 162 |
journal = {GitHub repository, HuggingFace repository},
|
|
|
|
| 164 |
}
|
| 165 |
```
|
| 166 |
```
|
| 167 |
+
@software{touvron2023llama,
|
| 168 |
+
title={LLaMA: Open and Efficient Foundation Language Models},
|
| 169 |
+
author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
|
| 170 |
+
journal={arXiv preprint arXiv:2302.13971},
|
| 171 |
+
year={2023}
|
|
|
|
| 172 |
}
|
| 173 |
```
|
| 174 |
```
|
|
|
|
| 190 |
journal = {GitHub repository},
|
| 191 |
howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
|
| 192 |
}
|
| 193 |
+
```
|
| 194 |
+
```
|
| 195 |
+
@online{DatabricksBlog2023DollyV2,
|
| 196 |
+
author = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin},
|
| 197 |
+
title = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM},
|
| 198 |
+
year = {2023},
|
| 199 |
+
url = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm},
|
| 200 |
+
urldate = {2023-06-30}
|
| 201 |
+
}
|
| 202 |
+
```
|
| 203 |
+
```
|
| 204 |
+
@misc{xu2023wizardlm,
|
| 205 |
+
title={WizardLM: Empowering Large Language Models to Follow Complex Instructions},
|
| 206 |
+
author={Can Xu and Qingfeng Sun and Kai Zheng and Xiubo Geng and Pu Zhao and Jiazhan Feng and Chongyang Tao and Daxin Jiang},
|
| 207 |
+
year={2023},
|
| 208 |
+
eprint={2304.12244},
|
| 209 |
+
archivePrefix={arXiv},
|
| 210 |
+
primaryClass={cs.CL}
|
| 211 |
+
}
|
| 212 |
```
|