Commit
·
8c92147
1
Parent(s):
c271818
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ tags:
|
|
| 11 |
# MPT-7B-Instruct
|
| 12 |
|
| 13 |
MPT-7B-Instruct is a model for short-form instruction following.
|
| 14 |
-
It is built by finetuning [MPT-7B
|
| 15 |
* License: _CC-By-SA-3.0_ (commercial use permitted)
|
| 16 |
* [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
|
| 17 |
|
|
@@ -99,10 +99,30 @@ For more details on the pretraining process, see [MPT-7B](https://huggingface.co
|
|
| 99 |
|
| 100 |
The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
|
| 101 |
|
| 102 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
-
This model was finetuned on 440 A100-40GBs for about half a day using the [MosaicML Platform](https://www.mosaicml.com/platform).
|
| 105 |
|
| 106 |
## Acknowledgements
|
| 107 |
|
| 108 |
-
This model was finetuned by Sam Havens and the MosaicML NLP team
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
# MPT-7B-Instruct
|
| 12 |
|
| 13 |
MPT-7B-Instruct is a model for short-form instruction following.
|
| 14 |
+
It is built by finetuning [MPT-7B](https://huggingface.co/spaces/mosaicml/mpt-7b) on a [dataset](https://huggingface.co/datasets/sam-mosaic/dolly_hhrlhf) derived from the [Databricks Dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and the [Anthropic Helpful and Harmless (HH-RLHF)](https://huggingface.co/datasets/Anthropic/hh-rlhf) datasets.
|
| 15 |
* License: _CC-By-SA-3.0_ (commercial use permitted)
|
| 16 |
* [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
|
| 17 |
|
|
|
|
| 99 |
|
| 100 |
The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
|
| 101 |
|
| 102 |
+
## Limitations and Biases
|
| 103 |
+
|
| 104 |
+
_The following language is modified from [EleutherAI's GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b)_
|
| 105 |
+
|
| 106 |
+
MPT-7B-Chat can produce factually incorrect output, and should not be relied on to produce factually accurate information.
|
| 107 |
+
MPT-7B-CHat was trained on various public datasets.
|
| 108 |
+
While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
|
| 109 |
|
|
|
|
| 110 |
|
| 111 |
## Acknowledgements
|
| 112 |
|
| 113 |
+
This model was finetuned by Sam Havens and the MosaicML NLP team
|
| 114 |
+
|
| 115 |
+
## Citation
|
| 116 |
+
|
| 117 |
+
Please cite this model using the following format:
|
| 118 |
+
|
| 119 |
+
```
|
| 120 |
+
@online{MosaicML2023Introducing,
|
| 121 |
+
author = {MosaicML NLP Team},
|
| 122 |
+
title = {Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs},
|
| 123 |
+
year = {2023},
|
| 124 |
+
url = {www.mosaicml.com/blog/mpt-7b},
|
| 125 |
+
note = {Accessed: 2023-03-28}, % change this date
|
| 126 |
+
urldate = {2023-03-28} % change this date
|
| 127 |
+
}
|
| 128 |
+
```
|