---
license: bsd-3-clause
---

Modification of the base ProGen2-large model by [Nijkamp, et al.](https://arxiv.org/abs/2206.13517). The vocab_size is changed from 51200 to 32, the same as the ProGen2-small and ProGen2-medium. The model follows the original code of [Progen2](https://github.com/salesforce/progen/tree/main/progen2).  


Example usage:

```python
from models.modeling_progen import ProGenForCausalLM
from tokenizers import Tokenizer
import torch

# load model and tokenizer
model = ProGenForCausalLM.from_pretrained("IDEA-XL/progen2-large", torch_dtype="auto")
tokenizer = Tokenizer.from_pretrained("IDEA-XL/progen2-large")
tokenizer.no_padding()

# prepare input
prompt = "1MEVVIVTGMSGAGK"
input_ids = torch.tensor(tokenizer.encode(prompt).ids).to(model.device)

# forward
logits = model(input_ids).logits