Add tokenizer.json
#3
by jonatanklosko - opened
Generated with:
from transformers import BlenderbotTokenizerFast
tokenizer = BlenderbotTokenizerFast.from_pretrained("/Users/jonatanklosko/git/hf/blenderbot-400M-distill/")
tokenizer.save_pretrained("/Users/jonatanklosko/git/hf/blenderbot-400M-distill/")
jonatanklosko changed pull request status to open
Works for me!
patrickvonplaten changed pull request status to merged