tokenizer.json missing
#39
by
s3dev-ai
- opened
When converting this lovely model to GGUF (via llama_cpp) I get an error saying the tokenizer.json file is missing. After re-examining the HF repo, there isn't a tokenizer.json file in the repo; although there is one for the other models Mistral I searched up.
Is this file actually missing, or do I need to do something different when converting to GGUF, as I see others have done it successfully?
Alternatively, can I use the tokenizer.json file from the base model?