GGUF usage with GPT4All

GPT4All is an open-source LLM application developed by Nomic. Version 2.7.2 introduces a brand new, experimental feature called Model Discovery.

Model Discovery provides a built-in way to search for and download GGUF models from the Hub. To get started, open GPT4All and click Download Models. From here, you can use the search bar to find a model.

After you have selected and downloaded a model, you can go to Settings and provide an appropriate prompt template in the GPT4All format (%1 and %2 placeholders).

Then from the main page, you can select the model from the list of installed models and start a conversation.

Update on GitHub