README.md

metadata

license: other
license_name: llama4
license_link: LICENSE
base_model:
  - meta-llama/Llama-4-Scout-17B-16E-Instruct
pipeline_tag: image-text-to-text
tags:
  - gguf-connector

llama-4-scout-17b-16e-instruct-gguf

• base model from meta-llama
• tested on gguf-connector with nightly llama-cpp-python

example workflow (run it locally)

• download the different parts of the model; for example q2_k
• llama-4-scout-17b-16e-it-q2_k-00001-of-00004.gguf
• llama-4-scout-17b-16e-it-q2_k-00002-of-00004.gguf
• llama-4-scout-17b-16e-it-q2_k-00003-of-00004.gguf
• llama-4-scout-17b-16e-it-q2_k-00004-of-00004.gguf
• pull them all into an empty folder; then execute the merge command: ggc m2
• the merged gguf is around 36.8GB for q2_k (setup once)
• execute connector command: ggc gpp or ggc cpp
• select the merged gguf then start your prompt to interact with llama4

for q4_0 and q8_0

• don't need to merge (linked already); just execute: ggc gpp (or ggc cpp for ui)
• select the first part of the model (i.e., 00001-of-xxxxx)
• start your prompt to interact with llama4