dwojcik/gemma4-26b-a4b-it-codex-gguf-4bit

Gemma 4 26B Codex (GGUF 4-bit)

Goal: The ultimate goal of this project is to create the best Gemma 4 coder model available.

This model is a highly fine-tuned version of google/gemma-4-26B-A4B-it, optimized heavily on complex programming and software engineering tasks (such as the Evol-Instruct-Code dataset). It has been specifically quantized and converted to the GGUF format at 4-bit precision (q4_k_m), making it widely compatible with Windows, Linux, and Mac setups through tools like LM Studio, Ollama, and llama.cpp.

Key Features

• Unmatched Coding Ability: Fine-tuned specifically for reasoning, complex debugging, algorithmic generation, and software architecture.
• Universal GGUF Format: Compatible with almost any modern local LLM runner (llama.cpp, LM Studio, Text Generation WebUI).
• 4-bit Quantization: Uses the q4_k_m quantization method to squeeze the massive 26B parameter intelligence into a memory footprint that comfortably runs on 16GB+ RAM setups while preserving high precision.

How to use with LM Studio

1. Download and install LM Studio.
2. In the search bar, look for this repository.
3. Download the .gguf file.
4. Load the model and start chatting!

How to use with llama.cpp

./main -m gemma4-26b-a4b-it-codex-unsloth-Q4_K_M.gguf -n 512 --color -i -cml -f prompts/chat-with-bob.txt

Training Details

• Base Model: google/gemma-4-26B-A4B-it
• Dataset: Evol-Instruct-Code-80k-v1
• Method: QLoRA via Unsloth (Rank 16, Alpha 32)
• Epochs: 3.0