🧠 SALAMA LLM — Swahili Instruction-Tuned Text Generation Model

👨‍💻 Developer: AI4NNOV
✍️ Authors: AI4NNOV
📦 Version: v1.0
📜 License: Apache 2.0
🛠️ Model Type: Instruction-Tuned Large Language Model
🧩 Base Model: Jacaranda/UlizaLlama

🌍 Overview

SALAMA LLM is the language understanding and generation engine of the SALAMA Framework — a modular Speech-to-Speech (STS) AI pipeline built for African languages.
The model is fine-tuned on Swahili instruction datasets to enable natural, culturally relevant responses in text generation, summarization, question answering, and translation.

This model represents a major step in bridging the linguistic digital divide by providing high-quality Swahili AI text generation capabilities within an open, scalable framework.

🧱️ Model Architecture

SALAMA LLM is based on Jacaranda/UlizaLlama, fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) via LoRA/QLoRA.
The architecture supports mixed Swahili-English text inputs while focusing on fluent Swahili text generation for both casual and formal domains.

Parameter	Value
Base Model	`Jacaranda/UlizaLlama`
Fine-Tuning	QLoRA / LoRA (PEFT)
Precision	4-bit quantization
Optimizer	AdamW
Learning Rate	2e-5
Epochs	3–5
Frameworks	Transformers, TRL, PEFT, Unsloth
Languages	Swahili (sw), English (en)

📚 Datasets

Dataset	Description	Purpose
`saillab/alpaca_swahili_taco`	Swahili Alpaca-style instruction-response dataset	Instruction tuning
`Jacaranda/kiswallama-pretrained`	321M Swahili tokens, custom tokenizer (20K vocab)	Base Swahili adaptation
Custom Swahili QA corpus	Curated Q&A and summarization samples	Conversational fine-tuning

🧠 Model Capabilities

✅ Text generation in Swahili and English
✅ Instruction-following, summarization, and dialogue
✅ Question answering and translation (EN ↔ SW)
✅ Sentiment and named-entity recognition
✅ Contextually and culturally aligned text generation

📊 Evaluation Metrics

Metric	Score	Description
BLEU	0.49	Measures fluency and translation accuracy
ROUGE-L	0.61	Summarization recall and overlap
Accuracy (QA)	95.5%	Accuracy on Swahili QA tasks
CER	0.28	Character Error Rate
F1 (avg)	0.90+	Weighted average across tasks

⚙️ Usage (Python Example)

Below is a quick example to load and use SALAMA LLM for Swahili text generation:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "EYEDOL/salama-llm"  # Change to your Hugging Face repo name
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Swahili text prompt
prompt = "Andika sentensi fupi kuhusu umuhimu wa elimu."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=120,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.05
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

🦩 Example Output:

“Elimu ni msingi wa maendeleo, humwezesha mtu kuelewa dunia na kuboresha maisha yake na jamii kwa ujumla.”

⚡ Key Features

🧩 Optimized for African low-resource NLP contexts
💬 Instruction-following in Swahili and English
⚙️ Lightweight and efficient (QLoRA fine-tuned; runs on single 24 GB GPU)
🌍 Culturally aligned text generation
🦶 Open-source and extendable to other African languages

🚫 Limitations

⚠️ May underperform with heavy code-switching (Swahili-English mix)
👤 Not yet optimized for rare dialects or poetic forms
📚 Limited exposure to specialized (medical/legal) corpora
🔊 Relies on accurate STT transcription in end-to-end speech-to-speech use

🔗 Related Models

Model	Description
`EYEDOL/salama-stt`	Swahili Speech-to-Text model (Whisper-small fine-tuned)
`EYEDOL/salama-tts`	Swahili Text-to-Speech model (VITS architecture)

🧾 Citation

If you use SALAMA LLM, please cite:

@misc{salama_llm_2025,
  title={SALAMA LLM: Swahili Instruction-Tuned Text Generation Model},
  author={AI4NNOV},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/EYEDOL/salama-llm}}
}

💡 “Elimu ni msingi wa maendeleo — Knowledge is the foundation of progress.”

Downloads last month: -; Downloads are not tracked for this model. How to track

EYEDOL
/

SALAMA_LLM