binxx71/aift-ncm-deepseek-1.41b-sft

AIFT-NCM-DeepSeek-1.41B (SFT GGUF Checkpoint)

This repository contains the Supervised Fine-Tuned (SFT) checkpoint formatted in GGUF for AIFT-NCM-DeepSeek-1.41B, an independently developed 1.41-billion parameter hybrid transformer architecture created by Binyam.

Files Included

• aift_1.41b_sft.gguf: GGUF binary container packing all model weights, hyperparameters, and tensor metadata.
• aift_1.41b_sft_checkpoint.pt: Native PyTorch state dictionary checkpoint.

Architectural Highlights

• Parameter Scale: 1.41 Billion parameters across 24 transformer blocks ($d_{model} = 2048$, 16 attention heads, $d_{ff} = 8192$).
• Attention Mechanism: Scaled DeepSeek Multi-Head Latent Attention (MLA) featuring low-rank KV compression ($d_c = 512$) to reduce cache memory overhead.
• Tokenization & Output: Next-Concept Modeling (NCM) multiscale parallel output heads ($L=4$) predicting multi-resolution semantic features.
• Specification Alignment: Built upon AI Intent Fidelity Theory (AIFT), mapping natural language prompts into structured tuples ($\mathcal{O}, \mathcal{C}, \mathcal{P}, \mathcal{V}$).

Training Specs

• Pre-training Dataset: HuggingFaceFW/fineweb-edu
• SFT Dataset: yahma/alpaca-cleaned (Formatted under AIFT instruction blocks)
• Hardware Target: Single NVIDIA Tesla T4 (15 GB VRAM) using 8-bit AdamW quantization (bitsandbytes), gradient checkpointing, mixed precision (torch.amp), and disk memory-mapping (mmap=True).