Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,286 @@
+---
+language:
+- en
+license: llama2
+tags:
+- code-generation
+- verilog
+- systemverilog
+- rtl
+- hardware-design
+- fifo
+- semiconductor
+- ip-design
+- lora
+- codellama
+base_model: codellama/CodeLlama-7B-Instruct-hf
+datasets:
+- Elinnos/verilog-fifo-dataset
+library_name: peft
+pipeline_tag: text-generation
+---
+
+# CodeLlama-7B Fine-Tuned for Verilog FIFO Generation
+
+## Model Description
+
+This is a **LoRA fine-tuned CodeLlama-7B-Instruct** model specialized in generating clean, synthesizable Verilog/SystemVerilog code for FIFO (First-In-First-Out) designs. The model was trained on comprehensive FIFO specifications following industry best practices for RTL design.
+
+**Model Type:** Causal Language Model (LoRA Adapter)  
+**Base Model:** [codellama/CodeLlama-7B-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7B-Instruct-hf)  
+**Training Date:** 2025-11-25  
+**Organization:** Elinnos Systems
+
+## Key Features
+
+✅ **Complete Code Generation** - Generates full Verilog modules with proper structure  
+✅ **Synthesizable RTL** - Clean, synthesizable code following best practices  
+✅ **FIFO Variants** - Supports various FIFO architectures (synchronous, asynchronous, parameterized)  
+✅ **Protocol Compliance** - Proper signal handling, full/empty flags, error handling  
+✅ **Production-Ready** - No comments, no debug statements, clean functional code  
+
+## Training Details
+
+### Dataset
+
+- **Training Samples:** 70
+- **Validation Samples:** 9
+- **Test Samples:** 15
+- **Total Samples:** 94
+- **Dataset Format:** CodeLlama chat template format
+- **Average Sequence Length:** 1536 tokens
+
+### Training Configuration
+
+- **Base Model:** codellama/CodeLlama-7B-Instruct-hf
+- **Training Method:** LoRA (Low-Rank Adaptation)
+- **Quantization:** 4-bit NF4 (for GPU memory efficiency)
+- **Training Steps:** 25 steps (5 epochs)
+- **Epochs:** 5
+- **Batch Size:** 4
+- **Effective Batch Size:** 16 (with gradient accumulation)
+- **Learning Rate:** 2e-05
+- **Max Sequence Length:** 1536
+- **Hardware:** NVIDIA A100-SXM4-40GB
+
+### LoRA Configuration
+
+```python
+from peft import LoraConfig
+
+LoraConfig(
+    r=48,                    # Rank
+    lora_alpha=96,  # Scaling factor
+    target_modules=['q_proj', 'up_proj', 'down_proj', 'gate_proj', 'o_proj', 'k_proj', 'v_proj'],
+    lora_dropout=0.15,
+    bias="none",
+    task_type="CAUSAL_LM"
+)
+```
+
+**Target Modules:**
+- `q_proj`
+- `up_proj`
+- `down_proj`
+- `gate_proj`
+- `o_proj`
+- `k_proj`
+- `v_proj`
+
+**Trainable Parameters:** ~672M (LoRA adapter only)
+
+## Model Capabilities
+
+### What This Model Can Do
+
+1. **Synchronous FIFO Design**
+   - Dual-port memory architecture
+   - Write/Read pointer management
+   - Full/Empty flag generation
+   - Configurable width and depth
+   - Error handling (write_err, read_err)
+   - Threshold flags (almost_full, almost_empty)
+   - Occupancy output
+
+2. **Asynchronous FIFO Design**
+   - Gray code pointer synchronization
+   - Clock domain crossing (CDC)
+   - Separate read/write clocks
+   - Data valid signals
+
+3. **Advanced Features**
+   - Parameterized FIFO modules
+   - AXI-like handshake protocols
+   - Pipelined output stages
+   - Peek capability
+   - Clear signal support
+
+4. **Code Quality**
+   - Clean, synthesizable RTL
+   - No comments or debug statements
+   - Proper signal naming conventions
+   - Modular design structure
+
+## Usage
+
+### Installation
+
+```bash
+pip install transformers peft torch bitsandbytes accelerate
+```
+
+### Loading the Model
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+from transformers import BitsAndBytesConfig
+
+# Load base model with 4-bit quantization
+base_model = AutoModelForCausalLM.from_pretrained(
+    "codellama/CodeLlama-7B-Instruct-hf",
+    quantization_config=BitsAndBytesConfig(
+        load_in_4bit=True,
+        bnb_4bit_quant_type="nf4",
+        bnb_4bit_compute_dtype=torch.float16,
+    ),
+    device_map="auto",
+    trust_remote_code=True
+)
+
+# Load fine-tuned LoRA adapter
+model = PeftModel.from_pretrained(
+    base_model, 
+    "Elinnos/codellama-7b-fifo-verilog"
+)
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("codellama/CodeLlama-7B-Instruct-hf")
+if tokenizer.pad_token is None:
+    tokenizer.pad_token = tokenizer.eos_token
+    tokenizer.pad_token_id = tokenizer.eos_token_id
+```
+
+### Generating Code
+
+```python
+# Create prompt using CodeLlama chat template
+system_prompt = "You are Elinnos RTL Code Generator v1.0, a specialized Verilog/SystemVerilog code generation agent. Your role: Generate clean, synthesizable RTL code for hardware design tasks. Output ONLY functional RTL code with no $display, assertions, comments, or debug statements."
+
+user_prompt = "Generate a synchronous FIFO with 8-bit data width, depth 4, write_enable, read_enable, full flag, empty flag."
+
+messages = [
+    {"role": "system", "content": system_prompt},
+    {"role": "user", "content": user_prompt}
+]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+# Generate
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=800,
+    temperature=0.3,
+    do_sample=True,
+    top_p=0.9,
+    repetition_penalty=1.2,
+    eos_token_id=tokenizer.eos_token_id
+)
+
+# Decode response
+response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=False)
+print(response)
+```
+
+### Using the Simple Inference Script
+
+```bash
+# From the codellama-migration directory
+python3 simple_finetuned_model_inference.py \
+    --prompt "Generate a synchronous FIFO with 32-bit data width, depth 16" \
+    --model-path "Elinnos/codellama-7b-fifo-verilog"
+```
+
+## Training Code
+
+The model was trained using the following repository:
+- **Repository:** [Elinnos/codellama-fifo-finetuning](https://huggingface.co/Elinnos/codellama-fifo-finetuning)
+- **Training Script:** `scripts/training/finetune_codellama.py`
+
+### Training Command
+
+```bash
+python scripts/training/finetune_codellama.py \
+    --base-model codellama/CodeLlama-7B-Instruct-hf \
+    --dataset datasets/processed/split_chat_format/train.jsonl \
+    --output-dir training-outputs/codellama-fifo-v2-chat \
+    --max-length 1536 \
+    --num-epochs 5 \
+    --batch-size 4 \
+    --gradient-accumulation 4 \
+    --learning-rate 2e-05 \
+    --lora-r 48 \
+    --lora-alpha 96 \
+    --lora-dropout 0.15
+```
+
+## Evaluation
+
+### Training Metrics
+
+- **Final Training Loss:** 0.626
+- **Final Validation Loss:** 0.609
+- **Training Loss Progression:**
+  - Epoch 1: 1.113
+  - Epoch 2: 0.996
+  - Epoch 3: 0.834
+  - Epoch 4: 0.704
+  - Epoch 5: 0.626
+
+### Model Performance
+
+The model demonstrates strong capability in:
+- Generating syntactically correct Verilog code
+- Following FIFO design specifications
+- Producing clean, synthesizable RTL
+- Handling various FIFO configurations (width, depth, features)
+
+## Limitations
+
+- Model size: Requires base CodeLlama-7B-Instruct model (~13GB)
+- Context window: Maximum sequence length of 1536 tokens
+- Domain: Specialized for FIFO designs, may not generalize to other RTL components
+- Language: Primarily English prompts
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@model{{codellama_fifo_elinnos}},
+  title={CodeLlama-7B Fine-Tuned for Verilog FIFO Generation},
+  author={Elinnos Systems},
+  year={2025},
+  url={https://huggingface.co/Elinnos/codellama-7b-fifo-verilog}
+}
+```
+
+## License
+
+This model follows the same license as the base CodeLlama-7B-Instruct model (Llama 2 Community License).
+
+## Contact
+
+For questions or issues, please contact:
+- **Organization:** Elinnos Systems
+- **Repository:** [Elinnos/codellama-fifo-finetuning](https://huggingface.co/Elinnos)
+
+---
+
+*Model trained by Elinnos Systems for Verilog FIFO code generation.*