🔄 More intelligent

README.md

@@ -1,75 +1 @@
----
-license: mit
-language:
-  - en
-datasets:
-  - open-web-math/open-web-math
-  - vietgpt/the_pile_openwebtext2
-  - li2017dailydialog/daily_dialog
-  - OpenAssistant/oasst1
-tags:
-  - gpt2
-  - decoder-only
-  - from-scratch
-  - small-model
-  - conversational
-  - open-source
-  - transformers
----
 
-# 🧠 GPT-1.5 High
-
-**GPT-1.5 High** is a lightweight, from-scratch language model designed for text generation, conversational AI, and light reasoning tasks. It follows the GPT2 architecture but is specifically designed to reduce complexity and resource usage while maintaining strong language generation capabilities.
-
-## 🚧 Built From Scratch
-
-- **Architecture:** Decoder-only transformer (GPT-style)
-- **Layers:** 12 transformer blocks
-- **Hidden size:** 1024
-- **Heads:** 16 attention heads
-- **Context length:** 1024 tokens
-- **Total parameters:** ~200M
-- **Tokenizer:** Custom tokenizer, trained from scratch using a BPE (Byte Pair Encoding) approach.
-
-This model was carefully trained from the ground up, using a custom tokenizer to ensure optimal tokenization and vocabulary for the tasks at hand.
-
-## 🛠️ Training Setup
-
-This model was trained from scratch using the following resources:
-
-- **GPU:** NVIDIA A100 (40GB VRAM)
-- **Framework:** 🤗 Hugging Face Transformers + PyTorch
-- **Batching:** Gradient accumulation to handle larger context sequences
-- **Precision:** Mixed (fp16) and full (fp32) experiments
-- **Training time:** Several days (depending on GPU availability)
-
-## 📚 Datasets Used
-
-To create a diverse and capable language model, **GPT-1.5 High** was trained on a variety of datasets:
-
-- 🧮 `open-web-math/open-web-math` – Mathematical reasoning
-- 🌐 `vietgpt/the_pile_openwebtext2` – General web knowledge
-- 💬 `li2017dailydialog/daily_dialog` – Dialogue for everyday conversations
-- 🤖 `OpenAssistant/oasst1` – Assistant-style tasks and prompts
-
-These datasets provide a good mix of reasoning, everyday conversation, and general knowledge for diverse use cases.
-
-## 💡 Intended Use
-
-This model is ideal for:
-
-- Prototyping smaller, efficient language models
-- Educational and research purposes
-- Local/edge deployments where size and latency are critical
-- Conversational AI and chatbots
-
-⚠️ Please note: This model is **not fine-tuned** for safety, ethical guidelines, or content moderation. Use with caution in public-facing applications.
-
-## 🧪 Example Usage
-
-```python
-from transformers import pipeline
-
-generator = pipeline("text-generation", model="WolfInk/GPT-1.5-High", tokenizer="WolfInk/GPT-1.5-High")
-response = generator("Hey", max_length=30, do_sample=True, temperature=0.9)
-print(response[0]["generated_text"])