Update README.md
Browse files
README.md
CHANGED
|
@@ -33,7 +33,7 @@ This version doesn't have the parameter "g". It freezes the MLP and uses 7B logi
|
|
| 33 |
|
| 34 |
**ALL YOU NEED IS RWKV**
|
| 35 |
|
| 36 |
-
This is an **early preview** of our 7B parameter
|
| 37 |
|
| 38 |
- ✅ RWKV-7's efficient recurrence mechanism
|
| 39 |
- ✅ No self-attention, fully O(n)
|
|
|
|
| 33 |
|
| 34 |
**ALL YOU NEED IS RWKV**
|
| 35 |
|
| 36 |
+
This is an **early preview** of our 7B parameter RNN-based model, trained on 2k context length **(only stage-2 applied, without SFT or DPO)** through 3-stage knowledge distillation from Qwen2.5-7B-Instruct. While being a foundational version, it demonstrates:
|
| 37 |
|
| 38 |
- ✅ RWKV-7's efficient recurrence mechanism
|
| 39 |
- ✅ No self-attention, fully O(n)
|