Hugging Face's logo
Matricardi Fabio's picture
11 37 415

Matricardi Fabio

FM-1976
Fishtiks's profile picture lucazsh's profile picture puettmann's profile picture
·
https://medium.com/@fabio.matricardi

AI & ML interests

control system engineering, AI, LLM with python. ThePoorGPUguy on substack

Recent Activity

liked a model 4 days ago
Tiiny/SmallThinker-3B-Preview
liked a model 8 days ago
LiquidAI/LFM2-2.6B-Exp
reacted to codelion's post with 🚀 8 days ago
Introducing Dhara-70M: A diffusion language model that achieves 3.8x higher throughput than autoregressive models! Key findings from our research on optimal architectures for small language models: → Depth beats width: 32 layers outperforms 12 layers at the same parameter count → Best-in-class factuality: 47.5% on TruthfulQA → 10x training efficiency using WSD (Warmup-Stable-Decay) conversion → Canon layers add only 0.13% parameters but improve reasoning We trained on 1B tokens using the optimal 50-30-20 dataset mix (PDFs + filtered web + educational content), then converted to diffusion with just 100M additional tokens. Blog: https://huggingface.co/blog/codelion/optimal-model-architecture Model: https://huggingface.co/codelion/dhara-70m
View all activity

Organizations

None yet