E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
Paper • 2609.37533 • Published • 67
E-MoE
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models. LM1B checkpoints: E-MoE and the MDLM / MDLM-MoE baselines.
-
ArsenyIvanov/emoe-lm1b
Text Generation • 0.5B • Updated • 8Note E-MoE: MoE routes as a discrete shared latent (536M total / 139M active).
ArsenyIvanov/mdlm-moe-lm1b
Text Generation • 0.5B • Updated • 6Note Capacity-matched baseline: same MoE backbone, deterministic routing.
ArsenyIvanov/mdlm-lm1b
Text Generation • 0.1B • Updated • 7Note Factorized baseline: dense DiT (139M).