VERY cool!
Thanks for doing this huge run for all of us, thanks for maintaining apache-2.0, this is really rad. Heretic rules.
I'm wondering if you're at all interested in doing a similarly (actually even more) daunting run by checking out MiniMax M2?
There's the original weights, but Cerebras has also been REAPing away at the model, and just released https://huggingface.co/cerebras/MiniMax-M2-REAP-139B-A10B which is fairly close to the 120b you've already conquered! Would be VERY cool to see.
Regardless, thank you for this contribution!!
I'll take a look at it :)
@CyborgPaloma
When running cerebras/MiniMax-M2-REAP-139B-A10B I already only get 1/100 refusals, so I don't think it makes sense to decensor it, as it seems to already comply pretty well.
WOW that is extremely surprising but also I appreciate your work! Sorry I didn't check into it a little more before asking. Cheers and thanks! @kldzj