FP8 doesn't load properly?
Hi! Thank you for FP8 version! However, when I load it, either as ckpt (as in example workflow) or as diffusion model, it gives me an error, and the result is just plain black image:
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type FLOW
unet unexpected: ['t_embedder.mlp.0.scale_weight', 't_embedder.mlp.2.scale_weight', 'cond_seq_linear.scale_weight', 'init_x_linear.scale_weight', 'double_layers.0.mlpC.c_fc1.scale_weight', 'double_layers.0.mlpC.c_fc2.scale_weight', 'double_layers.0.mlpC.c_proj.scale_weight', 'double_layers.0.modC.1.scale_weight', 'double_layers.0.mlpX.c_fc1.scale_weight', 'double_layers.0.mlpX.c_fc2.scale_weight', 'double_layers.0.mlpX.c_proj.scale_weight', 'double_layers.0.modX.1.scale_weight', 'double_layers.0.attn.w1q.scale_weight', 'double_layers.0.attn.w1k.scale_weight', 'double_layers.0.attn.w1v.scale_weight', 'double_layers.0.attn.w1o.scale_weight', 'double_layers.0.attn.w2q.scale_weight', 'double_layers.0.attn.w2k.scale_weight', 'double_layers.0.attn.w2v.scale_weight', 'double_layers.0.attn.w2o.scale_weight', 'double_layers.1.mlpC.c_fc1.scale_weight', 'double_layers.1.mlpC.c_fc2.scale_weight', 'double_layers.1.mlpC.c_proj.scale_weight', 'double_layers.1.modC.1.scale_weight', 'double_layers.1.mlpX.c_fc1.scale_weight', 'double_layers.1.mlpX.c_fc2.scale_weight', 'double_layers.1.mlpX.c_proj.scale_weight', 'double_layers.1.modX.1.scale_weight', 'double_layers.1.attn.w1q.scale_weight', 'double_layers.1.attn.w1k.scale_weight', 'double_layers.1.attn.w1v.scale_weight', 'double_layers.1.attn.w1o.scale_weight', 'double_layers.1.attn.w2q.scale_weight', 'double_layers.1.attn.w2k.scale_weight', 'double_layers.1.attn.w2v.scale_weight', 'double_layers.1.attn.w2o.scale_weight', 'double_layers.2.mlpC.c_fc1.scale_weight', 'double_layers.2.mlpC.c_fc2.scale_weight', 'double_layers.2.mlpC.c_proj.scale_weight', 'double_layers.2.modC.1.scale_weight', 'double_layers.2.mlpX.c_fc1.scale_weight', 'double_layers.2.mlpX.c_fc2.scale_weight', 'double_layers.2.mlpX.c_proj.scale_weight', 'double_layers.2.modX.1.scale_weight', 'double_layers.2.attn.w1q.scale_weight', 'double_layers.2.attn.w1k.scale_weight', 'double_layers.2.attn.w1v.scale_weight', 'double_layers.2.attn.w1o.scale_weight', 'double_layers.2.attn.w2q.scale_weight', 'double_layers.2.attn.w2k.scale_weight', 'double_layers.2.attn.w2v.scale_weight', 'double_layers.2.attn.w2o.scale_weight', 'double_layers.3.mlpC.c_fc1.scale_weight', 'double_layers.3.mlpC.c_fc2.scale_weight', 'double_layers.3.mlpC.c_proj.scale_weight', 'double_layers.3.modC.1.scale_weight', 'double_layers.3.mlpX.c_fc1.scale_weight', 'double_layers.3.mlpX.c_fc2.scale_weight', 'double_layers.3.mlpX.c_proj.scale_weight', 'double_layers.3.modX.1.scale_weight', 'double_layers.3.attn.w1q.scale_weight', 'double_layers.3.attn.w1k.scale_weight', 'double_layers.3.attn.w1v.scale_weight', 'double_layers.3.attn.w1o.scale_weight', 'double_layers.3.attn.w2q.scale_weight', 'double_layers.3.attn.w2k.scale_weight', 'double_layers.3.attn.w2v.scale_weight', 'double_layers.3.attn.w2o.scale_weight', 'single_layers.0.modCX.1.scale_weight', 'single_layers.0.attn.w1q.scale_weight', 'single_layers.0.attn.w1k.scale_weight', 'single_layers.0.attn.w1v.scale_weight', 'single_layers.0.attn.w1o.scale_weight', 'single_layers.0.mlp.c_fc1.scale_weight', 'single_layers.0.mlp.c_fc2.scale_weight', 'single_layers.0.mlp.c_proj.scale_weight', 'single_layers.1.modCX.1.scale_weight', 'single_layers.1.attn.w1q.scale_weight', 'single_layers.1.attn.w1k.scale_weight', 'single_layers.1.attn.w1v.scale_weight', 'single_layers.1.attn.w1o.scale_weight', 'single_layers.1.mlp.c_fc1.scale_weight', 'single_layers.1.mlp.c_fc2.scale_weight', 'single_layers.1.mlp.c_proj.scale_weight', 'single_layers.2.modCX.1.scale_weight', 'single_layers.2.attn.w1q.scale_weight', 'single_layers.2.attn.w1k.scale_weight', 'single_layers.2.attn.w1v.scale_weight', 'single_layers.2.attn.w1o.scale_weight', 'single_layers.2.mlp.c_fc1.scale_weight', 'single_layers.2.mlp.c_fc2.scale_weight', 'single_layers.2.mlp.c_proj.scale_weight', 'single_layers.3.modCX.1.scale_weight', 'single_layers.3.attn.w1q.scale_weight', 'single_layers.3.attn.w1k.scale_weight', 'single_layers.3.attn.w1v.scale_weight', 'single_layers.3.attn.w1o.scale_weight', 'single_layers.3.mlp.c_fc1.scale_weight', 'single_layers.3.mlp.c_fc2.scale_weight', 'single_layers.3.mlp.c_proj.scale_weight', 'single_layers.4.modCX.1.scale_weight', 'single_layers.4.attn.w1q.scale_weight', 'single_layers.4.attn.w1k.scale_weight', 'single_layers.4.attn.w1v.scale_weight', 'single_layers.4.attn.w1o.scale_weight', 'single_layers.4.mlp.c_fc1.scale_weight', 'single_layers.4.mlp.c_fc2.scale_weight', 'single_layers.4.mlp.c_proj.scale_weight', 'single_layers.5.modCX.1.scale_weight', 'single_layers.5.attn.w1q.scale_weight', 'single_layers.5.attn.w1k.scale_weight', 'single_layers.5.attn.w1v.scale_weight', 'single_layers.5.attn.w1o.scale_weight', 'single_layers.5.mlp.c_fc1.scale_weight', 'single_layers.5.mlp.c_fc2.scale_weight', 'single_layers.5.mlp.c_proj.scale_weight', 'single_layers.6.modCX.1.scale_weight', 'single_layers.6.attn.w1q.scale_weight', 'single_layers.6.attn.w1k.scale_weight', 'single_layers.6.attn.w1v.scale_weight', 'single_layers.6.attn.w1o.scale_weight', 'single_layers.6.mlp.c_fc1.scale_weight', 'single_layers.6.mlp.c_fc2.scale_weight', 'single_layers.6.mlp.c_proj.scale_weight', 'single_layers.7.modCX.1.scale_weight', 'single_layers.7.attn.w1q.scale_weight', 'single_layers.7.attn.w1k.scale_weight', 'single_layers.7.attn.w1v.scale_weight', 'single_layers.7.attn.w1o.scale_weight', 'single_layers.7.mlp.c_fc1.scale_weight', 'single_layers.7.mlp.c_fc2.scale_weight', 'single_layers.7.mlp.c_proj.scale_weight', 'single_layers.8.modCX.1.scale_weight', 'single_layers.8.attn.w1q.scale_weight', 'single_layers.8.attn.w1k.scale_weight', 'single_layers.8.attn.w1v.scale_weight', 'single_layers.8.attn.w1o.scale_weight', 'single_layers.8.mlp.c_fc1.scale_weight', 'single_layers.8.mlp.c_fc2.scale_weight', 'single_layers.8.mlp.c_proj.scale_weight', 'single_layers.9.modCX.1.scale_weight', 'single_layers.9.attn.w1q.scale_weight', 'single_layers.9.attn.w1k.scale_weight', 'single_layers.9.attn.w1v.scale_weight', 'single_layers.9.attn.w1o.scale_weight', 'single_layers.9.mlp.c_fc1.scale_weight', 'single_layers.9.mlp.c_fc2.scale_weight', 'single_layers.9.mlp.c_proj.scale_weight', 'single_layers.10.modCX.1.scale_weight', 'single_layers.10.attn.w1q.scale_weight', 'single_layers.10.attn.w1k.scale_weight', 'single_layers.10.attn.w1v.scale_weight', 'single_layers.10.attn.w1o.scale_weight', 'single_layers.10.mlp.c_fc1.scale_weight', 'single_layers.10.mlp.c_fc2.scale_weight', 'single_layers.10.mlp.c_proj.scale_weight', 'single_layers.11.modCX.1.scale_weight', 'single_layers.11.attn.w1q.scale_weight', 'single_layers.11.attn.w1k.scale_weight', 'single_layers.11.attn.w1v.scale_weight', 'single_layers.11.attn.w1o.scale_weight', 'single_layers.11.mlp.c_fc1.scale_weight', 'single_layers.11.mlp.c_fc2.scale_weight', 'single_layers.11.mlp.c_proj.scale_weight', 'single_layers.12.modCX.1.scale_weight', 'single_layers.12.attn.w1q.scale_weight', 'single_layers.12.attn.w1k.scale_weight', 'single_layers.12.attn.w1v.scale_weight', 'single_layers.12.attn.w1o.scale_weight', 'single_layers.12.mlp.c_fc1.scale_weight', 'single_layers.12.mlp.c_fc2.scale_weight', 'single_layers.12.mlp.c_proj.scale_weight', 'single_layers.13.modCX.1.scale_weight', 'single_layers.13.attn.w1q.scale_weight', 'single_layers.13.attn.w1k.scale_weight', 'single_layers.13.attn.w1v.scale_weight', 'single_layers.13.attn.w1o.scale_weight', 'single_layers.13.mlp.c_fc1.scale_weight', 'single_layers.13.mlp.c_fc2.scale_weight', 'single_layers.13.mlp.c_proj.scale_weight', 'single_layers.14.modCX.1.scale_weight', 'single_layers.14.attn.w1q.scale_weight', 'single_layers.14.attn.w1k.scale_weight', 'single_layers.14.attn.w1v.scale_weight', 'single_layers.14.attn.w1o.scale_weight', 'single_layers.14.mlp.c_fc1.scale_weight', 'single_layers.14.mlp.c_fc2.scale_weight', 'single_layers.14.mlp.c_proj.scale_weight', 'single_layers.15.modCX.1.scale_weight', 'single_layers.15.attn.w1q.scale_weight', 'single_layers.15.attn.w1k.scale_weight', 'single_layers.15.attn.w1v.scale_weight', 'single_layers.15.attn.w1o.scale_weight', 'single_layers.15.mlp.c_fc1.scale_weight', 'single_layers.15.mlp.c_fc2.scale_weight', 'single_layers.15.mlp.c_proj.scale_weight', 'single_layers.16.modCX.1.scale_weight', 'single_layers.16.attn.w1q.scale_weight', 'single_layers.16.attn.w1k.scale_weight', 'single_layers.16.attn.w1v.scale_weight', 'single_layers.16.attn.w1o.scale_weight', 'single_layers.16.mlp.c_fc1.scale_weight', 'single_layers.16.mlp.c_fc2.scale_weight', 'single_layers.16.mlp.c_proj.scale_weight', 'single_layers.17.modCX.1.scale_weight', 'single_layers.17.attn.w1q.scale_weight', 'single_layers.17.attn.w1k.scale_weight', 'single_layers.17.attn.w1v.scale_weight', 'single_layers.17.attn.w1o.scale_weight', 'single_layers.17.mlp.c_fc1.scale_weight', 'single_layers.17.mlp.c_fc2.scale_weight', 'single_layers.17.mlp.c_proj.scale_weight', 'single_layers.18.modCX.1.scale_weight', 'single_layers.18.attn.w1q.scale_weight', 'single_layers.18.attn.w1k.scale_weight', 'single_layers.18.attn.w1v.scale_weight', 'single_layers.18.attn.w1o.scale_weight', 'single_layers.18.mlp.c_fc1.scale_weight', 'single_layers.18.mlp.c_fc2.scale_weight', 'single_layers.18.mlp.c_proj.scale_weight', 'single_layers.19.modCX.1.scale_weight', 'single_layers.19.attn.w1q.scale_weight', 'single_layers.19.attn.w1k.scale_weight', 'single_layers.19.attn.w1v.scale_weight', 'single_layers.19.attn.w1o.scale_weight', 'single_layers.19.mlp.c_fc1.scale_weight', 'single_layers.19.mlp.c_fc2.scale_weight', 'single_layers.19.mlp.c_proj.scale_weight', 'single_layers.20.modCX.1.scale_weight', 'single_layers.20.attn.w1q.scale_weight', 'single_layers.20.attn.w1k.scale_weight', 'single_layers.20.attn.w1v.scale_weight', 'single_layers.20.attn.w1o.scale_weight', 'single_layers.20.mlp.c_fc1.scale_weight', 'single_layers.20.mlp.c_fc2.scale_weight', 'single_layers.20.mlp.c_proj.scale_weight', 'single_layers.21.modCX.1.scale_weight', 'single_layers.21.attn.w1q.scale_weight', 'single_layers.21.attn.w1k.scale_weight', 'single_layers.21.attn.w1v.scale_weight', 'single_layers.21.attn.w1o.scale_weight', 'single_layers.21.mlp.c_fc1.scale_weight', 'single_layers.21.mlp.c_fc2.scale_weight', 'single_layers.21.mlp.c_proj.scale_weight', 'single_layers.22.modCX.1.scale_weight', 'single_layers.22.attn.w1q.scale_weight', 'single_layers.22.attn.w1k.scale_weight', 'single_layers.22.attn.w1v.scale_weight', 'single_layers.22.attn.w1o.scale_weight', 'single_layers.22.mlp.c_fc1.scale_weight', 'single_layers.22.mlp.c_fc2.scale_weight', 'single_layers.22.mlp.c_proj.scale_weight', 'single_layers.23.modCX.1.scale_weight', 'single_layers.23.attn.w1q.scale_weight', 'single_layers.23.attn.w1k.scale_weight', 'single_layers.23.attn.w1v.scale_weight', 'single_layers.23.attn.w1o.scale_weight', 'single_layers.23.mlp.c_fc1.scale_weight', 'single_layers.23.mlp.c_fc2.scale_weight', 'single_layers.23.mlp.c_proj.scale_weight', 'single_layers.24.modCX.1.scale_weight', 'single_layers.24.attn.w1q.scale_weight', 'single_layers.24.attn.w1k.scale_weight', 'single_layers.24.attn.w1v.scale_weight', 'single_layers.24.attn.w1o.scale_weight', 'single_layers.24.mlp.c_fc1.scale_weight', 'single_layers.24.mlp.c_fc2.scale_weight', 'single_layers.24.mlp.c_proj.scale_weight', 'single_layers.25.modCX.1.scale_weight', 'single_layers.25.attn.w1q.scale_weight', 'single_layers.25.attn.w1k.scale_weight', 'single_layers.25.attn.w1v.scale_weight', 'single_layers.25.attn.w1o.scale_weight', 'single_layers.25.mlp.c_fc1.scale_weight', 'single_layers.25.mlp.c_fc2.scale_weight', 'single_layers.25.mlp.c_proj.scale_weight', 'single_layers.26.modCX.1.scale_weight', 'single_layers.26.attn.w1q.scale_weight', 'single_layers.26.attn.w1k.scale_weight', 'single_layers.26.attn.w1v.scale_weight', 'single_layers.26.attn.w1o.scale_weight', 'single_layers.26.mlp.c_fc1.scale_weight', 'single_layers.26.mlp.c_fc2.scale_weight', 'single_layers.26.mlp.c_proj.scale_weight', 'single_layers.27.modCX.1.scale_weight', 'single_layers.27.attn.w1q.scale_weight', 'single_layers.27.attn.w1k.scale_weight', 'single_layers.27.attn.w1v.scale_weight', 'single_layers.27.attn.w1o.scale_weight', 'single_layers.27.mlp.c_fc1.scale_weight', 'single_layers.27.mlp.c_fc2.scale_weight', 'single_layers.27.mlp.c_proj.scale_weight', 'single_layers.28.modCX.1.scale_weight', 'single_layers.28.attn.w1q.scale_weight', 'single_layers.28.attn.w1k.scale_weight', 'single_layers.28.attn.w1v.scale_weight', 'single_layers.28.attn.w1o.scale_weight', 'single_layers.28.mlp.c_fc1.scale_weight', 'single_layers.28.mlp.c_fc2.scale_weight', 'single_layers.28.mlp.c_proj.scale_weight', 'single_layers.29.modCX.1.scale_weight', 'single_layers.29.attn.w1q.scale_weight', 'single_layers.29.attn.w1k.scale_weight', 'single_layers.29.attn.w1v.scale_weight', 'single_layers.29.attn.w1o.scale_weight', 'single_layers.29.mlp.c_fc1.scale_weight', 'single_layers.29.mlp.c_fc2.scale_weight', 'single_layers.29.mlp.c_proj.scale_weight', 'single_layers.30.modCX.1.scale_weight', 'single_layers.30.attn.w1q.scale_weight', 'single_layers.30.attn.w1k.scale_weight', 'single_layers.30.attn.w1v.scale_weight', 'single_layers.30.attn.w1o.scale_weight', 'single_layers.30.mlp.c_fc1.scale_weight', 'single_layers.30.mlp.c_fc2.scale_weight', 'single_layers.30.mlp.c_proj.scale_weight', 'single_layers.31.modCX.1.scale_weight', 'single_layers.31.attn.w1q.scale_weight', 'single_layers.31.attn.w1k.scale_weight', 'single_layers.31.attn.w1v.scale_weight', 'single_layers.31.attn.w1o.scale_weight', 'single_layers.31.mlp.c_fc1.scale_weight', 'single_layers.31.mlp.c_fc2.scale_weight', 'single_layers.31.mlp.c_proj.scale_weight', 'final_linear.scale_weight', 'modF.1.scale_weight']
What am I doing wrong?
@notafraud needs https://github.com/silveroxides/ComfyUI_Hybrid-Scaled_fp8-Loader
Completely forgot about it, thank you! It's probably worth warning about it in the description (at least a link to github).
What is the reason for it being different from normal FP8 scaled models, exactly?
@notafraud needs https://github.com/silveroxides/ComfyUI_Hybrid-Scaled_fp8-Loader
Completely forgot about it, thank you! It's probably worth warning about it in the description (at least a link to github).
If people are going looking for this they can do some basic sleuthing and figure it out π
@notafraud needs https://github.com/silveroxides/ComfyUI_Hybrid-Scaled_fp8-Loader
Completely forgot about it, thank you! It's probably worth warning about it in the description (at least a link to github).
If people are going looking for this they can do some basic sleuthing and figure it out π
But the name of the repo says "fp8_scaled", no mentioning of hybrid in it, so it's easy to miss. Luckily there's already a commit to fix the Readme.