Spaces:
Running
on
Zero
Running
on
Zero
Complied error
#1
by
rahul7star
- opened
I Hit this error , i am debugging FYI
File "/usr/local/lib/python3.10/site-packages/spaces/zero/torch/aoti.py", line 52, in call
compiled_model.load_constants(self.weights.constants_map, check_full_update=True, user_managed=True)
File "/usr/local/lib/python3.10/site-packages/torch/export/pt2_archive/_package.py", line 458, in load_constants
self.loader.load_constants(
RuntimeError: Constant not found: single_transformer_blocks.37.attn.to_q.bias
I don't know how. I just ran the demo with a prompt and it worked.
Sorry I was referring to new compiled model base + lora , will it work with base +lora ?
model => rahul7star/flux-aot.
Just used below steps to upload
pipe = DiffusionPipeline.from_pretrained("black-forest-labs/Flux.1-Dev", torch_dtype=dtype).to(device)
pipe.load_lora_weights("rahul7star/ra3hul")
pipe.transformer.set_attn_processor(FlashFluxAttnProcessor3_0())
not sure why shape messing up hmm did the lora didnt make it to GPU lol during swap
in general AOT compilation during runtime works very well , tested same lora
pipe = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-dev",torch_dtype=torch.bfloat16)
pipe.load_lora_weights("rahul7star/ra3hul")
pipe.to('cuda')
def compile_transformer():
"""Compile the transformer for faster inference"""
with spaces.aoti_capture(pipe.transformer) as call:
pipe("optimization test prompt", num_inference_steps=28)
exported = torch.export.export(
pipe.transformer,
args=call.args,
kwargs=call.kwargs,
)
return spaces.aoti_compile(exported)
# Apply compiled model during startup
print("Compiling model for optimal performance...")
compiled_transformer = compile_transformer()
spaces.aoti_apply(compiled_transformer, pipe.transformer)
print("✅ Model compiled successfully!")