|
|
--- |
|
|
base_model: black-forest-labs/FLUX.1-dev |
|
|
library_name: diffusers |
|
|
license: other |
|
|
tags: |
|
|
- text-to-image |
|
|
- diffusers-training |
|
|
- diffusers |
|
|
- lora |
|
|
- flux |
|
|
- flux-diffusers |
|
|
- template:sd-lora |
|
|
instance_prompt: a puppy, yarn art style |
|
|
widget: [] |
|
|
--- |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the training script had access to. You |
|
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
|
|
|
# Flux DreamBooth LoRA - sayakpaul/yarn_art_lora_flux_nf4 |
|
|
|
|
|
<Gallery /> |
|
|
|
|
|
## Model description |
|
|
|
|
|
These are sayakpaul/yarn_art_lora_flux_nf4 DreamBooth LoRA weights for black-forest-labs/FLUX.1-dev. |
|
|
|
|
|
The weights were trained using [DreamBooth](https://dreambooth.github.io/) with the [Flux diffusers trainer](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_flux.md). |
|
|
|
|
|
Was LoRA for the text encoder enabled? False. |
|
|
|
|
|
Trained with quantization: Yes. |
|
|
|
|
|
Quantization config: |
|
|
|
|
|
```json |
|
|
BitsAndBytesConfig { |
|
|
"_load_in_4bit": true, |
|
|
"_load_in_8bit": false, |
|
|
"bnb_4bit_compute_dtype": "bfloat16", |
|
|
"bnb_4bit_quant_storage": "uint8", |
|
|
"bnb_4bit_quant_type": "nf4", |
|
|
"bnb_4bit_use_double_quant": false, |
|
|
"llm_int8_enable_fp32_cpu_offload": false, |
|
|
"llm_int8_has_fp16_weight": false, |
|
|
"llm_int8_skip_modules": null, |
|
|
"llm_int8_threshold": 6.0, |
|
|
"load_in_4bit": true, |
|
|
"load_in_8bit": false, |
|
|
"quant_method": "bitsandbytes" |
|
|
} |
|
|
|
|
|
``` |
|
|
|
|
|
To know more on how this was trained, follow: https://gist.github.com/sayakpaul/05afd428bc089b47af7c016e42004527. |
|
|
|
|
|
## Trigger words |
|
|
|
|
|
You should use `a puppy, yarn art style` to trigger the image generation. |
|
|
|
|
|
## Download model |
|
|
|
|
|
[Download the *.safetensors LoRA](sayakpaul/yarn_art_lora_flux_nf4/tree/main) in the Files & versions tab. |
|
|
|
|
|
## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers) |
|
|
|
|
|
```py |
|
|
from diffusers import AutoPipelineForText2Image, FluxTransformer2DModel, BitsAndBytesConfig |
|
|
import torch |
|
|
|
|
|
ckpt_id = "black-forest-labs/FLUX.1-dev" |
|
|
bnb_4bit_compute_dtype = torch.bfloat16 |
|
|
nf4_config = BitsAndBytesConfig( |
|
|
load_in_4bit=True, |
|
|
bnb_4bit_quant_type="nf4", |
|
|
bnb_4bit_compute_dtype=bnb_4bit_compute_dtype, |
|
|
) |
|
|
transformer = FluxTransformer2DModel.from_pretrained( |
|
|
ckpt_id, |
|
|
subfolder="transformer", |
|
|
quantization_config=nf4_config, |
|
|
torch_dtype=bnb_4bit_compute_dtype, |
|
|
) |
|
|
pipeline = AutoPipelineForText2Image.from_pretrained( |
|
|
ckpt_id, transformer=transformer, torch_dtype=torch.bfloat16 |
|
|
) |
|
|
pipeline.load_lora_weights("yarn_art_lora_flux_nf4", weight_name="pytorch_lora_weights.safetensors") |
|
|
|
|
|
pipeline.fuse_lora() |
|
|
pipeline.unload_lora_weights() |
|
|
pipeline.enable_model_cpu_offload() |
|
|
|
|
|
image = pipeline("a puppy in a pond, yarn art style", guidance_scale=3.5, height=768).images[0] |
|
|
image.save("yarn.png") |
|
|
``` |
|
|
|
|
|
For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters) |
|
|
|
|
|
## License |
|
|
|
|
|
Please adhere to the licensing terms as described [here](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). |
|
|
|
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
#### How to use |
|
|
|
|
|
```python |
|
|
# TODO: add an example code snippet for running this diffusion pipeline |
|
|
``` |
|
|
|
|
|
#### Limitations and bias |
|
|
|
|
|
[TODO: provide examples of latent issues and potential remediations] |
|
|
|
|
|
## Training details |
|
|
|
|
|
[TODO: describe the data used to train the model] |
