Update README.md
Browse files
README.md
CHANGED
|
@@ -44,6 +44,28 @@ Source code is available at https://github.com/NVlabs/Sana.
|
|
| 44 |
|
| 45 |
refer to [Sana-ControlNet Guidance](https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/controlnet/controlnet_app.jpg) for more details.
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
### Model Description
|
| 49 |
|
|
@@ -66,10 +88,6 @@ which is more suitable for both training and inference and for which most advanc
|
|
| 66 |
- **Repository:** ttps://github.com/NVlabs/Sana
|
| 67 |
- **Demo:** https://nv-sana.mit.edu/
|
| 68 |
|
| 69 |
-
### 🧨 Diffusers
|
| 70 |
-
|
| 71 |
-
PR developing: [Sana](https://github.com/huggingface/diffusers/pull/9982) and [DC-AE](https://github.com/huggingface/diffusers/pull/9708)
|
| 72 |
-
|
| 73 |
|
| 74 |
## Uses
|
| 75 |
|
|
|
|
| 44 |
|
| 45 |
refer to [Sana-ControlNet Guidance](https://raw.githubusercontent.com/NVlabs/Sana/refs/heads/main/asset/controlnet/controlnet_app.jpg) for more details.
|
| 46 |
|
| 47 |
+
```python
|
| 48 |
+
import torch
|
| 49 |
+
from PIL import Image
|
| 50 |
+
from app.sana_controlnet_pipeline import SanaControlNetPipeline
|
| 51 |
+
|
| 52 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 53 |
+
|
| 54 |
+
pipe = SanaControlNetPipeline("configs/sana_controlnet_config/Sana_1600M_1024px_controlnet_bf16.yaml")
|
| 55 |
+
pipe.from_pretrained("hf://Efficient-Large-Model/Sana_1600M_1024px_BF16_ControlNet_HED/checkpoints/Sana_1600M_1024px_BF16_ControlNet_HED.pth")
|
| 56 |
+
|
| 57 |
+
ref_image = Image.open("asset/controlnet/ref_images/A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a la.jpg")
|
| 58 |
+
prompt = "A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a landscape."
|
| 59 |
+
|
| 60 |
+
images = pipe(
|
| 61 |
+
prompt=prompt,
|
| 62 |
+
ref_image=ref_image,
|
| 63 |
+
guidance_scale=4.5,
|
| 64 |
+
num_inference_steps=10,
|
| 65 |
+
sketch_thickness=2,
|
| 66 |
+
generator=torch.Generator(device=device).manual_seed(0),
|
| 67 |
+
)
|
| 68 |
+
```
|
| 69 |
|
| 70 |
### Model Description
|
| 71 |
|
|
|
|
| 88 |
- **Repository:** ttps://github.com/NVlabs/Sana
|
| 89 |
- **Demo:** https://nv-sana.mit.edu/
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
## Uses
|
| 93 |
|