shirley430316 commited on
Commit
0c19774
·
verified ·
1 Parent(s): 9a9b492

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -10
README.md CHANGED
@@ -32,35 +32,42 @@ pip install xfuser>=0.4.3 absl-py peft lightning pandas deepspeed wandb av
32
 
33
  ### Model Preparation
34
  ```
35
- pip install -U "huggingface_hub[cli]"
36
- huggingface-cli download https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B/resolve/main/Wan2.1_VAE.pth ./model_zoo/Wan2.1/base/Wan2.1_VAE.pth
37
- huggingface-cli download https://huggingface.co/Wan-AI/Wan2.1-T2V-1.3B/resolve/main/models_t5_umt5-xxl-enc-bf16.pth ./model_zoo/Wan2.1/base/models_t5_umt5-xxl-enc-bf16.pth
38
- huggingface-cli download shirley430316/LivePhoto-Wan2.1 --local-dir ./model_zoo/LivePhoto-Wan2.1/
39
  ```
40
 
41
  After proper preparation, the directory looks like:
42
  ```
43
  ./model_zoo
44
- - LivePhoto-Wan2.1
45
- - Wan2.1-T2V-1.3B
 
 
 
46
  - base.safetensors
47
  - lora.safetensors
48
  ```
 
49
  ### Usage Example
50
  #### I2V with Motion Intensity Levels
51
  ```
52
- # make sure you are in ~/LivePhoto-Wan2.1/PusaV1
53
  CUDA_VISIBLE_DEVICES=0 python examples/pusavideo/wan_14b_multi_frames_pusa.py \
54
  --image_paths "./demos/input_image.jpg" \
55
  --prompt "A cute orange kitten with big round eyes stands upright on its hind legs on a smooth wooden floor. The kitten begins to move its tiny front paws up and down rhythmically, swaying its body left and right as if dancing. Its fluffy tail flicks slightly behind it, and the soft lighting creates a warm, cozy indoor atmosphere. The kitten’s ears twitch gently as it keeps its balance, adding to the charm of its playful little dance. The background stays softly blurred, keeping focus on the kitten’s adorable movements." \
56
  --cond_position "0" \
57
  --noise_multipliers "0" \
58
- --dit_path "./model_zoo/Wan2.1/base.safetensors" \
59
  --lora_path "./model_zoo/Wan2.1/lora.safetensors" \
60
  --lora_alpha 1.2 \
61
  --num_inference_steps 30 \
62
- --motion_intensity 5 \ # valid motion intensity levels from 1 through 6
 
63
  ```
64
 
 
 
 
 
65
  ### Acknowledgement
66
- This version is developped on the codebase of [Pusa-VidGen](https://github.com/Yaofang-Liu/Pusa-VidGen). Much appreciation for the insightful project.
 
32
 
33
  ### Model Preparation
34
  ```
35
+ pip install -U "huggingface_hub[cli]==0.34.0"
36
+ hf download Wan-AI/Wan2.1-T2V-1.3B Wan2.1_VAE.pth models_t5_umt5-xxl-enc-bf16.pth --local-dir ./model_zoo/Wan2.1/base/
37
+ hf download Wan-AI/Wan2.1-T2V-1.3B --include="google/*" --local-dir ./model_zoo/Wan2.1/base
38
+ hf download shirley430316/LivePhoto-Wan2.1 lora.safetensors base.safetensors --local-dir ./model_zoo/Wan2.1/
39
  ```
40
 
41
  After proper preparation, the directory looks like:
42
  ```
43
  ./model_zoo
44
+ - Wan2.1
45
+ - base
46
+ - Wan2.1_VAE.pth
47
+ - models_t5_umt5-xxl-enc-bf16.pth
48
+ - google
49
  - base.safetensors
50
  - lora.safetensors
51
  ```
52
+
53
  ### Usage Example
54
  #### I2V with Motion Intensity Levels
55
  ```
 
56
  CUDA_VISIBLE_DEVICES=0 python examples/pusavideo/wan_14b_multi_frames_pusa.py \
57
  --image_paths "./demos/input_image.jpg" \
58
  --prompt "A cute orange kitten with big round eyes stands upright on its hind legs on a smooth wooden floor. The kitten begins to move its tiny front paws up and down rhythmically, swaying its body left and right as if dancing. Its fluffy tail flicks slightly behind it, and the soft lighting creates a warm, cozy indoor atmosphere. The kitten’s ears twitch gently as it keeps its balance, adding to the charm of its playful little dance. The background stays softly blurred, keeping focus on the kitten’s adorable movements." \
59
  --cond_position "0" \
60
  --noise_multipliers "0" \
 
61
  --lora_path "./model_zoo/Wan2.1/lora.safetensors" \
62
  --lora_alpha 1.2 \
63
  --num_inference_steps 30 \
64
+ --cfg_scale 5 \
65
+ --motion_intensity 6 # valid motion intensity levels from 1 through 6
66
  ```
67
 
68
+ ### Limitations
69
+ - Video generation quality is inherently limited by the capabilities of the base Wan2.1-T2V-1.3B model, e.g., camera motion control is not currently supported.
70
+ - Certain image types are likely to generate low quality videos, e.g. cartoon and animated contents, possibly due to dataset biases.
71
+
72
  ### Acknowledgement
73
+ This version is developed upon the codebase of [Pusa-VidGen](https://github.com/Yaofang-Liu/Pusa-VidGen). Much appreciation for the insightful project.