Hugging Face's logo

PEFT documentation

LyCORIS

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

LyCORIS

LyCORIS (Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion) are LoRA-like matrix decomposition adapters that modify the cross-attention layer of the UNet. The LoHa and LoKr methods inherit from the Lycoris classes here.

LycorisConfig

class peft.tuners.lycoris_utils.LycorisConfig

< >

( task_type: Optional[Union[str, TaskType]] = None peft_type: Optional[Union[str, PeftType]] = None auto_mapping: Optional[dict] = None peft_version: Optional[str] = None base_model_name_or_path: Optional[str] = None revision: Optional[str] = None inference_mode: bool = False rank_pattern: Optional[dict] = <factory> alpha_pattern: Optional[dict] = <factory> )

A base config for LyCORIS like adapters

LycorisLayer

class peft.tuners.lycoris_utils.LycorisLayer

< >

( base_layer: nn.Module )

A base layer for LyCORIS like adapters

merge

< >

( safe_merge: bool = False adapter_names: Optional[list[str]] = None )

Parameters

  • safe_merge (bool, optional) — If True, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to False.
  • adapter_names (List[str], optional) — The list of adapter names that should be merged. If None, all active adapters will be merged. Defaults to None.

Merge the active adapter weights into the base weights

unmerge

< >

( )

This method unmerges all merged adapter layers from the base weights.

LycorisTuner

class peft.tuners.lycoris_utils.LycorisTuner

< >

( model peft_config: Union[PeftConfig, dict[str, PeftConfig]] adapter_name: str low_cpu_mem_usage: bool = False state_dict: Optional[dict[str, torch.Tensor]] = None )

Parameters

  • model (torch.nn.Module) — The model to be adapted.
  • config (LoraConfig) — The configuration of the Lora model.
  • adapter_name (str) — The name of the adapter, defaults to "default".
  • low_cpu_mem_usage (bool, optional, defaults to False) — Create empty adapter weights on meta device. Useful to speed up the loading process.

A base tuner for LyCORIS like adapters

Update on GitHub

←X-LoRA Multitask Prompt Tuning→