noxneural/lilium_albanicum_eng_alb
Viewer • Updated • 508k • 53 • 4
How to use nisten/shqiponja-15b-v1 with PEFT:
Task type is invalid.
15.6b 2expert MoE
axolotl version: 0.4.0
base_model: nisten/shqiponja15
model_type: AutoModelForCausalLM
tokenizer_type: LlamaTokenizer
trust_remote_code: true
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
- path: iamshnoo/alpaca-cleaned-albanian
type: alpaca
shards: 10
- path: noxneural/lilium_albanicum_eng_alb
shards: 20
type:
field_system: system
field_instruction: question
field_output: response
format: "[INST] {instruction} [/INST]"
dataset_prepared_path: last_run_prepared
val_set_size: 0.0
output_dir: ./alora-out
# - model.layers.2[7-9]+.block_sparse_moe.experts.*
# - model.layers.3[0-9]+.block_sparse_moe.experts.*
# - model.layers.2[7-9]+.b
</details><br>
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
Unable to build the model tree, the base model loops to the model itself. Learn more.