Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Paper • 2203.05482 • Published • 8
This is a merge of pre-trained language models created using mergekit.
This model was merged using the Linear merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: MrRobotoAI/Nord-v1.2-8b-Uncensored-BASE-128k+svjack/Genshin_Impact_aya_23_8B_v3_Plot_Chat_roleplay_chat_lora_small
- model: MrRobotoAI/Nord-v1.2-8b-Uncensored-BASE-128k+svjack/DPO_Genshin_Impact_Mistral_Plot_Engine_Step_Json_Short_lora_small
- model: MrRobotoAI/Nord-v1.2-8b-Uncensored-BASE-128k+svjack/Genshin_Impact_Mistral_v3_Plot_Chat_roleplay_chat_lora_small
- model: MrRobotoAI/Nord-v1.2-8b-Uncensored-BASE-128k+multimodalai/talent-critique-llama3_1_8b-tt_lora-model_4_2k-adapter-rev_3
parameters:
weight: 1.0
merge_method: linear
normalize: true
dtype: float16
Install from pip and serve model
# Install vLLM from pip: pip install vllm# Start the vLLM server: vllm serve "MrRobotoAI/X3"# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MrRobotoAI/X3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'