|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- merge |
|
|
- evolutionary |
|
|
- Darwin-A2AP |
|
|
base_model: |
|
|
- Qwen/Qwen3-4B-Instruct-2507 |
|
|
- Qwen/Qwen3-4B-Thinking-2507 |
|
|
--- |
|
|
|
|
|
<div align="center"> |
|
|
<span style="font-family: default; font-size: 1.5em;">Darwin-Qwen3-4B</span> |
|
|
<div> |
|
|
evolutionary algorithm 'Darwin A2AP' 🤔 |
|
|
</div> |
|
|
</div> |
|
|
<br> |
|
|
<div align="center" style="line-height: 1;"> |
|
|
<a href=" https://discord.gg/openfreeai" style="margin: 2px;"> |
|
|
<img alt="OpenFree AI Discord Server" src="https://img.shields.io/badge/Discord-000000?style=for-the-badge&logo=discord&logoColor=000&logoColor=white" style="display: inline-block; vertical-align: middle;"/> |
|
|
</a> |
|
|
<a href="https://huggingface.co/VIDraft" style="margin: 2px;"> |
|
|
<img alt="HF Page" src="https://img.shields.io/badge/VIDraft-fcd022?style=for-the-badge&logo=huggingface&logoColor=000&labelColor" style="display: inline-block; vertical-align: middle;"/> |
|
|
</a> |
|
|
</div> |
|
|
|
|
|
# openfree/Darwin-Qwen3-4B |
|
|
This model is automatically merged using evolutionary algorithm 'Darwin A2AP' v3.2 |
|
|
|
|
|
# Overview |
|
|
This study introduces a new paradigm of AI model fusion. Traditional "model merging" techniques have been restricted to models of the same family (e.g., transformer-based LLMs). We transcend this limitation by proposing a method to directly collide and fuse the core representational structures (DNA) of entirely different species — such as transformers and diffusion models. This approach acts as an "AI particle accelerator," colliding fundamentally distinct elements of intelligence to uncover new possibilities. |
|
|
The paper and source code (to be released on GitHub and Hugging Face) are currently under preparation and will be made publicly available soon. They will be released in a reproducible and extensible form for anyone to explore. |
|
|
|
|
|
## Contribution |
|
|
Breaking the Species Barrier |
|
|
Fusion of fundamentally different models such as transformers and diffusion architectures. |
|
|
Realization of cross-species model merging once deemed impossible. |
|
|
|
|
|
## AI Embryo Creation |
|
|
Formation of an initial “AI embryo” based on fused DNA. |
|
|
The embryo is not confined to a single domain or function but serves as the foundation for multi-capability intelligence. |
|
|
|
|
|
## Virtual Evolutionary Environment |
|
|
AI embryos are placed into a simulated environment spanning thousands of generations. |
|
|
Through survival and adaptation, natural selection drives evolution beyond the limitations of parent models, producing new offspring models. |
|
|
|
|
|
## Merge Information |
|
|
Father Model 1: Qwen/Qwen3-4B-Instruct-2507 |
|
|
Mother Model 2: Qwen/Qwen3-4B-Thinking-2507 |
|
|
Validation Task Accuracy: 88.56% |
|
|
Note: The above accuracy is a proxy metric used for merge ratio optimization. |
|
|
Algorithm Version: Darwin A2AP Enhanced v3.2 |
|
|
|
|
|
## ⚠️ Notice |
|
|
The actual language generation performance of this model requires separate evaluation. |
|
|
The validation score above is not an LLM benchmark score. |
|
|
|
|
|
## ⚠️ Benchmarking Test Results |
|
|
<p align="center"> <img src="BenchmarkResult.png" alt="Darwin-Qwen3-4B BenchMark Result" width="600"/> </p> |
|
|
|
|
|
|
|
|
## Use_Example |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("openfree/Darwin-Qwen3-4B") |
|
|
tokenizer = AutoTokenizer.from_pretrained("openfree/Darwin-Qwen3-4B") |
|
|
|
|
|
# 추론 예시 |
|
|
inputs = tokenizer("Hello, how are you?", return_tensors="pt") |
|
|
outputs = model.generate(**inputs) |
|
|
``` |
|
|
|
|
|
|
|
|
# Strengths & Features |
|
|
## Cross-Domain Intelligence |
|
|
Example: Legal LLM × Medical LLM → instantly produces a “Forensic LLM.” |
|
|
This is not mere knowledge aggregation but the creation of new intelligence at the intersection of domains. |
|
|
|
|
|
## Extreme Efficiency |
|
|
Achieves results at roughly 1/10,000 of the time and cost compared to training a new foundation model. |
|
|
Accessible via a simple click-based process. |
|
|
|
|
|
## Unified Intelligence |
|
|
Escapes confinement to a single domain by organically merging multiple expertises. |
|
|
Provides an experimental basis for integrated reasoning and creativity with AGI-like qualities. |
|
|
|
|
|
## Reproducibility & Openness |
|
|
Source code and models will be fully released on GitHub and Hugging Face. |
|
|
Researchers and developers can freely reproduce, experiment, and expand. |
|
|
|
|
|
# Outlook |
|
|
This research opens the door to a new generation of model creation, expressed as “Foundation a + Foundation b = Foundation abXc.” It represents far more than a reduction in training costs, serving as a critical turning point for future studies on the evolution and fusion of AI intelligence. |
|
|
|
