|
|
--- |
|
|
license: other |
|
|
license_name: qwen-research |
|
|
license_link: LICENSE |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- multimodal |
|
|
- mlx |
|
|
library_name: mlx |
|
|
pipeline_tag: text-generation |
|
|
base_model: Qwen/Qwen2.5-Omni-3B |
|
|
--- |
|
|
|
|
|
# giangndm/qwen2.5-omni-3b-mlx-8bit |
|
|
|
|
|
This model [giangndm/qwen2.5-omni-3b-mlx-8bit](https://huggingface.co/giangndm/qwen2.5-omni-3b-mlx-8bit) was |
|
|
converted to MLX format from [Qwen/Qwen2.5-Omni-3B](https://huggingface.co/Qwen/Qwen2.5-Omni-3B) |
|
|
using mlx-lm version **0.24.0**. |
|
|
|
|
|
## Use with mlx (https://github.com/giangndm/mlx-lm-omni) |
|
|
|
|
|
```bash |
|
|
uv add mlx-lm-omni |
|
|
# or |
|
|
uv add https://github.com/giangndm/mlx-lm-omni.git |
|
|
``` |
|
|
|
|
|
```python |
|
|
from mlx_lm_omni import load, generate |
|
|
import librosa |
|
|
from io import BytesIO |
|
|
from urllib.request import urlopen |
|
|
|
|
|
model, tokenizer = load("giangndm/qwen2.5-omni-3b-mlx-8bit") |
|
|
|
|
|
audio_path = "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2-Audio/audio/1272-128104-0000.flac" |
|
|
audio = librosa.load(BytesIO(urlopen(audio_path).read()), sr=16000)[0] |
|
|
|
|
|
messages = [ |
|
|
{"role": "system", "content": "You are a speech recognition model."}, |
|
|
{"role": "user", "content": "Transcribe the English audio into text without any punctuation marks.", "audio": audio}, |
|
|
] |
|
|
prompt = tokenizer.apply_chat_template( |
|
|
messages, add_generation_prompt=True |
|
|
) |
|
|
|
|
|
text = generate(model, tokenizer, prompt=prompt, verbose=True) |
|
|
``` |
|
|
|
|
|
|
