Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

974

Base only

Active filters: 4bit

YoozLabs/Yooz-Quality-v2.1-Qwen3.5-0.8B

Text Generation • 0.1B • Updated 10 days ago • 29

carlesonielfa/MinerU2.5-Pro-2604-1.2B-mlx-4bit

0.7B • Updated May 24 • 23

Meguriai/Qwen3.6-27B-OptiQ-4bit

Text Generation • 27B • Updated May 25 • 93

ekryski/SmolLM2-360M-Instruct-4bit

96.4M • Updated May 27 • 12

ekryski/Soprano-1.1-80M-4bit

24.7M • Updated May 27 • 1

ekryski/Nemotron-H-4B-Base-8K-4bit

1B • Updated May 27 • 12

ekryski/FastVLM-0.5B-4bit

0.3B • Updated May 27 • 32

vanch007/SenseVoiceSmall-4bit

Updated May 26 • 57

chukhrovns/qwen3-vl-4b-russian-handwriting-merged-4bit

Image-Text-to-Text • 5B • Updated May 26 • 99 • 1

nemozxy123/Huihui-Qwen3.5-9B-abliterated-AWQ-4bit

Image-Text-to-Text • 10B • Updated May 29 • 950

AlicanKiraz0/Titus-CybersecurityLLM-v1.0-mlx-4Bit

Text Generation • 35B • Updated May 27 • 1k • 44

ekryski/SmolLM-360M-Instruct-4bit

96.4M • Updated May 27 • 6

ekryski/Llama-3.1-Nemotron-Nano-VL-8B-V1-4bit

3B • Updated May 27 • 87 • 1

ekryski/OLMo-2-0425-1B-Instruct-4bit

0.6B • Updated May 27 • 20

ekryski/starcoder2-3b-4bit

0.6B • Updated May 27 • 58

ekryski/granite-3.0-2b-instruct-4bit

0.6B • Updated May 27 • 22 • 1

ekryski/internlm2-chat-1_8b-4bit

0.5B • Updated May 27 • 8

ekryski/Nemotron-Nano-12B-v2-VL-4bit

4B • Updated May 28 • 28

ekryski/fish-audio-s2-pro-4bit

1B • Updated May 27 • 74

ekryski/diar_streaming_sortformer_4spk-v2.1-4bit

30.2M • Updated May 27 • 7

MLXBits/huihui-qwen3-vl-30b-abliterated-4bit

Image-Text-to-Text • 5B • Updated May 28 • 138

TerminatorPower/Qwen3.5-4B-MLX-4bit

1.0B • Updated May 29 • 13

ekryski/LFM2-8B-A1B-4bit

1B • Updated May 30 • 8

ekryski/LFM2.5-8B-A1B-4bit

2B • Updated May 30 • 11

nemozxy123/Huihui-Qwen3-VL-8B-Thinking-abliterated-AWQ-W4A16

Image-Text-to-Text • 3B • Updated May 30 • 4

DuoNeural/Cosmos3-Nano-GPTQ-4bit

Text-to-Video • Updated Jun 2 • 161 • 1

DuoNeural/Cosmos3-Nano-GPTQ-4bit-Abliterated

Text-to-Video • Updated Jun 2 • 67 • 2

pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit

Text Generation • 24B • Updated Jun 2 • 106

mlx-community/NVIDIA-Nemotron-3-Nano-4B-OptiQ-4bit

Text Generation • 0.8B • Updated 15 days ago • 895 • 3

ekryski/gemma-4-12B-it-mlx-4bit

3B • Updated Jun 4 • 33 • 1