README.md

---
library_name: transformers
license: mit
base_model:
- Qwen/Qwen2.5-VL-8B-Instruct
pipeline_tag: image-text-to-text
---

<p align="center">
    <a href="https://nuextract.ai/">
        <img src="logo_nuextract.svg" width="200"/>
    </a>
</p>
<p align="center">
        🖥️ <a href="https://nuextract.ai/">API / Platform</a>&nbsp&nbsp | &nbsp&nbsp📑 <a href="https://numind.ai/blog">Blog</a>&nbsp&nbsp | &nbsp&nbsp🗣️ <a href="https://discord.gg/3tsEtJNCDe">Discord</a>&nbsp&nbsp | &nbsp&nbsp🔗 <a href="https://github.com/numindai/nuextract">GitHub</a>
</p>

# NuExtract 2.0 2B GGUF by NuMind 🔥

NuExtract 2.0 is a family of models trained specifically for structured information extraction tasks. It supports both multimodal inputs and is multilingual.

We provide several versions of different sizes, all based on pre-trained models from the QwenVL family.
| Model Size | Model Name | Base Model | License | Huggingface Link |
|------------|------------|------------|---------|------------------|
| 2B | NuExtract-2.0-2B | [Qwen2-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct) | MIT | 🤗 [NuExtract-2.0-2B](https://huggingface.co/numind/NuExtract-2.0-2B) |
| 2B | NuExtract-2.0-2B-GGUF | [Qwen2-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct) | MIT | 🤗 [NuExtract-2.0-2B-GGUF](https://huggingface.co/numind/NuExtract-2.0-2B-GGUF) |
| 4B | NuExtract-2.0-4B | [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) | Qwen Research License | 🤗 [NuExtract-2.0-4B](https://huggingface.co/numind/NuExtract-2.0-4B) |
| 4B | NuExtract-2.0-4B-GGUF | [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) | Qwen Research License | 🤗 [NuExtract-2.0-4B-GGUF](https://huggingface.co/numind/NuExtract-2.0-4B-GGUF) |
| 8B | NuExtract-2.0-8B | [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) | MIT | 🤗 [NuExtract-2.0-8B](https://huggingface.co/numind/NuExtract-2.0-8B) |
| 8B | NuExtract-2.0-8B-GGUF | [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) | MIT | 🤗 [NuExtract-2.0-8B-GGUF](https://huggingface.co/numind/NuExtract-2.0-8B-GGUF) |

❗️Note: `NuExtract-2.0-2B` is based on Qwen2-VL rather than Qwen2.5-VL because the smallest Qwen2.5-VL model (3B) has a more restrictive, non-commercial license. We therefore include `NuExtract-2.0-2B` as a small model option that can be used commercially.

## Benchmark
Performance on collection of ~1,000 diverse extraction examples containing both text and image inputs.
<a href="https://nuextract.ai/">
    <img src="nuextract2_bench.png" width="500"/>
</a>

## Overview

To use the model, provide an input text/image and a JSON template describing the information you need to extract. The template should be a JSON object, specifying field names and their expected type.

Support types include:
* `verbatim-string` - instructs the model to extract text that is present verbatim in the input.
* `string` - a generic string field that can incorporate paraphrasing/abstraction.
* `integer` - a whole number.
* `number` - a whole or decimal number.
* `date-time` - ISO formatted date.
* Array of any of the above types (e.g. `["string"]`)
* `enum` - a choice from set of possible answers (represented in template as an array of options, e.g. `["yes", "no", "maybe"]`).
* `multi-label` - an enum that can have multiple possible answers (represented in template as a double-wrapped array, e.g. `[["A", "B", "C"]]`).

If the model does not identify relevant information for a field, it will return `null` or `[]` (for arrays and multi-labels).

The following is an example template:
```json
{
  "first_name": "verbatim-string",
  "last_name": "verbatim-string",
  "description": "string",
  "age": "integer",
  "gpa": "number",
  "birth_date": "date-time",
  "nationality": ["France", "England", "Japan", "USA", "China"],
  "languages_spoken": [["English", "French", "Japanese", "Mandarin", "Spanish"]]
}
```
An example output:
```json
{
  "first_name": "Susan",
  "last_name": "Smith",
  "description": "A student studying computer science.",
  "age": 20,
  "gpa": 3.7,
  "birth_date": "2005-03-01",
  "nationality": "England",
  "languages_spoken": ["English", "French"]
}
```

⚠️ We recommend using NuExtract with a temperature at or very close to 0. Some inference frameworks, such as Ollama, use a default of 0.7 which is not well suited to many extraction tasks.

## Using NuExtract with llama.cpp

### Download the model

```bash
mkdir models
hf download numind/NuExtract-2.0-2B-GGUF --local-dir ./models
```

### Start the llama.cpp server
```bash
docker run --gpus all -it -p 8000:8080 -v ./models:/models --entrypoint /app/llama-server ghcr.io/ggml-org/llama.cpp:full-cuda -m /models/NuExtract-2.0-2B-Q8_0.gguf --mmproj /models/mmproj-BF16.gguf --host 0.0.0.0
```

## Text Extraction
The `docker run` command above maps the port 8080 of the llama.cpp container to the port 8000 of the host.
```python
import openai
import json

client = openai.OpenAI(
    api_key="EMPTY",
    base_url="http://localhost:8000",
)
```
llama.cpp is not compatible with vllm's `chat_template_kwargs`. Thus, the template has to be applied manually
## Text extraction
```python
flight_text = """Date: Tuesday March 25th 2025
User info: Male, 32 yo

Book me a flight this Saturday morning to go to Marrakesh and come back on April 5th. I want it to be business class. Air France if possible."""
flight_template = """{
    "Destination": "verbatim-string",
    "Departure date range": {
        "beginning": "date-time",
        "end": "date-time"
    },
    "Return date range": {
        "beginning": "date-time",
        "end": "date-time"
    },
    "Requested Class": [
        "1st",
        "business",
        "economy"
    ],
    "Preferred airlines": [
        "string"
    ]
}"""

response = client.chat.completions.create(
    model="NuExtract",
    temperature=0.0,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text", 
                    "text": f"# Template:\n{json.dumps(json.loads(flight_template), indent=4)}\n{flight_text}",
                },
            ],
        },
    ],
)
```

## Image Extraction

```python
identity_template = """{
    "Last name": "verbatim-string",
    "First names": [
        "verbatim-string"
    ],
    "Document number": "verbatim-string",
    "Date of birth": "date-time",
    "Gender": [
        "Male",
        "Female",
        "Other"
    ],
    "Expiration date": "date-time",
    "Country ISO code": "string"
}"""

response = client.chat.completions.create(
    model="NuExtract",
    temperature=0.0,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text", 
                    "text": f"# Template:\n{json.dumps(json.loads(identity_template), indent=4)}\n<image>",
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"https://upload.wikimedia.org/wikipedia/commons/thumb/4/49/Carte_identit%C3%A9_%C3%A9lectronique_fran%C3%A7aise_%282021%2C_recto%29.png/2880px-Carte_identit%C3%A9_%C3%A9lectronique_fran%C3%A7aise_%282021%2C_recto%29.png"
                    },
                },
            ],
        },
    ],
)
```