Update README.md
Browse files
README.md
CHANGED
|
@@ -144,6 +144,12 @@ To achieve optimal results, we recommend always including a system prompt that c
|
|
| 144 |
|
| 145 |
### Basic Instruct Template (V7)
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
```
|
| 148 |
<s>[SYSTEM_PROMPT] system prompt[/SYSTEM_PROMPT][INST] user message[/INST] assistant response</s>[INST][IMG][IMG][IMG]...[IMG][IMG_BREAK][IMG]...[IMG][IMG_END] new user message[/INST]
|
| 149 |
```
|
|
@@ -155,9 +161,9 @@ To achieve optimal results, we recommend always including a system prompt that c
|
|
| 155 |
| Model | MathVista (CoT) | MMMU (CoT) | ChartQA (CoT) | DocVQA (ANLS) | VQAv2 (VQA Match) | AIZD (BBox) | MM MT-Bench |
|
| 156 |
|:----------------------------:|:---------------:|:----------:|:-------------:|:--------------:|:-----------------:|:-----------:|:-----------:|
|
| 157 |
| **Mistral Large Vision** | **<u>69.4</u>** | 64.0 | 88.1 | **<u>93.3</u>**| **<u>80.9</u>** | 93.8 | 7.4 |
|
| 158 |
-
|
|
| 159 |
-
|
|
| 160 |
-
|
|
| 161 |
|
| 162 |
## Usage
|
| 163 |
|
|
|
|
| 144 |
|
| 145 |
### Basic Instruct Template (V7)
|
| 146 |
|
| 147 |
+
Without vision:
|
| 148 |
+
```
|
| 149 |
+
<s>[SYSTEM_PROMPT] system prompt[/SYSTEM_PROMPT][INST] user message[/INST] assistant response</s>[INST] new user message[/INST]
|
| 150 |
+
```
|
| 151 |
+
|
| 152 |
+
With vision:
|
| 153 |
```
|
| 154 |
<s>[SYSTEM_PROMPT] system prompt[/SYSTEM_PROMPT][INST] user message[/INST] assistant response</s>[INST][IMG][IMG][IMG]...[IMG][IMG_BREAK][IMG]...[IMG][IMG_END] new user message[/INST]
|
| 155 |
```
|
|
|
|
| 161 |
| Model | MathVista (CoT) | MMMU (CoT) | ChartQA (CoT) | DocVQA (ANLS) | VQAv2 (VQA Match) | AIZD (BBox) | MM MT-Bench |
|
| 162 |
|:----------------------------:|:---------------:|:----------:|:-------------:|:--------------:|:-----------------:|:-----------:|:-----------:|
|
| 163 |
| **Mistral Large Vision** | **<u>69.4</u>** | 64.0 | 88.1 | **<u>93.3</u>**| **<u>80.9</u>** | 93.8 | 7.4 |
|
| 164 |
+
| Gemini-1.5 Pro (measured)| 67.8 | 66.3 | 83.8 | 92.3 | 70.6 | **<u>94.6</u>**| 6.8 |
|
| 165 |
+
| GPT-4o (measured) | 64.6 | **<u>68.6</u>**| 85.1 | 88.9 | 77.8 | 92.8 | **<u>7.7</u>**|
|
| 166 |
+
| Claude-3.5 Sonnet (measured)| 67.1 | 68.4 | **<u>89.1</u>**| 88.6 | 69.5 | 76.9 | 7.3 |
|
| 167 |
|
| 168 |
## Usage
|
| 169 |
|