Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,22 @@ datasets:
|
|
| 11 |
|
| 12 |
# ALBERT for ARQMath 3
|
| 13 |
|
| 14 |
-
This repository contains our best model for ARQMath 3, the math_10 model. It was initialised from ALBERT-base-v2 and further pre-trained on Math StackExchange in three different stages. We also added more LaTeX tokens to the tokenizer to enable a better tokenization of mathematical formulas. math_10 was fine-tuned on a classification task to determine whether a given question (sequence 1) matches a given answer (sequence 2). The classification output can be used for ranking the best answers. For further details, please read our paper: http://ceur-ws.org/Vol-3180/paper-07.pdf.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
# Usage
|
| 17 |
|
|
|
|
| 11 |
|
| 12 |
# ALBERT for ARQMath 3
|
| 13 |
|
| 14 |
+
This repository contains our best model for ARQMath 3, the math_10 model. It was initialised from ALBERT-base-v2 and further pre-trained on Math StackExchange in three different stages. We also added more LaTeX tokens to the tokenizer to enable a better tokenization of mathematical formulas. math_10 was fine-tuned on a classification task to determine whether a given question (sequence 1) matches a given answer (sequence 2). The classification output can be used for ranking the best answers. For further details, please read our paper: http://ceur-ws.org/Vol-3180/paper-07.pdf.
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
## Other Models for ARQMath 3
|
| 18 |
+
We plan on also publishing the other fine-tuned models as well as the base models. Links to these repositories will be added here soon.
|
| 19 |
+
|
| 20 |
+
| Model | Initialised from | Pre-training | Fine-Tuning | Link |
|
| 21 |
+
|-------------|------------------|----------------------------|-------------------------------------|------|
|
| 22 |
+
| roberta_10 | RoBERTa | MathSE (1) | yes, N=10 MathSE | |
|
| 23 |
+
| base_10 | ALBERT | MathSE (1) | yes, N=10 MathSE | |
|
| 24 |
+
| math_10_add | ALBERT | MathSE (1)-(3) | yes, N=10 MathSE and annotated data | |
|
| 25 |
+
| Khan_SE_10 | ALBERT | MathSE (1) | yes, N=10 MathSE | |
|
| 26 |
+
| roberta | RoBERTa | MathSE (1) | no | |
|
| 27 |
+
| math albert | ALBERT | MathSE (1)-(3) | no | |
|
| 28 |
+
| base | ALBERT | MathSE (1) | no | |
|
| 29 |
+
| Khan_SE | ALBERT | MathSE (1) mixed with Khan | no | |
|
| 30 |
|
| 31 |
# Usage
|
| 32 |
|