Update README.md

README.md

@@ -1,3 +1,49 @@
 ---
 license: bsd-3-clause
+datasets:
+- mocha
+language:
+- en
 ---
+
+# Answer Overlap Module of QAFactEval Metric
+
+
+This is the span scorer module, used in [RQUGE paper]() to evaluate the generated questions of the question generation task.
+The model was originally used in [QAFactEval]() for computing the semantic similarity of the generated answer span, given the reference answer, context, and question in the question answering task. 
+It outputs a 1-5 answer overlap score. The scorer is trained on their MOCHA dataset (initialized from [Jia et al. (2021)]()), consisting of 40k crowdsourced judgments on QA model outputs.
+
+The input to the model is defined as:
+```
+[CLS] cand. question [q] gold answer [r] pred answer [c] context
+```
+
+# Citations
+
+```
+@inproceedings{fabbri-etal-2022-qafacteval,
+    title = "{QAF}act{E}val: Improved {QA}-Based Factual Consistency Evaluation for Summarization",
+    author = "Fabbri, Alexander  and
+      Wu, Chien-Sheng  and
+      Liu, Wenhao  and
+      Xiong, Caiming",
+    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
+    month = jul,
+    year = "2022",
+    address = "Seattle, United States",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2022.naacl-main.187",
+    doi = "10.18653/v1/2022.naacl-main.187",
+    pages = "2587--2601",
+    abstract = "Factual consistency is an essential quality of text summarization models in practical settings. Existing work in evaluating this dimension can be broadly categorized into two lines of research, entailment-based and question answering (QA)-based metrics, and different experimental setups often lead to contrasting conclusions as to which paradigm performs the best. In this work, we conduct an extensive comparison of entailment and QA-based metrics, demonstrating that carefully choosing the components of a QA-based metric, especially question generation and answerability classification, is critical to performance. Building on those insights, we propose an optimized metric, which we call QAFactEval, that leads to a 14{\%} average improvement over previous QA-based metrics on the SummaC factual consistency benchmark, and also outperforms the best-performing entailment-based metric. Moreover, we find that QA-based and entailment-based metrics can offer complementary signals and be combined into a single metric for a further performance boost.",
+}
+
+@misc{mohammadshahi2022rquge,
+    title={RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question},
+    author={Alireza Mohammadshahi and Thomas Scialom and Majid Yazdani and Pouya Yanki and Angela Fan and James Henderson and Marzieh Saeidi},
+    year={2022},
+    eprint={2211.01482},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```