---
license: apache-2.0
language:
- en
- vi
metrics:
- f1
base_model:
- FacebookAI/roberta-base
pipeline_tag: fill-mask
tags:
- finance
- esg
- text-classification
- fill-mask
- bert
library_name: transformers
datasets:
- nguyen599/EnVi-ESG-200
widget:
- text: "Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation."
---

ESG analysis can help investors determine a business' long-term sustainability and identify associated risks. MaskESG-RoBERTa-base is a [FacebookAI/roberta-base](https://huggingface.co/FacebookAI/roberta-base) model fine-tuned on [EnVi-ESG-200](https://huggingface.co/nguyen599/EnVi-ESG-200) dataset, include 200,000 annotated sentences from Vietnam, English news and ESG reports.

**Input**: A financial text.

**Output**: Environmental, Social, Governance or Neural.

**Language support**: English, Vietnamese

# How to use
You can use this model with Transformers pipeline for ESG classification or fill mask task.
```python
# tested in transformers==4.53.0
from transformers import AutoTokenizer, AutoModelForMaskedLM, pipeline

maskesg = AutoModelForMaskedLM.from_pretrained('nguyen599/MaskESG-RoBERTa-base')
tokenizer = AutoTokenizer.from_pretrained('nguyen599/MaskESG-RoBERTa-base')
nlp = pipeline("fill-mask", model=maskesg, tokenizer=tokenizer)
# Classification as fill-mask
results = nlp(f'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is {tokenizer.mask_token}')
print(results)
# [{'score': 0.9015821814537048,
#   'token': 444,
#   'token_str': ' E',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is E'},
#  {'score': 0.09723947197198868,
#   'token': 427,
#   'token_str': ' N',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is N'},
#  {'score': 0.0010556845227256417,
#   'token': 322,
#   'token_str': ' S',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is S'},
#  {'score': 0.0001152529803221114,
#   'token': 443,
#   'token_str': ' G',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is G'},
#  {'score': 1.14425779429439e-06,
#   'token': 299,
#   'token_str': ' e',
#   'sequence': 'Over three chapters, it covers a range of topics from energy efficiency and renewable energy to the circular economy and sustainable transportation. This sentence is e'}]

```