Sirius35/Fintuned-distilbert-NER-for-FinTech

Finance-Oriented NER Model with MISC Extension

This model is based on dslim/distilbert-NER and fine-tuned for Named Entity Recognition (NER) with an additional focus on financial domain terminology.

---

Model Overview

• Base Model: dslim/distilbert-NER
• Task Type: Token Classification / NER
• Modified Label: MISC — Expanded for more financial-specific terms (e.g., financial instruments, policy names, industry area，financial terminology).
• Objective: Extend standard NER to not only capture named entities (such as people, organizations, and locations) but also to recognize domain-specific financial terms that describe events and their potential impacts.

---

Dataset

• Source: 50 news articles collected from public sources.
• Processing Steps:
  1. News articles were summarized using an abstractive summarization model.
  2. Summaries were manually annotated to mark standard entities and the new MILC class.
• Entity Schema:
  • Standard labels: PER, ORG, LOC, MISC
  • Modified label: MISC (financial-specific terms are included)
  • Abbreviation and Description

    Abbreviation	Description
    O	Outside of a named entity
    B-MISC	Beginning of a miscellaneous entity right after another miscellaneous entity
    I-MISC	Miscellaneous entity
    B-PER	Beginning of a person’s name right after another person’s name
    I-PER	Person’s name
    B-ORG	Beginning of an organization right after another organization
    I-ORG	organization
    B-LOC	Beginning of a location right after another location
    I-LOC	Location

Note: The MISC label is currently a single broad category. More fine-grained classification will be addressed in the next stage.

---

Eval results

Metric	Score
Loss	0.4912
Precision	0.7010
Recall	0.7967
F1	0.7458
Accuracy	0.8914

Limitation: Overuse of MISC lowers precision

Annotating many generic financial-specific as MISC turns MISC into a broad catch-all class. This creates a fuzzy decision boundary and the model learns low-specificity rules (“financial-specific tokens → MISC”), which over-predicts MISC, inflates recall, and depresses precision, reducing overall F1.

Why this happens

MISC becomes a high-frequency, heterogeneous label with weak lexical anchors, conflating named entities with topical vocabulary. The classifier then favors MISC for many ambiguous tokens, producing systematic false positives and occasional span fragmentation.

Note on current results (intentional high-recall phase)

The observed precision drop tied to broad MISC usage is largely expected at this stage. Our near-term objective is to surface domain-specific financial terms that describe entities and their potential impacts, so I intentionally bias for recall and allow MISC to act as a provisional umbrella label. This high-recall bootstrapping helps collect a candidate lexicon and error patterns for the next iteration. In subsequent releases, I will narrow MISC, re-annotate with stricter guidelines to recover precision while maintaining coverage by introducing more dedicated labels.

Usage

from transformers import pipeline

ner_pipe = pipeline("token-classification", 
                    model="Sirius35/Fintuned-distilbert-NER-for-FinTech", 
                    aggregation_strategy="simple")

text = "Citi analysts believe that the Federal Reserve's rate cut will strongly impact the US bond market."
print(ner_pipe(text))