---
language:
- nl
license: other
license_name: link-attribution
license_link: https://dejanmarketing.com/link-attribution/
library_name: transformers
tags:
- token-classification
- link-prediction
- seo
- dutch
- mdeberta
base_model: microsoft/mdeberta-v3-base
datasets:
- custom
metrics:
- f1
- precision
- recall
pipeline_tag: token-classification
---

# LinkjeBERT

A Dutch language model that predicts where a human editor would place a hyperlink in plain text. Given a passage, LinkjeBERT returns per-token confidence scores indicating how likely each word is to be part of a link anchor.

Developed by [DEJAN AI](https://dejan.ai).

Read the full write-up: [LinkjeBERT: A Dutch Language Model for Link Prediction](https://dejan.ai/blog/linkjebert/)

## How It Works

LinkjeBERT is a binary token classifier fine-tuned on `microsoft/mdeberta-v3-base`. Every token is classified as either **O** (not a link) or **LINK** (part of a link anchor). The model outputs probabilities rather than hard labels, enabling heatmap-style visualization and tuneable thresholds for production pipelines.

![image](https://cdn-uploads.huggingface.co/production/uploads/64732e7f7be71eb8b1b572a8/sl1T3thY9_mmpekkjOfkw.png)

## Key Innovation: Markdown-Aware Training

Unlike its English predecessor [LinkBERT](https://linkbert.com), LinkjeBERT is trained on **structure-preserving Markdown** rather than flat text. The training data retains headings (`#`, `##`), bold (`**`), italics (`*`), lists (`-`), and blockquotes (`>`). This gives the model the same structural context a human editor uses when deciding where a link belongs.

## Training Details

| Parameter | Value |
|---|---|
| Base model | `microsoft/mdeberta-v3-base` |
| Task | Binary token classification (O / LINK) |
| Loss function | Focal loss (γ=2.0) |
| Training tokens | 200M |
| Training sources | 5 Dutch editorial domains (news, tech, science, business, lifestyle) |
| Learning rate | 2e-5 with linear warmup (10%) |
| Batch size | 32 (16 × 2 gradient accumulation) |
| Epochs | 10 (early stopping, patience 3, monitoring F1) |
| Precision | bf16 |
| Max sequence length | 512 |
| Hardware | NVIDIA RTX 4090 |

## Usage

```python
from transformers import AutoModelForTokenClassification, AutoTokenizer
import torch

model_id = "dejanseo/LinkjeBERT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

text = "De minister liet weten dat het plan doorgaat."
inputs = tokenizer(text, return_tensors="pt", return_offsets_mapping=True)
offsets = inputs.pop("offset_mapping")[0]

with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)[0, :, 1]

for i, (start, end) in enumerate(offsets):
    if start == end:
        continue
    word = text[start:end]
    p = probs[i].item()
    if p > 0.1:
        print(f"{word:20s} {p:.1%}")
```

## Long Document Inference

For documents exceeding 512 tokens, use a sliding window with overlap:

- `MAX_LENGTH = 512`
- `DOC_STRIDE = 128`
- Aggregate overlapping token probabilities with `np.maximum`

This ensures consistent predictions across article-length text.

## Input Format

Feed the model **Markdown-formatted plain text with links stripped**. The model was trained on editorial content where existing `<a>` tags were removed, so it predicts link placement from context alone. Preserve structural markers (`#`, `**`, `-`, `>`) as the model relies on them.

## Labels

| ID | Label |
|---|---|
| 0 | O |
| 1 | LINK |

## Intended Use

- Anchor text suggestion for internal linking
- Evaluating naturalness of existing link placements
- Link placement guidance for content writers
- Detection of unnatural or spammy link patterns

![image](https://cdn-uploads.huggingface.co/production/uploads/64732e7f7be71eb8b1b572a8/3uuMsVIPRlS9EpyXpu9v1.png)

## Citation

```
@misc{linkjebert2026,
  title={LinkjeBERT: A Dutch Language Model for Link Prediction},
  author={DEJAN AI},
  year={2026},
  url={https://dejan.ai/blog/linkjebert/}
}
```