| library_name: transformers | |
| pipeline_tag: fill-mask | |
| tags: | |
| - genomics | |
| - dna | |
| - masked-lm | |
| - ntv3 | |
| - long-range | |
| - base-model | |
| license: other | |
| language: | |
| - code | |
| model_parameter_count: 7692475 | |
| # InstaDeepAI/ntv3_base_model | |
| **Unified base model repository for NTv3 models.** | |
| This repository contains shared modeling code used by both: | |
| - **Pre-trained models** (masked language models) | |
| - **Post-trained models** (conditioned multi-species models with functional genomics heads) | |
| **Note:** This repo should not be used standalone. It provides modeling code that is referenced by individual model checkpoints via `trust_remote_code=True`. | |
| ## Contents | |
| | File | Purpose | | |
| |------|---------| | |
| | `configuration_ntv3_pretrained.py` | Config class: `Ntv3PreTrainedConfig` | | |
| | `configuration_ntv3_posttrained.py` | Config classes: `DiscreteConditionedNTv3Config`, `NTv3PostTrainedConfig` | | |
| | `modeling_ntv3_pretrained.py` | Pre-trained model: `NTv3PreTrained` | | |
| | `modeling_ntv3_posttrained.py` | Post-trained model: `NTv3PostTrained` with conditioned towers and heads | | |
| | `tokenization_ntv3.py` | Tokenizer: `NTv3Tokenizer` (DNA) | | |
| ## Architecture | |
| - U-Net style conv tower → Transformer stack → deconv tower → LM head | |
| - Post-trained models add adaptive layer norms and multi-species prediction heads | |
| - Tokenizer: character-level over A T C G N + specials (`<unk>` `<pad>` `<mask>` `<cls>` `<eos>` `<bos>`) | |