You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

The OpenMidnight model weights and associated code are released under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at:
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Please note that the primary email used to sign up for your Hugging Face account must match your institutional email to receive approval. By downloading the OpenMidnight model weights, you attest that all information (affiliation, research use) is correct and up-to-date. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use.
By using the OpenMidnight model, you acknowledge that you have read and understood these terms.

OpenMidnight

Overview of the OpenMidnight pathology foundation model

State-of-the-art pathology foundation model trained on 12K slides

Developed by Sophont

Blog Post | GitHub | Demo

What is OpenMidnight?

OpenMidnight is our open replication of Kaiko.AI's Midnight, a 1.1 billion parameter Vision Transformer foundation model for computational pathology. OpenMidnight achieves state-of-the-art performance despite being trained on significantly less data than Kaiko.AI's Midnight or comparable models.

Key advantages:

🏆 State-of-the-art performance: Achieves 0.775 average score across 14 benchmarks
⚡ Efficient training: Trained in ~83 hours on 8× H100 GPUs for only $1,600 USD (estimated)
📊 Minimal data requirements: Uses only 12K slides from TCGA for training
🔓 Fully open source: Complete model weights, training code, and pipeline publicly available

OpenMidnight is aimed for computational pathology tasks including:

Tumor detection and classification
Histological grading
Tissue segmentation
Margin assessment
Clinical outcome prediction

Model Description

Developed by: Sophont
Model type: Finetuned DINOv2 ViT-G for H&E pathology images
Training data: TCGA, 12M H&E WSI
Training repository: https://github.com/MedARC-AI/OpenMidnight/tree/main

Usage

Requirements

pip install torch torchvision huggingface_hub

Recommended: Run on GPU with mixed precision for optimal performance.

Quick Start: Loading the Model

import torch
from huggingface_hub import hf_hub_download

#Downloads to hf cache location
download_location = hf_hub_download(repo_id="SophontAI/OpenMidnight", filename="teacher_checkpoint_load.pt")
model = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14_reg', weights = None)

#Load OpenMidnight weights
checkpoint = torch.load(download_location, map_location = "cpu")

#Required because dinov2 is baseline 392 and we are baseline 224 resolution
pos_embed = checkpoint["pos_embed"]
model.pos_embed = torch.nn.parameter.Parameter(pos_embed)
model.load_state_dict(checkpoint)
model.eval()

print(f"Model loaded with {sum(p.numel() for p in model.parameters()):,} parameters")

Extracting Embeddings from Tissue Patches

from PIL import Image
import torchvision.transforms as transforms

# Standard preprocessing for pathology images
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],  # ImageNet normalization
        std=[0.229, 0.224, 0.225]
    )
])

# Load and preprocess an H&E tissue patch
image = Image.open("path/to/tissue_patch.jpg")
input_tensor = transform(image).unsqueeze(0)  # Shape: [1, 3, 224, 224]

# Extract embeddings
with torch.no_grad():
    embeddings = model(input_tensor)  # Shape: [1, 1536]

print(f"Embedding shape: {embeddings.shape}")
print(f"Embedding norm: {embeddings.norm().item():.4f}")

Model Performance

OpenMidnight achieves competitive or superior performance compared to models trained on 8-30× more data:

Benchmark Comparison

Average performance of top pathology foundation models and baselines across computational pathology benchmarks

Detailed Benchmark Results

Model	#WSIs	PCam (10 shots)	BACH	BRACS	BreakHis	CRC-100K	Gleason	MHIST	PCam	Cam16 (small)	Panda (small)	CoNSeP	MoNuSAC	HEST	Average
OpenMidnight (Ours)	12K	0.790	0.916	0.661	0.873	0.961	0.817	0.844	0.938	0.946	0.652	0.631	0.655	0.390	0.775
Midnight	92K	0.900	0.906	0.642	0.850	0.964	0.809	0.825	0.951	0.831	0.633	0.663	0.707	0.384	0.774
UNI-2	350K	0.887	0.914	0.661	0.860	0.965	0.778	0.823	0.949	0.868	0.659	0.628	0.644	0.414	0.773
UNI-2/392	350K	0.821	0.917	0.663	0.829	0.965	0.791	0.849	0.927	0.858	0.653	0.629	0.659	0.407	0.767
Virchow2	3.1M	0.851	0.884	0.624	0.823	0.966	0.778	0.861	0.936	0.865	0.656	0.639	0.676	0.398	0.766
Midnight 92k	92K	0.876	0.896	0.616	0.789	0.966	0.820	0.811	0.950	0.861	0.625	0.629	0.656	0.392	0.761
Midnight 12k	12K	0.791	0.904	0.644	0.841	0.966	0.801	0.807	0.930	0.850	0.663	0.626	0.663	0.395	0.760
H-Optimus-0	500K	0.824	0.757	0.615	0.808	0.956	0.771	0.842	0.942	0.838	0.670	0.644	0.685	0.415	0.751
Kaiko-B8	29K	0.786	0.872	0.617	0.825	0.957	0.748	0.828	0.917	0.831	0.642	0.643	0.686	0.373	0.748
TCGA-100M	12K	0.774	0.864	0.615	0.779	0.967	0.799	0.792	0.927	0.852	0.667	0.622	0.656	0.396	0.747
Prov-GigaPath	171K	0.852	0.766	0.616	0.821	0.951	0.720	0.831	0.942	0.791	0.660	0.626	0.687	0.393	0.743
Hibou-L	1.1M	0.804	0.811	0.637	0.740	0.933	0.763	0.839	0.952	0.823	0.634	0.645	0.668	0.388	0.740
UNI	100K	0.815	0.791	0.593	0.789	0.948	0.757	0.840	0.938	0.822	0.655	0.627	0.659	0.386	0.740
UNI/512	100K	0.737	0.877	0.612	0.732	0.950	0.754	0.814	0.883	0.814	0.654	0.621	0.658	0.364	0.728
Phikon	12K	0.820	0.735	0.568	0.713	0.942	0.729	0.804	0.923	0.809	0.644	0.623	0.644	0.367	0.717
Phikon v2	60K	0.741	0.734	0.600	0.716	0.939	0.755	0.784	0.893	0.803	0.631	0.626	0.645	0.375	0.711
DINOv2-giant (pretrained)	0	0.719	0.725	0.583	0.832	0.935	0.744	0.862	0.874	0.507	0.382	0.564	0.614	0.342	0.668
DINOv2-giant (random)	0	0.649	0.473	0.411	0.427	0.748	0.464	0.569	0.755	0.566	0.308	0.461	0.428	0.172	0.495

Performance comparison of OpenMidnight to existing pathology foundation models on eva+HEST benchmarks. Scores for existing models are taken from Midnight paper. We report balanced accuracy for the classification tasks, Dice score for semantic segmentation (CoNSeP and MoNuSAC), and the average Pearson correlation for the nine HEST regression tasks. Only performance with [CLS] token is reported. Best score per dataset is bolded.

Model Details

Architecture

Parameter	Value
Base Architecture	ViT-G/14
Parameters	1.1 billion
Patch Size	14×14 pixels
Input Resolution	224×224 pixels
Embedding Dimension	1536
Number of Layers	40
Number of Heads	16
Initialization	Meta's DINOv2 pre-trained weights

Training Data

Dataset: TCGA (The Cancer Genome Atlas)
Slides: 12k FFPE H&E-stained whole slide images
Cancer Types: 32 different cancer types
Total Patches: 96 million
Unique Patches: 29 million
Stain Type: Hematoxylin and Eosin (H&E)
Preprocessing: Non-informative patch filtering

Training Configuration

Hardware: 8× NVIDIA H100 GPUs (80GB each)
Batch Size: 48 per GPU (384 global batch size)
Training Steps: 250,000
Optimizer: AdamW
Learning Rate: 2.0e-4
Regularization: KDE regularizer for training stability
Augmentation: Hematoxylin-Eosin-DAB colorspace transformations
Training Time: ~83 hours wall-clock time (667 GPU-hrs)
Training Cost: ~$1,600 USD (at $2.50/H100/hour)

Blog Post

For an in-depth discussion of OpenMidnight, read the full blog post.

Contact

For questions, feedback, or collaboration opportunities:

📧 Email: [email protected]
🌐 Website: sophont.med
🐦 Twitter/X: @SophontAI
💬 GitHub Issues: github.com/MedARC-AI/OpenMidnight/issues

We welcome:

Bug reports and feature requests
Contributions to the training code
Benchmark results on new datasets
Applications of OpenMidnight to novel tasks

Acknowledgments

We thank Mikhail Karasikov for answering questions about Midnight. We thank Nicolas Känzig for answering questions about eva. We thank the members of MedARC and the broader research community for their feedback and support. We are very grateful to FAL AI for granting compute to support this open-source research.

Citation

If you use OpenMidnight in your research, please cite:

@article{kaplan2025openmidnight,
            author = {Kaplan, Daniel and Grandhi, Ratna Sagari and Lane, Connor and Warner, Benjamin and Abraham, Tanishq Mathew and Scotti, Paul S.},
            title = {How to Train a State-of-the-Art Pathology Foundation Model with \$1.6k},
            year = {2025},
            url = {https://sophont.med/blog/openmidnight},
          }

License

This model is released under the Apache 2.0 License.

Terms of Use

Research Use: This model is primarily intended for research purposes in computational pathology, medical imaging, and related fields.

Clinical Use: This model is not intended for use in medical diagnosis, treatment, or prevention of disease of real patients. It should not be used as a substitute for professional medical advice.

Responsible Use: Users should:

Validate model performance on their specific use cases
Be aware of potential biases in the training data (TCGA)
Consider demographic and geographic limitations
Respect privacy rights and comply with applicable data protection laws
Follow applicable regulations and ethical guidelines

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

SophontAI
/

OpenMidnight