Instructions to use massaki75/meditok with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use massaki75/meditok with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("massaki75/meditok", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Improve model card: add metadata and sample usage (#1)
Browse files- Improve model card: add metadata and sample usage (2d516056c7a9958d1195776ca65402cc0caa97a8)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,5 +1,7 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-4.0
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
<div align="center">
|
|
@@ -14,13 +16,46 @@ license: cc-by-4.0
|
|
| 14 |
</p>
|
| 15 |
|
| 16 |
## 🚀 Introduction
|
| 17 |
-
MedITok is the first unified visual tokenizer for medical images. Trained on
|
| 18 |
- effectively encodes visual details and clinical semantics into a unified token space
|
| 19 |
- achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
|
| 20 |
- can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
|
| 21 |
|
| 22 |
This work is supported by Shanghai Innovation Institute (SII).
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## ✏️ Citation
|
| 26 |
|
|
@@ -31,5 +66,4 @@ This work is supported by Shanghai Innovation Institute (SII).
|
|
| 31 |
journal={arXiv preprint arXiv:2505.19225},
|
| 32 |
year={2025}
|
| 33 |
}
|
| 34 |
-
|
| 35 |
```
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-4.0
|
| 3 |
+
library_name: transformers
|
| 4 |
+
pipeline_tag: text-to-image
|
| 5 |
---
|
| 6 |
|
| 7 |
<div align="center">
|
|
|
|
| 16 |
</p>
|
| 17 |
|
| 18 |
## 🚀 Introduction
|
| 19 |
+
MedITok is the first unified visual tokenizer for medical images, introduced in [Unified Medical Image Tokenizer for Autoregressive Synthesis and Understanding](https://huggingface.co/papers/2505.19225). Trained on 33M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok:
|
| 20 |
- effectively encodes visual details and clinical semantics into a unified token space
|
| 21 |
- achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
|
| 22 |
- can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
|
| 23 |
|
| 24 |
This work is supported by Shanghai Innovation Institute (SII).
|
| 25 |
|
| 26 |
+
## 🎯 Sample Usage
|
| 27 |
+
|
| 28 |
+
### Image feature extraction
|
| 29 |
+
The following snippet demonstrates how to use the model for extracting features (requires the model implementation from the [official repository](https://github.com/masaaki-75/meditok)):
|
| 30 |
+
|
| 31 |
+
```python
|
| 32 |
+
import torch
|
| 33 |
+
import numpy as np
|
| 34 |
+
from PIL import Image
|
| 35 |
+
|
| 36 |
+
def read_image(img, img_size=256):
|
| 37 |
+
if isinstance(img, str):
|
| 38 |
+
img = Image.open(img)
|
| 39 |
+
|
| 40 |
+
if isinstance(img, Image.Image):
|
| 41 |
+
img = img.convert('RGB')
|
| 42 |
+
if img.size[0] != img_size:
|
| 43 |
+
img = img.resize((img_size, img_size), Image.LANCZOS)
|
| 44 |
+
return img
|
| 45 |
+
|
| 46 |
+
def image_to_tensor(x):
|
| 47 |
+
# [H, W, C] -> [B, C, H, W]
|
| 48 |
+
x = torch.FloatTensor(np.array(x)).permute(2, 0, 1)
|
| 49 |
+
x = (x / 255.) * 2. - 1.
|
| 50 |
+
return x.unsqueeze(0)
|
| 51 |
+
|
| 52 |
+
# Assuming 'net' is the loaded MedITok model
|
| 53 |
+
img_path = 'assets/vis_imgs/sample1.png'
|
| 54 |
+
img = read_image(img_path)
|
| 55 |
+
x = image_to_tensor(img)
|
| 56 |
+
with torch.no_grad():
|
| 57 |
+
f = net.forward_features(x)
|
| 58 |
+
```
|
| 59 |
|
| 60 |
## ✏️ Citation
|
| 61 |
|
|
|
|
| 66 |
journal={arXiv preprint arXiv:2505.19225},
|
| 67 |
year={2025}
|
| 68 |
}
|
|
|
|
| 69 |
```
|