📝 Paper • 🤗 Hugging Face • 🧩 Github
## 🚀 Introduction MedITok is the first unified visual tokenizer for medical images, introduced in [Unified Medical Image Tokenizer for Autoregressive Synthesis and Understanding](https://huggingface.co/papers/2505.19225). Trained on 33M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok: - effectively encodes visual details and clinical semantics into a unified token space - achieves state-of-the-art performance across diverse medical imaging modalities and tasks. - can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation. This work is supported by Shanghai Innovation Institute (SII). ## 🎯 Sample Usage ### Image feature extraction The following snippet demonstrates how to use the model for extracting features (requires the model implementation from the [official repository](https://github.com/masaaki-75/meditok)): ```python import torch import numpy as np from PIL import Image def read_image(img, img_size=256): if isinstance(img, str): img = Image.open(img) if isinstance(img, Image.Image): img = img.convert('RGB') if img.size[0] != img_size: img = img.resize((img_size, img_size), Image.LANCZOS) return img def image_to_tensor(x): # [H, W, C] -> [B, C, H, W] x = torch.FloatTensor(np.array(x)).permute(2, 0, 1) x = (x / 255.) * 2. - 1. return x.unsqueeze(0) # Assuming 'net' is the loaded MedITok model img_path = 'assets/vis_imgs/sample1.png' img = read_image(img_path) x = image_to_tensor(img) with torch.no_grad(): f = net.forward_features(x) ``` ## ✏️ Citation ``` @article{ma2025meditok, title={MedITok: A Unified Tokenizer for Medical Image Synthesis and Interpretation}, author={Ma, Chenglong and Ji, Yuanfeng and Ye, Jin and Li, Zilong and Wang, Chenhui and Ning, Junzhi and Li, Wei and Liu, Lihao and Guo, Qiushan and Li, Tianbin and He, Junjun and Shan, Hongming}, journal={arXiv preprint arXiv:2505.19225}, year={2025} } ```