Text-to-Image
Transformers
Safetensors
massaki75 nielsr HF Staff commited on
Commit
fdf18d1
·
1 Parent(s): 5ffb89f

Improve model card: add metadata and sample usage (#1)

Browse files

- Improve model card: add metadata and sample usage (2d516056c7a9958d1195776ca65402cc0caa97a8)


Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +36 -2
README.md CHANGED
@@ -1,5 +1,7 @@
1
  ---
2
  license: cc-by-4.0
 
 
3
  ---
4
 
5
  <div align="center">
@@ -14,13 +16,46 @@ license: cc-by-4.0
14
  </p>
15
 
16
  ## 🚀 Introduction
17
- MedITok is the first unified visual tokenizer for medical images. Trained on 30M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok:
18
  - effectively encodes visual details and clinical semantics into a unified token space
19
  - achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
20
  - can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
21
 
22
  This work is supported by Shanghai Innovation Institute (SII).
23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## ✏️ Citation
26
 
@@ -31,5 +66,4 @@ This work is supported by Shanghai Innovation Institute (SII).
31
  journal={arXiv preprint arXiv:2505.19225},
32
  year={2025}
33
  }
34
-
35
  ```
 
1
  ---
2
  license: cc-by-4.0
3
+ library_name: transformers
4
+ pipeline_tag: text-to-image
5
  ---
6
 
7
  <div align="center">
 
16
  </p>
17
 
18
  ## 🚀 Introduction
19
+ MedITok is the first unified visual tokenizer for medical images, introduced in [Unified Medical Image Tokenizer for Autoregressive Synthesis and Understanding](https://huggingface.co/papers/2505.19225). Trained on 33M medical images and 2M image-caption pairs via a two-stage representation learning framework, MedITok:
20
  - effectively encodes visual details and clinical semantics into a unified token space
21
  - achieves state-of-the-art performance across diverse medical imaging modalities and tasks.
22
  - can be incorporated into prevelant generative models (e.g., autoregressive architectures) for downstream medical image synthesis and interpretation.
23
 
24
  This work is supported by Shanghai Innovation Institute (SII).
25
 
26
+ ## 🎯 Sample Usage
27
+
28
+ ### Image feature extraction
29
+ The following snippet demonstrates how to use the model for extracting features (requires the model implementation from the [official repository](https://github.com/masaaki-75/meditok)):
30
+
31
+ ```python
32
+ import torch
33
+ import numpy as np
34
+ from PIL import Image
35
+
36
+ def read_image(img, img_size=256):
37
+ if isinstance(img, str):
38
+ img = Image.open(img)
39
+
40
+ if isinstance(img, Image.Image):
41
+ img = img.convert('RGB')
42
+ if img.size[0] != img_size:
43
+ img = img.resize((img_size, img_size), Image.LANCZOS)
44
+ return img
45
+
46
+ def image_to_tensor(x):
47
+ # [H, W, C] -> [B, C, H, W]
48
+ x = torch.FloatTensor(np.array(x)).permute(2, 0, 1)
49
+ x = (x / 255.) * 2. - 1.
50
+ return x.unsqueeze(0)
51
+
52
+ # Assuming 'net' is the loaded MedITok model
53
+ img_path = 'assets/vis_imgs/sample1.png'
54
+ img = read_image(img_path)
55
+ x = image_to_tensor(img)
56
+ with torch.no_grad():
57
+ f = net.forward_features(x)
58
+ ```
59
 
60
  ## ✏️ Citation
61
 
 
66
  journal={arXiv preprint arXiv:2505.19225},
67
  year={2025}
68
  }
 
69
  ```