razaimam45 commited on
Commit
fea5052
Β·
verified Β·
1 Parent(s): 8e4a230

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +42 -5
README.md CHANGED
@@ -1,4 +1,12 @@
1
  # TΒ³: Test-Time Model Merging for Medical Vision-Language Models
 
 
 
 
 
 
 
 
2
 
3
  ![TΒ³ Workflow](figures/method.png)
4
  *Figure 1: Dynamic test-time merging workflow of TΒ³*
@@ -22,13 +30,14 @@ Official implementation of **TΒ³: Test-Time Model Merging in Vision-Language Mod
22
  - [Folder Structure](#folder-structure)
23
  - [Reproducing Results](#reproducing-results)
24
  - [Pretrained Weights](#pretrained-weights)
 
25
  - [Citation](#citation)
26
 
27
  ## Installation
28
 
29
  1. Clone repository:
30
  ```bash
31
- git clone https://github.com/yourusername/T3.git
32
  cd T3
33
  ```
34
 
@@ -91,12 +100,14 @@ Empirically, \(I(x)\) correlates positively with \(R(x)\), but better distinguis
91
  ---
92
 
93
  ## Folder Structure
94
-
95
  ```
96
  T3/
97
  β”œβ”€β”€ clip/ # CLIP model adaptations
98
  β”œβ”€β”€ data/ # Data Utilities
99
  β”œβ”€β”€ utils/ # Helper functions
 
 
100
  β”œβ”€β”€ baselines.py # Comparison methods
101
  β”œβ”€β”€ t_cube.py # Core TΒ³ implementation
102
  β”œβ”€β”€ BetaMixture.py # Auxiliary models
@@ -109,7 +120,7 @@ T3/
109
 
110
  To reproduce the results from the paper, you can run the `t_cube.py` script. This script handles the evaluation of TΒ³ and its baselines across multiple datasets and severity levels. Additional baselines are available in `baselines.py`.
111
 
112
- To understand the script better:
113
  - Refer to the `compute_samplewise_tcube_weights` and `compute_samplewise_tcube_weights_MI` functions for entropy (DaWiN baseline) and Our mutual information-based merging.
114
  - Check the `evaluate_on_test_set` function for how datasets and severities are processed.
115
  - Explore the `evaluate_tcube` function for the merging and evaluation logic.
@@ -126,12 +137,38 @@ We provide pretrained weights for the following models:
126
  - Cell Microscopy
127
  - Retinal OCT
128
 
129
- If you would like access to these weights, please contact us directly at [Raza Imam](mailto:[email protected]).
 
130
 
131
  ---
132
 
 
 
 
 
 
 
 
 
 
 
 
133
  ## License
134
  This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
135
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
  ## Contact
137
- For questions or collaborations, contact [Raza Imam](mailto:[email protected]).
 
1
  # TΒ³: Test-Time Model Merging for Medical Vision-Language Models
2
+ [Raza Imam](https://razaimam45.github.io/), Hu Wang, Dwarikanath Mahapatra, Mohammad Yaqub \
3
+ Mohamed bin Zayed University of Artificial Intelligence
4
+
5
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
6
+ [![Paper](https://img.shields.io/badge/Paper-PDF-blue)](https://arxiv.org/abs/2510.27265)
7
+ [![Weights](https://img.shields.io/badge/Model-MediMeta--C-yellow)](https://huggingface.co/razaimam45/TCube_Merging)
8
+
9
+ This repository provides the official PyTorch implementation of our TΒ³ Medical Model-Merging paper:
10
 
11
  ![TΒ³ Workflow](figures/method.png)
12
  *Figure 1: Dynamic test-time merging workflow of TΒ³*
 
30
  - [Folder Structure](#folder-structure)
31
  - [Reproducing Results](#reproducing-results)
32
  - [Pretrained Weights](#pretrained-weights)
33
+ - [Datasets](#datasets)
34
  - [Citation](#citation)
35
 
36
  ## Installation
37
 
38
  1. Clone repository:
39
  ```bash
40
+ git clone https://github.com/Razaimam45/TCube.git
41
  cd T3
42
  ```
43
 
 
100
  ---
101
 
102
  ## Folder Structure
103
+ Do check our [HuggingFace page](https://huggingface.co/razaimam45/TCube_Merging) for Expert Models and Evaluation Datasets.
104
  ```
105
  T3/
106
  β”œβ”€β”€ clip/ # CLIP model adaptations
107
  β”œβ”€β”€ data/ # Data Utilities
108
  β”œβ”€β”€ utils/ # Helper functions
109
+ β”œβ”€β”€ models/ # Put your finetuned models HERE
110
+ β”œβ”€β”€ dataset/ # Put your medimeta/medmnist-c eval data HERE
111
  β”œβ”€β”€ baselines.py # Comparison methods
112
  β”œβ”€β”€ t_cube.py # Core TΒ³ implementation
113
  β”œβ”€β”€ BetaMixture.py # Auxiliary models
 
120
 
121
  To reproduce the results from the paper, you can run the `t_cube.py` script. This script handles the evaluation of TΒ³ and its baselines across multiple datasets and severity levels. Additional baselines are available in `baselines.py`.
122
 
123
+ To understand the script better; in `t_cube.py`:
124
  - Refer to the `compute_samplewise_tcube_weights` and `compute_samplewise_tcube_weights_MI` functions for entropy (DaWiN baseline) and Our mutual information-based merging.
125
  - Check the `evaluate_on_test_set` function for how datasets and severities are processed.
126
  - Explore the `evaluate_tcube` function for the merging and evaluation logic.
 
137
  - Cell Microscopy
138
  - Retinal OCT
139
 
140
+ <!-- If you would like access to these weights, please contact us directly at [Raza Imam](mailto:[email protected]). -->
141
+ If you would like to access these weights, please find them at model card at [https://huggingface.co/razaimam45/TCube_Merging](https://huggingface.co/razaimam45/TCube_Merging) under `models/finetuned` subfolder.
142
 
143
  ---
144
 
145
+ ## Datasets
146
+
147
+ We provided `Breast Imaging` evaluation sets on [HuggingFace page](https://huggingface.co/razaimam45/TCube_Merging). Please download from there.
148
+
149
+ If you need to run multiple modalities datasets, just pass `--testset` arg with `'bloodmnist/breastmnist/'`. This will evaluate medmnist-c and medimeta from each modality, resulting in 4 datasets evaluation.
150
+
151
+ If you need all modality datasets, you can find them as follows:
152
+ * [MedMNIST datasets](https://zenodo.org/records/10519652) | In-Domain _Fine-Tune_ Datasets
153
+ * [MediMeta datasets](https://zenodo.org/records/7884735) | OOD-B2N _Eval_ Datasets
154
+ * [MedMNIST-C datasets](https://github.com/francescodisalvo05/medmnistc-api) | OOD-Corruptions _Eval_ Datasets
155
+
156
  ## License
157
  This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
158
 
159
+ ### Citation
160
+ If you find this work useful, please cite the arXiv version below:
161
+ ```
162
+ @misc{imam2025t3testtimemodelmerging,
163
+ title={T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis},
164
+ author={Raza Imam and Hu Wang and Dwarikanath Mahapatra and Mohammad Yaqub},
165
+ year={2025},
166
+ eprint={2510.27265},
167
+ archivePrefix={arXiv},
168
+ primaryClass={cs.CV},
169
+ url={https://arxiv.org/abs/2510.27265},
170
+ }
171
+ ```
172
+
173
  ## Contact
174
+ For questions or collaborations, contact [Raza Imam](mailto:[email protected]). Please feel free to raise an issue in facing error in reproducing the results.