Upload README.md

README.md

@@ -1,3 +1,56 @@
----
-license: llama2
----
+---
+license: llama2
+base_model:
+  - llava-hf/llava-1.5-7b-hf
+pipeline_tag: image-text-to-text
+tags:
+  - image-captioning
+  - reinforcement-learning
+  - GRPO
+---
+
+# CIM-LLaVA1.5-7B
+
+This model is fine-tuned from [LLaVA-1.5-7B](https://huggingface.co/llava-hf/llava-1.5-7b-hf) using **GRPO** with the **Cross-modal Identity Mapping (CIM)** reward, as described in our CVPR 2026 paper.
+
+[![arXiv](https://img.shields.io/badge/cs.CV-2603.01696-b31b1b?logo=arxiv&logoColor=red)](https://arxiv.org/abs/2603.01696)
+[![GitHub](https://img.shields.io/badge/GitHub-CIM-blue?logo=github)](https://github.com/Jia-hn/CIM)
+
+## Overview
+
+CIM is a reinforcement learning framework that improves image captioning by minimizing information loss during modality conversion. It uses two reward signals — **Gallery Representation Consistency (GRC)** and **Query-gallery Image Relevance (QIR)** — to encourage LVLMs to generate fine-grained and precise captions without extra annotations.
+
+## Usage
+
+```python
+from transformers import LlavaForConditionalGeneration, AutoProcessor
+from PIL import Image
+
+model = LlavaForConditionalGeneration.from_pretrained("kkk5/CIM-LLaVA1.5-7B", torch_dtype="auto", device_map="auto")
+processor = AutoProcessor.from_pretrained("kkk5/CIM-LLaVA1.5-7B")
+
+image = Image.open("your_image.jpg").convert("RGB")
+messages = [{"role": "user", "content": [
+    {"type": "image"},
+    {"type": "text", "text": "Caption this image as accurately as possible, without speculation. Describe what you see."},
+]}]
+
+text = processor.apply_chat_template(messages, add_generation_prompt=True)
+inputs = processor(images=image, text=text, return_tensors="pt").to(model.device)
+
+output_ids = model.generate(**inputs, max_new_tokens=1024)
+output_text = processor.batch_decode(output_ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]
+print(output_text)
+```
+
+## Citation
+
+```bibtex
+@inproceedings{jia2026cross,
+    title     = {Cross-modal Identity Mapping: Minimizing Information Loss in Modality Conversion via Reinforcement Learning},
+    author    = {Jia, Haonan and Dong, Shichao and Dong, Xin and Sun, Zenghui and Wang, Jin and Lan, Jinsong and Zhu, Xiaoyong and Zheng, Bo and Zhang, Kaifu},
+    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+    pages     = {766--777},
+    year      = {2026}
+}
+```