johnny711 commited on
Commit
b6992f6
·
verified ·
1 Parent(s): 18931dc

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +146 -4
README.md CHANGED
@@ -6,16 +6,158 @@ tags:
6
  - unsloth
7
  - lfm2_vl
8
  - trl
 
 
 
 
 
 
 
9
  license: apache-2.0
10
  language:
11
  - en
 
 
 
 
12
  ---
13
 
14
- # Uploaded model
15
 
16
- - **Developed by:** johnny711
17
- - **License:** apache-2.0
18
- - **Finetuned from model :** LiquidAI/LFM2.5-VL-450M
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  This lfm2_vl model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
21
 
 
6
  - unsloth
7
  - lfm2_vl
8
  - trl
9
+ - peft
10
+ - lora
11
+ - satellite-imagery
12
+ - object-detection
13
+ - military
14
+ - edge-ai
15
+ - space
16
  license: apache-2.0
17
  language:
18
  - en
19
+ datasets:
20
+ - HichTala/dota
21
+ - baidongls/MVRSD
22
+ pipeline_tag: image-text-to-text
23
  ---
24
 
25
+ # Project ARGUS — LFM2.5-VL Military Satellite Detection Adapter
26
 
27
+ > **Autonomous Reconnaissance & Ground Understanding System**
28
+ > *Hackathon: Liquid AI x DPhi Space "AI in Space"*
29
+
30
+ ## Overview
31
+
32
+ This is a **LoRA adapter** fine-tuned on top of [LiquidAI/LFM2.5-VL-450M](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) for **military object detection in satellite imagery**. It enables the base VLM to output structured JSON tactical reports directly from overhead reconnaissance images — replacing traditional multi-stage YOLO detection pipelines with a single unified inference pass.
33
+
34
+ ### Key Capabilities
35
+
36
+ | Capability | Description |
37
+ |---|---|
38
+ | **Military Vehicle Detection** | Tanks, APCs, trucks, artillery, civilian vehicles |
39
+ | **Aerial Asset Detection** | Aircraft, helicopters, UAVs at airfields |
40
+ | **Naval Detection** | Ships, submarines, harbor installations |
41
+ | **Infrastructure Analysis** | Bridges, storage tanks, port cranes, helipads |
42
+ | **Threat Assessment** | LOW / MEDIUM / HIGH classification per target |
43
+ | **Tactical Reasoning** | Natural language assessment for each detection |
44
+
45
+ ## Training Details
46
+
47
+ - **Base Model:** LiquidAI/LFM2.5-VL-450M
48
+ - **Method:** QLoRA (4-bit) via [Unsloth](https://github.com/unslothai/unsloth)
49
+ - **LoRA Config:** r=16, alpha=32, all linear layers
50
+ - **Trainable Parameters:** 1,376,256 / 450,095,104 (0.31%)
51
+ - **Training Data:** 3,512 samples (MVRSD military vehicles + DOTA aerial objects)
52
+ - **Epochs:** 3 (1,317 steps)
53
+ - **Final Loss:** 0.4017
54
+ - **Hardware:** NVIDIA T4 GPU
55
+ - **Training Time:** ~60 minutes
56
+
57
+ ### Datasets
58
+
59
+ | Dataset | Samples | Source |
60
+ |---|---|---|
61
+ | [MVRSD](https://github.com/baidongls/MVRSD) | 12 (demo) | Military Vehicle Remote Sensing Dataset |
62
+ | [DOTA](https://huggingface.co/datasets/HichTala/dota) | 3,500 | Large-scale aerial object detection |
63
+
64
+ ## Usage
65
+
66
+ ### With PEFT (recommended)
67
+
68
+ ```python
69
+ from transformers import AutoProcessor, AutoModelForImageTextToText
70
+ from peft import PeftModel
71
+ from PIL import Image
72
+
73
+ # Load base + adapter
74
+ base = AutoModelForImageTextToText.from_pretrained(
75
+ "LiquidAI/LFM2.5-VL-450M",
76
+ device_map="auto",
77
+ torch_dtype="auto",
78
+ )
79
+ model = PeftModel.from_pretrained(base, "johnny711/argus-lfm-lora")
80
+ model = model.merge_and_unload() # merge for faster inference
81
+
82
+ processor = AutoProcessor.from_pretrained("LiquidAI/LFM2.5-VL-450M")
83
+
84
+ # Run detection
85
+ image = Image.open("satellite_image.jpg")
86
+ prompt = """You are an orbital intelligence analyst examining satellite imagery \
87
+ from a defense reconnaissance satellite at ~800 km altitude.
88
+
89
+ Detect ALL military-relevant objects visible in this image. For each object, provide:
90
+ - "label": specific type of object
91
+ - "bbox": normalized bounding box [x1, y1, x2, y2] in [0,1]
92
+ - "threat_level": "LOW", "MEDIUM", or "HIGH"
93
+ - "confidence": 0.0 to 1.0
94
+ - "reasoning": brief tactical assessment
95
+
96
+ Return a JSON array. If no targets visible, return: []"""
97
+
98
+ messages = [{"role": "user", "content": [
99
+ {"type": "image", "image": image},
100
+ {"type": "text", "text": prompt},
101
+ ]}]
102
+
103
+ inputs = processor.apply_chat_template(
104
+ messages, add_generation_prompt=True,
105
+ return_tensors="pt", return_dict=True, tokenize=True,
106
+ ).to(model.device)
107
+
108
+ outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.1)
109
+ new_tokens = outputs[:, inputs["input_ids"].shape[1]:]
110
+ result = processor.batch_decode(new_tokens, skip_special_tokens=True)[0]
111
+ print(result)
112
+ ```
113
+
114
+ ### Example Output
115
+
116
+ ```json
117
+ [
118
+ {
119
+ "label": "Small Military Vehicle",
120
+ "bbox": [0.0, 0.3438, 0.0645, 0.0664],
121
+ "threat_level": "LOW",
122
+ "confidence": 0.85,
123
+ "reasoning": "Small Military Vehicle detected near tree cover, likely concealed staging area"
124
+ },
125
+ {
126
+ "label": "Naval Vessel",
127
+ "bbox": [0.0547, 0.5625, 0.0664, 0.0527],
128
+ "threat_level": "HIGH",
129
+ "confidence": 0.82,
130
+ "reasoning": "Naval Vessel visible in desert terrain, limited concealment"
131
+ }
132
+ ]
133
+ ```
134
+
135
+ ## Project ARGUS Architecture
136
+
137
+ ```
138
+ Satellite Image (GigaPixel)
139
+ |
140
+ [Phase 1] LFM2.5-VL + LoRA --> JSON detections (this model)
141
+ |
142
+ [Phase 2] Depth Anything 3 --> 3D reality check (decoy filtering)
143
+ |
144
+ [Phase 3] Report Assembly --> Tactical JSON downlink (bytes, not GB)
145
+ ```
146
+
147
+ **The Problem:** Military satellites capture massive images but have limited downlink bandwidth. Sending gigabytes of raw imagery to ground stations wastes hours.
148
+
149
+ **Our Solution:** Run AI at the edge (in orbit). This adapter enables a 450M-parameter VLM to perform unified detection, classification, and tactical reasoning in a single inference pass — producing a tiny JSON report instead of raw imagery.
150
+
151
+ ## Developed by
152
+
153
+ - **johnny711** — [GitHub](https://github.com/jatin711-debug)
154
+ - **Hackathon:** [Liquid AI x DPhi Space "AI in Space"](https://luma.com/n9cw58h0?tk=nVwuXw)
155
+
156
+ ## License
157
+
158
+ Apache 2.0
159
+
160
+ ---
161
 
162
  This lfm2_vl model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
163