prithivMLmods commited on
Commit
8d4c993
·
verified ·
1 Parent(s): e9db779

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +75 -1
README.md CHANGED
@@ -25,4 +25,78 @@ tags:
25
  - Intelligent Document Processing (IDP)
26
  - Intelligent Word Recognition (IWR)
27
  - Optical Mark Recognition (OMR)
28
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  - Intelligent Document Processing (IDP)
26
  - Intelligent Word Recognition (IWR)
27
  - Optical Mark Recognition (OMR)
28
+ ---
29
+
30
+ # **Gliese-OCR-7B-Post2.0-final**
31
+
32
+ > The **Gliese-OCR-7B-Post2.0-final** model is a refined and optimized version of **[Gliese-OCR-7B-Post1.0](https://huggingface.co/prithivMLmods/Gliese-OCR-7B-Post1.0)**, built upon the **Qwen2.5-VL** architecture. It represents the final iteration in the **Gliese-OCR** series, offering enhanced efficiency, precision, and visualization capabilities for **document OCR**, **visual analysis**, and **information extraction**.
33
+ >
34
+ > Fine-tuned with extended document visualization data and OCR-focused objectives, this model delivers superior accuracy across a wide range of document types, including scanned PDFs, handwritten pages, structured forms, and analytical reports.
35
+
36
+ ## Key Enhancements
37
+
38
+ * **Optimized Document Visualization and OCR Pipeline**: Significantly improved recognition of text, layout, and embedded visuals for structured document understanding.
39
+ * **Context-Aware Multimodal Linking**: Enhanced understanding of document context with stronger alignment between text, images, and layout components.
40
+ * **Refined Document Retrieval**: Improved retrieval accuracy from complex layouts and multi-page documents.
41
+ * **High-Fidelity Content Extraction**: Precise extraction of structured, semi-structured, and unstructured information with advanced text normalization.
42
+ * **Analytical Recognition**: Superior reasoning over charts, graphs, tables, and mathematical equations.
43
+ * **Improved Visual Reasoning and Layout Awareness**: Trained on document visualization datasets for advanced spatial and semantic comprehension.
44
+ * **State-of-the-Art Performance Across Resolutions**: Achieves top results on benchmarks such as DocVQA, InfographicVQA, MathVista, and RealWorldQA.
45
+ * **Extended Multimodal Duration Support**: Handles long document sequences and extended videos (20+ minutes).
46
+ * **Final Release Stability**: Consolidates all prior improvements for stable and reliable performance.
47
+
48
+ ## Quick Start with Transformers
49
+
50
+ ```python
51
+ from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
52
+ from qwen_vl_utils import process_vision_info
53
+
54
+ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
55
+ "prithivMLmods/Gliese-OCR-7B-Post2.0-final", torch_dtype="auto", device_map="auto"
56
+ )
57
+
58
+ processor = AutoProcessor.from_pretrained("prithivMLmods/Gliese-OCR-7B-Post2.0-final")
59
+
60
+ messages = [
61
+ {
62
+ "role": "user",
63
+ "content": [
64
+ {"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"},
65
+ {"type": "text", "text": "Describe the document structure and extract key text content."},
66
+ ],
67
+ }
68
+ ]
69
+
70
+ text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
71
+ image_inputs, video_inputs = process_vision_info(messages)
72
+ inputs = processor(
73
+ text=[text],
74
+ images=image_inputs,
75
+ videos=video_inputs,
76
+ padding=True,
77
+ return_tensors="pt",
78
+ ).to("cuda")
79
+
80
+ generated_ids = model.generate(**inputs, max_new_tokens=256)
81
+ generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
82
+ output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
83
+ print(output_text)
84
+ ```
85
+
86
+ ## Intended Use
87
+
88
+ * Document visualization and OCR extraction tasks.
89
+ * Context-aware document retrieval and multimodal linking.
90
+ * Extraction and LaTeX formatting of equations and structured content.
91
+ * Analytical document interpretation (charts, tables, graphs, and figures).
92
+ * Multilingual OCR for enterprise, academic, and research use cases.
93
+ * Summarization, question answering, and cross-modal reasoning over long documents.
94
+ * Intelligent robotic or mobile automation guided by visual document input.
95
+
96
+ ## Limitations
97
+
98
+ * Reduced accuracy on heavily degraded or occluded documents.
99
+ * High computational requirements for large-scale or real-time applications.
100
+ * Limited optimization for low-resource or edge devices.
101
+ * Occasional misalignment in text layout or minor hallucinations in outputs.
102
+ * Performance may vary depending on visual token configuration and context length settings.