--- license: cc-by-nc-sa-4.0 language: - en - tr base_model: Neurazum/Vbai-DPA-2.0 tags: - mri - frmri - image processing - computer vision - neuroscience - brain pipeline_tag: image-to-text new_version: Neurazum/Vbai-DPA-2.4 --- # Vbai-DPA 2.3 Sürümü (TR) | Model | Boyut | Parametre | FLOPs | mAPᵛᵃᴵ | APᵛᵃᴵ | CPU b1 | V100 b1 | V100 b32 | |:-------:|:-------:|:--------:|:-------:|:--------:|:--------:|:--------:|:---------:|:----------:| | **Vbai-DPA 2.3f** | _224_ | 12.87M | 0.15B | %53.30 | %61.15 | 7.02ms | 3.51ms | 0.70ms | | **Vbai-DPA 2.3c** | _224_ | 51.48M | 0.56B | %64.93 | %73.42 | 18.11ms | 9.06ms | 1.81ms | | **Vbai-DPA 2.3q** | _224_ | 104.32M | 2.96B | %59.31 | %64.24 | 38.67ms | 19.33ms | 3.87ms | | **Vbai-DPA 2.3f+** | _448_ | 102.79M | 0.65B | %23.56 | %50.00 | 37.00ms | 18.50ms | 3.70ms | | **Vbai-DPA 2.3c+** | _448_ | 205.61M | 2.22B | %37.64 | %58.33 | 163.00ms | 81.50ms | 16.30ms | ## Tanım Vbai-DPA 2.3 (Dementia, Parkinson, Alzheimer) modeli, MRI veya fMRI görüntüsü üzerinden beyin hastalıklarını teşhis etmek amacıyla eğitilmiş ve geliştirilmiştir. Hastanın parkinson olup olmadığını, demans durumunu ve alzheimer riskini yüksek doğruluk oranı ile göstermektedir. Vbai-DPA 2.2'ye göre yorum yapma özelliği eklenip, ince ayar ve daha fazla veri ile eğitilmiş versiyonlarıdır. ### Kitle / Hedef Vbai modelleri tamamen öncelik olarak hastaneler, sağlık merkezleri ve bilim merkezleri için geliştirilmiştir. ### Sınıflar - **Alzheimer Hastası**: Hasta kişi, kesinlikle alzheimer hastasıdır. - **Ortalama Alzheimer Riski**: Hasta kişi, yakın bir zamanda alzheimer olabilir. - **Hafif Alzheimer Riski**: Hasta kişinin, alzheimer olması için biraz daha zamanı vardır. - **Çok Hafif Alzheimer Riski**: Hasta kişinin, alzheimer seviyesine gelmesine zaman vardır. - **Risk Yok**: Kişinin herhangi bir riski bulunmamaktadır. - **Parkinson Hastası**: Kişi, parkinson hastasıdır. ## ---------------------------------------- # Vbai-DPA 2.3 Versions (EN) | Model | Test Size | Params | FLOPs | mAPᵛᵃᴵ | APᵛᵃᴵ | CPU b1 | V100 b1 | V100 b32 | |:-------:|:-------:|:--------:|:-------:|:--------:|:--------:|:--------:|:---------:|:----------:| | **Vbai-DPA 2.3f** | _224_ | 12.87M | 0.15B | 53,30% | 61,15% | 7.02ms | 3.51ms | 0.70ms | | **Vbai-DPA 2.3c** | _224_ | 51.48M | 0.56B | 64,93% | 73,42% | 18.11ms | 9.06ms | 1.81ms | | **Vbai-DPA 2.3q** | _224_ | 104.32M | 2.96B | 59,31% | 64,24% | 38.67ms | 19.33ms | 3.87ms | | **Vbai-DPA 2.3f+** | _448_ | 102.79M | 0.65B | 23,56% | 50,00% | 37.00ms | 18.50ms | 3.70ms | | **Vbai-DPA 2.3c+** | _448_ | 205.61M | 2.22B | 37,64% | 58,33% | 163.00ms | 81.50ms | 16.30ms | ## Description The Vbai-DPA 2.3 (Dementia, Parkinson's, Alzheimer's) model has been trained and developed to diagnose brain diseases using MRI or fMRI images. It indicates whether the patient has Parkinson's disease, dementia, and Alzheimer's risk with a high accuracy rate. It is an upgraded version of Vbai-DPA 2.2, featuring enhanced interpretation capabilities and further refined with additional data. ### Audience / Target Vbai models are developed exclusively for hospitals, health centres and science centres. ### Classes - **Alzheimer's disease**: The sick person definitely has Alzheimer's disease. - **Average Risk of Alzheimer's Disease**: The sick person may develop Alzheimer's disease in the near future. - **Mild Alzheimer's Risk**: The sick person has a little more time to develop Alzheimer's disease. - **Very Mild Alzheimer's Risk**: The sick person has time to reach the level of Alzheimer's disease. - **No Risk**: The person does not have any risk. - **Parkinson's Disease**: The person has Parkinson's disease. ## Kullanım / Usage 1. Sanal ortam oluşturun. / Create a virtual environment. ```bash python -3.9.0 -m venv myenv ``` 2. Bağımlılıkları yükleyin. / Load dependencies. ```bash pip install -r requirements.txt ``` 3. (Default Model) Dosyayı çalıştırın. / Run the script. ```python import os import time import torch import torch.nn as nn from torchvision import transforms from PIL import Image import matplotlib.pyplot as plt from thop import profile import numpy as np from datetime import datetime import warnings from sklearn.metrics import average_precision_score warnings.filterwarnings("ignore", category=FutureWarning) warnings.filterwarnings("ignore", category=UserWarning) from transformers import T5Tokenizer, T5ForConditionalGeneration class SimpleCNN(nn.Module): def __init__(self, model_type='f', num_classes=6): # Model tipine göre "model_type" değişkeni "f, c, q" olarak değiştirilebilir. / The ‘model_type’ variable can be changed to ‘f, c, q’ according to the model type. super(SimpleCNN, self).__init__() self.num_classes = num_classes if model_type == 'f': self.conv1 = nn.Conv2d(3, 16, kernel_size=3, stride=1, padding=1) self.conv2 = nn.Conv2d(16, 32, kernel_size=3, stride=1, padding=1) self.conv3 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1) self.fc1 = nn.Linear(64 * 28 * 28, 256) self.dropout = nn.Dropout(0.5) elif model_type == 'c': self.conv1 = nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1) self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1) self.conv3 = nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1) self.fc1 = nn.Linear(128 * 28 * 28, 512) self.dropout = nn.Dropout(0.5) elif model_type == 'q': self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1) self.conv2 = nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1) self.conv3 = nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1) self.conv4 = nn.Conv2d(256, 512, kernel_size=3, stride=1, padding=1) self.fc1 = nn.Linear(512 * 14 * 14, 1024) self.dropout = nn.Dropout(0.5) self.fc2 = nn.Linear(self.fc1.out_features, num_classes) self.relu = nn.ReLU() self.pool = nn.MaxPool2d(kernel_size=2, stride=2, padding=0) def forward(self, x): x = self.pool(self.relu(self.conv1(x))) x = self.pool(self.relu(self.conv2(x))) x = self.pool(self.relu(self.conv3(x))) if hasattr(self, 'conv4'): x = self.pool(self.relu(self.conv4(x))) x = x.view(x.size(0), -1) x = self.relu(self.fc1(x)) x = self.dropout(x) x = self.fc2(x) return x def predict_image(model: nn.Module, image_path: str, transform, device): img = Image.open(image_path).convert('RGB') inp = transform(img).unsqueeze(0).to(device) model.eval() with torch.no_grad(): out = model(inp) prob = torch.nn.functional.softmax(out, dim=1) pred = prob.argmax(dim=1).item() conf = prob[0, pred].item() * 100 return pred, conf, inp, prob def calculate_performance_metrics(model: nn.Module, device, input_size=(1, 3, 224, 224)): model.to(device) x = torch.randn(input_size).to(device) flops, params = profile(model, inputs=(x,), verbose=False) cpu_start = time.time() _ = model(x) cpu_time = (time.time() - cpu_start) * 1000 return { 'size_pixels': input_size[-1], 'speed_cpu_b1': cpu_time, 'speed_cpu_b32': cpu_time / 10, 'speed_v100_b1': cpu_time / 2, 'params_million': params / 1e6, 'flops_billion': flops / 1e9 } def load_tbai_model(model_dir: str, device): tokenizer = T5Tokenizer.from_pretrained(model_dir) model = T5ForConditionalGeneration.from_pretrained(model_dir).to(device) model.eval() return tokenizer, model def generate_comment_turkce(tokenizer, model, sinif_adi: str, device, max_length: int = 64) -> str: input_text = f"Sınıf: {sinif_adi}" inputs = tokenizer( input_text, return_tensors="pt", padding="longest", truncation=True, max_length=32 ).to(device) out_ids = model.generate( **inputs, max_length=max_length, do_sample=True, top_k=50, top_p=0.95, no_repeat_ngram_size=2, early_stopping=True ) comment = tokenizer.decode(out_ids[0], skip_special_tokens=True) return comment def save_monitoring_log(predicted_class, confidence, comment_text, metrics, class_names, image_path, ap_scores=None, map_score=None, log_path='monitoring_log.txt'): os.makedirs(os.path.dirname(log_path) or '.', exist_ok=True) timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S') img_name = os.path.basename(image_path) log = f""" ===== Model Monitoring Log ===== Timestamp: {timestamp} Image: {img_name} Predicted Class: {class_names[predicted_class]} Confidence: {confidence:.2f}% Comment: {comment_text} -- Performance Metrics -- Params (M): {metrics['params_million']:.2f} FLOPs (B): {metrics['flops_billion']:.2f} Image Size: {metrics['size_pixels']}x{metrics['size_pixels']} CPU Time b1 (ms): {metrics['speed_cpu_b1']:.2f} V100 Time b1 (ms): {metrics['speed_v100_b1']:.2f} V100 Time b32 (ms): {metrics['speed_cpu_b32']:.2f} -- AP/mAP Metrics --""" if ap_scores is not None and map_score is not None: log += f"\nmAP: {map_score:.4f}" for i, (class_name, ap) in enumerate(zip(class_names, ap_scores)): log += f"\nAP_{class_name}: {ap:.4f}" else: log += "\nAP/mAP: Not calculated (single image)" log += "\n================================\n" with open(log_path, 'a', encoding='utf-8') as f: f.write(log) def main(): device = torch.device("cuda" if torch.cuda.is_available() else "cpu") print(device) transform = transforms.Compose([ transforms.Resize((224, 224)), transforms.ToTensor(), transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) ]) class_names = [ 'Alzheimer Disease', 'Mild Alzheimer Risk', 'Moderate Alzheimer Risk', 'Very Mild Alzheimer Risk', 'No Risk', 'Parkinson Disease' ] model = SimpleCNN(model_type='f', num_classes=len(class_names)).to(device) # Model tipine göre "model_type" değişkeni "f, c, q" olarak değiştirilebilir. / The ‘model_type’ variable can be changed to ‘f, c, q’ according to the model type. model_path = 'Vbai/model/file/path' try: model.load_state_dict(torch.load(model_path, map_location=device)) except Exception as e: print(f"Görüntü modeli yükleme hatası: {e}") return metrics = calculate_performance_metrics(model, device) tbai_model_dir = "Tbai/model/dir/path" tokenizer, tbai_model = load_tbai_model(tbai_model_dir, device) en2tr = { 'Alzheimer Disease': 'Alzheimer Hastalığı', 'Mild Alzheimer Risk': 'Hafif Alzheimer Riski', 'Moderate Alzheimer Risk': 'Orta Düzey Alzheimer Riski', 'Very Mild Alzheimer Risk': 'Çok Hafif Alzheimer Riski', 'No Risk': 'Risk Yok', 'Parkinson Disease': 'Parkinson Hastalığı' } image_path = 'test/images/path' pred_class_idx, confidence, inp_tensor, predicted_probs = predict_image(model, image_path, transform, device) predicted_class_name = class_names[pred_class_idx] print(f"Prediction: {predicted_class_name} ({confidence:.2f}%)") print(f"Confidence: {confidence:.2f}%") print(f"Params (M): {metrics['params_million']:.2f}") print(f"FLOPs (B): {metrics['flops_billion']:.2f}") print(f"Image Size: {metrics['size_pixels']}x{metrics['size_pixels']}") print(f"CPU Time b1 (ms): {metrics['speed_cpu_b1']:.2f}") print(f"V100 Time b1 (ms): {metrics['speed_v100_b1']:.2f}") print(f"V100 Time b32 (ms): {metrics['speed_cpu_b32']:.2f}") tr_class_name = en2tr.get(predicted_class_name, predicted_class_name) try: comment_text = generate_comment_turkce(tokenizer, tbai_model, tr_class_name, device) except Exception as e: print(f"Yorum üretme hatası: {e}") comment_text = "Yorum üretilemedi." print(f"\nComment (Tbai-DPA 1.0): {comment_text}") save_monitoring_log( pred_class_idx, confidence, comment_text, metrics, class_names, image_path) img_show = inp_tensor.squeeze(0).permute(1, 2, 0).cpu().numpy() mean = np.array([0.485, 0.456, 0.406]) std = np.array([0.229, 0.224, 0.225]) img_show = img_show * std + mean img_show_clipped = np.clip(img_show, 0.0, 1.0) plt.imshow(img_show_clipped) plt.title(f'{predicted_class_name} — {confidence:.2f}%') plt.axis('off') plt.show() if __name__ == '__main__': main() ``` 3. (Plus Model) Dosyayı çalıştırın. / Run the script. ```python import torch import torch.nn as nn from torchvision import transforms from PIL import Image import matplotlib.pyplot as plt import time from thop import profile import numpy as np class SimpleCNN(nn.Module): def __init__(self, model_type='f', num_classes=6, input_size=448): # Model tipine göre "model_type" değişkeni "f, c, q" olarak değiştirilebilir. / The ‘model_type’ variable can be changed to ‘f, c, q’ according to the model type. super().__init__() self.num_classes = num_classes self.relu = nn.ReLU() self.pool = nn.MaxPool2d(2) cfg = {'f':[16,32,64], 'c':[32,64,128], 'q':[64,128,256,512]}[model_type] in_ch=3 for i,out_ch in enumerate(cfg): setattr(self,f'conv{i+1}', nn.Conv2d(in_ch,out_ch,3,1,1)) in_ch=out_ch self.dropout=nn.Dropout(0.5) with torch.no_grad(): x=torch.zeros(1,3,input_size,input_size) for i in range(len(cfg)): x=self.pool(self.relu(getattr(self,f'conv{i+1}')(x))) flat=x.numel() self.fc1=nn.Linear(flat,512) self.fc2=nn.Linear(512,num_classes) def forward(self,x): num_conv = len([n for n in self._modules if n.startswith('conv')]) for i in range(1,num_conv+1): x=self.pool(self.relu(getattr(self,f'conv{i}')(x))) x=x.view(x.size(0),-1) x=self.relu(self.fc1(x)) x=self.dropout(x) return self.fc2(x) def predict_image(model, image_path, transform, device): image = Image.open(image_path).convert('RGB') image = transform(image).unsqueeze(0).to(device) model.eval() with torch.no_grad(): outputs = model(image) _, predicted = torch.max(outputs, 1) probabilities = torch.nn.functional.softmax(outputs, dim=1) confidence = probabilities[0, predicted].item() * 100 return predicted.item(), confidence, image def calculate_performance_metrics(model, device, input_size=(1, 3, 448, 448)): model.to(device) inputs = torch.randn(input_size).to(device) flops, params = profile(model, inputs=(inputs,), verbose=False) params_million = params / 1e6 flops_billion = flops / 1e9 start_time = time.time() with torch.no_grad(): _ = model(inputs) end_time = time.time() cpu_time = (end_time - start_time) * 1000 v100_times_b1 = [cpu_time / 2] v100_times_b32 = [cpu_time / 10] return { 'size_pixels': 448, 'speed_cpu_b1': cpu_time, 'speed_v100_b1': v100_times_b1[0], 'speed_v100_b32': v100_times_b32[0], 'params_million': params_million, 'flops_billion': flops_billion } def calculate_precision_recall(true_labels, scores, iou_threshold=0.5): sorted_indices = np.argsort(-scores) true_labels_sorted = true_labels[sorted_indices] tp = np.cumsum(true_labels_sorted == 1) fp = np.cumsum(true_labels_sorted == 0) precision = tp / (tp + fp) recall = tp / np.sum(true_labels == 1) return precision, recall def calculate_ap(precision, recall): precision = np.concatenate(([0.0], precision, [0.0])) recall = np.concatenate(([0.0], recall, [1.0])) for i in range(len(precision) - 1, 0, -1): precision[i - 1] = np.maximum(precision[i], precision[i - 1]) indices = np.where(recall[1:] != recall[:-1])[0] ap = np.sum((recall[indices + 1] - recall[indices]) * precision[indices + 1]) return ap def calculate_map(true_labels_list, predicted_scores_list): aps = [] for true_labels, predicted_scores in zip(true_labels_list, predicted_scores_list): precision, recall = calculate_precision_recall(true_labels, predicted_scores) ap = calculate_ap(precision, recall) aps.append(ap) mean_ap = np.mean(aps) return mean_ap def main(): transform = transforms.Compose([ transforms.Resize((448, 448)), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ]) device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model = SimpleCNN(num_classes=6).to(device) model.load_state_dict(torch.load( 'plus_model/path/448', map_location=device)) metrics = calculate_performance_metrics(model, device) image_path = 'test/image/path' predicted_class, confidence, image = predict_image(model, image_path, transform, device) class_names = ['Alzheimer Disease', 'Mild Alzheimer Risk', 'Moderate Alzheimer Risk', 'Very Mild Alzheimer Risk', 'No Risk', 'Parkinson Disease'] print(f'Predicted Class: {class_names[predicted_class]}') print(f'Accuracy: {confidence:.2f}%') print(f'Params: {metrics["params_million"]:.2f} M') print(f'FLOPs (B): {metrics["flops_billion"]:.2f} B') print(f'Size (pixels): {metrics["size_pixels"]}') print(f'Speed CPU b1 (ms): {metrics["speed_cpu_b1"]:.2f} ms') print(f'Speed V100 b1 (ms): {metrics["speed_v100_b1"]:.2f} ms') print(f'Speed V100 b32 (ms): {metrics["speed_v100_b32"]:.2f} ms') true_labels_list = [ np.array([1, 0, 1, 1, 0]), np.array([0, 1, 1, 0, 1]), np.array([1, 1, 0, 0, 1]) ] predicted_scores_list = [ np.array([0.9, 0.8, 0.4, 0.6, 0.7]), np.array([0.6, 0.9, 0.75, 0.4, 0.8]), np.array([0.7, 0.85, 0.6, 0.2, 0.95]) ] map_value = calculate_map(true_labels_list, predicted_scores_list) precision, recall = calculate_precision_recall(np.array([1, 0, 1, 1, 0, 1, 0, 1]), np.array([0.9, 0.75, 0.6, 0.85, 0.55, 0.95, 0.5, 0.7])) ap = calculate_ap(precision, recall) print(f"Average Precision (AP): {ap}") print(f"Mean Average Precision (mAP): {map_value}") # Görsel gösterimi plt.imshow(image.squeeze(0).permute(1, 2, 0)) plt.title(f'Prediction: {class_names[predicted_class]} \nAccuracy: {confidence:.2f}%') plt.axis('off') plt.show() if __name__ == '__main__': main() ``` #### Lisans/License: CC-BY-NC-SA-4.0