---
datasets:
- QCRI/CrisisMMD
language:
- en
metrics:
- accuracy
- f1
- recall
- precision
base_model:
- google-bert/bert-base-uncased
- microsoft/resnet-50
---
Source: CrisisMMD dataset (Alam et al., 2017)

✅Original Labels (8 classes from annotations):

Infrastructure and utility damage

Vehicle damage

Rescue, volunteering, or donation efforts

Affected individuals

Injured or dead people

Missing or found people

Other relevant information

Not humanitarian

✅Label Preprocessing (Class Merging):

Vehicle damage merged into Infrastructure and utility damage

Missing or found people merged into Affected individuals

Not humanitarian retained as a separate class

Removed very low-frequency categories (e.g., "Missing or found people" as a separate class)

✅Final Label Set (5 classes total):

Infrastructure and utility damage

Rescue, volunteering, or donation efforts

Affected individuals

Injured or dead people

Not humanitarian

✅Multimodal Consistency:

Selected only those posts where text and image annotations matched

Resulted in a total of 8,219 consistent samples:

Train set: 6,574 posts

Test set: 1,644 posts

✅ Preprocessing Done
Text:

Tokenized using BERT tokenizer (bert-base-uncased)

Extracted input_ids and attention_mask

Image:

Processed using ResNet-50

Extracted 2048-dimensional image features

The preprocessed data was saved in PyTorch .pt format:

train_human.pt and test_human.pt

Each contains: input_ids, attention_mask, image_vector, and label

✅ Model Architecture
A custom multimodal classifier that combines BERT and ResNet-50 outputs:

Component	Details
Text Encoder	BERT base (bert-base-uncased) – outputs pooler_output (768-d)
Image Encoder	Pre-extracted ResNet-50 image features (2048-d)
Fusion	Concatenation → FC layers → Softmax over 5 classes
Classifier	Fully connected layers with BatchNorm, ReLU, Dropout

✅ Training Setup
Loss Function: CrossEntropyLoss

Optimizer: AdamW

Scheduler: StepLR (γ = 0.9)

Epochs Tried: 1, 3, 5, 8, 10

Batch Size: 16

Runtime: ~2 minutes 20 seconds per epoch on Google Colab (T4 GPU)

✅ Evaluation Metrics
Accuracy

Precision

Recall

F1 Score

✅ Metrics(epoch 3 with highest accuracy)
 
✅ Test Accuracy : 0.8820
✅ Precision     : 0.6854
✅ Recall        : 0.7176
✅ F1 Score      : 0.7005

The new dataset created: https://huggingface.co/datasets/Henishma/crisisMMD_cleaned_task2