ZERO_SHOT_MODEL_SELECTION.md

Zero-Shot Model Selection Feature

Overview

You can now choose which AI model to use for zero-shot classification! This allows you to balance between accuracy and speed based on your needs.

Available Zero-Shot Models

1. BART-large-MNLI (Current Default)

• Size: 400M parameters
• Speed: Slow
• Best for: Maximum accuracy, works out of the box
• Description: Large sequence-to-sequence model, excellent zero-shot performance
• Model ID: facebook/bart-large-mnli

2. DeBERTa-v3-base-MNLI ⭐ Recommended

• Size: 86M parameters (4.5x smaller than BART)
• Speed: Fast
• Best for: Fast zero-shot classification with good accuracy
• Description: DeBERTa trained on NLI datasets, excellent zero-shot with better speed
• Model ID: MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli

3. DistilBART-MNLI

• Size: 134M parameters
• Speed: Medium
• Best for: Balanced zero-shot performance
• Description: Distilled BART for zero-shot, good balance of speed and accuracy
• Model ID: valhalla/distilbart-mnli-12-3

How to Use

Step 1: Go to Training Page

1. Navigate to Admin Panel → Training tab
2. Look for the "Zero-Shot Classification Model" section at the top

Step 2: View Current Model

• The dropdown shows the currently active model
• Below it, you'll see model information (size, speed, description)

Step 3: Change Model

1. Select a different model from the dropdown
2. The system will ask for confirmation
3. The analyzer will reload with the new model
4. All future classifications will use the selected model

Step 4: Test It

• Go to Submissions page
• Click "Re-analyze" on any submission
• The new model will be used for classification!

When to Use Each Model

Use BART-large-MNLI if:

• ✅ Accuracy is more important than speed
• ✅ You have powerful hardware
• ✅ You don't mind waiting a bit longer

Use DeBERTa-v3-base-MNLI if: ⭐ RECOMMENDED

• ✅ You want good accuracy with better speed
• ✅ You're working with many submissions
• ✅ You want to save computational resources
• ✅ You need faster response times

Use DistilBART-MNLI if:

• ✅ You want something in between
• ✅ You're familiar with BART but need better speed

Technical Details

How It Works

1. Settings Storage: The selected model is stored in the database (Settings table)
2. Dynamic Loading: The analyzer checks the setting and loads the selected model
3. Hot Reload: When you change models, the analyzer reloads automatically
4. No Data Loss: Changing models doesn't affect your training data or fine-tuned models

Model Persistence

• The selected model remains active even after app restart
• Each submission classification uses the currently active zero-shot model
• Fine-tuned models override zero-shot models when deployed

API Endpoints

Get Current Model:

GET /admin/api/get-zero-shot-model

Change Model:

POST /admin/api/set-zero-shot-model
Body: {"model_key": "deberta-v3-base-mnli"}

Performance Comparison

Model	Parameters	Classification Speed	Relative Accuracy
BART-large-MNLI	400M	1x (baseline)	100%
DeBERTa-v3-base-MNLI	86M	~4x faster	~95-98%
DistilBART-MNLI	134M	~2x faster	~92-95%

Note: Actual performance may vary based on your hardware and text length

Fine-Tuning vs Zero-Shot

Zero-Shot Model Selection

• When: Before you have training data
• What: Chooses which pre-trained model to use
• Where: Admin → Training → Zero-Shot Classification Model
• Effect: Affects all new classifications immediately

Fine-Tuning Model Selection

• When: When training with your labeled data
• What: Chooses which model architecture to fine-tune
• Where: Admin → Training → Base Model Architecture for Fine-Tuning
• Effect: Only affects that specific training run

Can I use both?

Yes! You can:

1. Select a zero-shot model (e.g., DeBERTa-v3-base-MNLI) for initial classifications
2. Fine-tune using any model (e.g., DeBERTa-v3-small) for better performance
3. Deploy the fine-tuned model, which will override the zero-shot model

Troubleshooting

Q: I changed the model but nothing happened?
A: The change affects new classifications. Try clicking "Re-analyze" on a submission to see the new model in action.

Q: Which model should I choose?
A: Start with DeBERTa-v3-base-MNLI - it's faster than BART with minimal accuracy loss.

Q: Does this affect my fine-tuned models?
A: No! Zero-shot models are only used when no fine-tuned model is deployed.

Q: Can I switch back to BART?
A: Yes! Just select BART-large-MNLI from the dropdown anytime.

Q: Will changing models break anything?
A: No, it's completely safe. Your data, training runs, and fine-tuned models are unaffected.

Best Practices

1. Start with DeBERTa-v3-base-MNLI for better speed
2. Compare results - try re-analyzing the same submission with different models
3. Consider your hardware - larger models need more RAM
4. Fine-tune eventually - zero-shot is great, but fine-tuning is better!

Example Workflow

1. Install app
   ↓
2. Select DeBERTa-v3-base-MNLI (for speed)
   ↓
3. Collect submissions
   ↓
4. Correct categories (builds training data)
   ↓
5. Fine-tune using DeBERTa-v3-small (best for small datasets)
   ↓
6. Deploy fine-tuned model (overrides zero-shot)
   ↓
7. Enjoy better accuracy! 🎉

What's Next?

After selecting your zero-shot model:

• Collect data: Let users submit and classify with the selected model
• Review & correct: Use the admin panel to fix any misclassifications
• Build training set: Corrections are automatically saved
• Fine-tune: Once you have 20+ examples, train a custom model
• Deploy: Your fine-tuned model will outperform any zero-shot model!

---

Ready to try it? Go to Admin → Training and select your model! 🚀

For questions or issues:

1. Check the model info displayed below the dropdown
2. Review this guide
3. Try switching back to BART if issues occur