HOW_TO_DEPLOY_SENTENCE_FEATURE.md

# 🚀 How to Deploy Sentence-Level Categorization

## Current Status

✅ **Implementation Complete!**

Branch: `feature/sentence-level-categorization`  
Status: Ready for testing and deployment

---

## Step 1: Push to GitHub

The branch is ready but needs to be pushed to GitHub. You have two options:

### Option A: Command Line (if you have GitHub credentials configured)

```bash
cd /home/thadillo/MyProjects/participatory_planner
git push origin feature/sentence-level-categorization
```

If prompted for credentials:
- Username: `thadillo`
- Password: Your GitHub personal access token (or password)

### Option B: GitHub Desktop or Web Interface

1. Open GitHub Desktop
2. Switch to branch: `feature/sentence-level-categorization`
3. Click "Push origin"

OR

1. Commit files manually through GitHub web interface
2. Upload changed files to new branch

---

## Step 2: Run Migration

Once the branch is pushed (or even before), run the migration locally:

```bash
cd /home/thadillo/MyProjects/participatory_planner
source venv/bin/activate
python migrations/migrate_to_sentence_level.py
```

**Expected output:**
```
Starting sentence-level categorization migration...
Creating new database tables...
✓ Tables created/verified
✓ Found 60 existing submissions
✓ Submissions marked for analysis

✓ MIGRATION COMPLETE!
```

---

## Step 3: Restart the App

```bash
# Kill current instance
pkill -f run.py

# Start fresh
cd /home/thadillo/MyProjects/participatory_planner
source venv/bin/activate
python run.py
```

The app will start on http://127.0.0.1:5000

---

## Step 4: Test the Feature

1. **Open**: http://127.0.0.1:5000/login
2. **Login**: `<see-startup-logs-or-set-ADMIN_TOKEN>`
3. **Go to**: Admin → Submissions

### Test Sentence Analysis:

1. Click **"Analyze All"** button
2. Wait for analysis to complete (1-2 minutes for 60 submissions)
3. Refresh the page

### Check Results:

Each submission should now show:
- ✅ **Category Distribution** badges (e.g., "50% Objective, 50% Problem")
- ✅ **"View Sentences (X)"** button
- ✅ Click to expand and see individual sentences
- ✅ Each sentence has its own category dropdown
- ✅ Confidence scores displayed

### Test Editing:

1. Expand a submission's sentences
2. Change a sentence category from the dropdown
3. Should see green highlight flash (success)
4. Parent submission category updates automatically

---

## Step 5: Merge to Main (When Ready)

After testing successfully:

```bash
cd /home/thadillo/MyProjects/participatory_planner

# Switch to main branch
git checkout main

# Merge feature branch
git merge feature/sentence-level-categorization

# Push to GitHub
git push origin main

# Push to HuggingFace (if deploying there)
git push hf main
```

---

## What You'll See

### Before (Submission-Level):
```
┌─────────────────────────────────┐
│ Community | Not categorized ▼   │
│                                 │
│ "Dallas should establish more   │
│  green spaces in South Dallas.  │
│  Areas lack accessible parks."  │
│                                 │
│ [Category: Objective ▼]         │
└─────────────────────────────────┘
```

### After (Sentence-Level):
```
┌─────────────────────────────────────────┐
│ Community | Objective ▼                 │
│                                         │
│ "Dallas should establish more green     │
│  spaces in South Dallas. Areas lack     │
│  accessible parks."                     │
│                                         │
│ Distribution: 50% Objective, 50% Problem│
│                                         │
│ [▼ View Sentences (2)]                  │
│ ┌─────────────────────────────────────┐│
│ │ 1. "Dallas should establish..."     ││
│ │    [Objective ▼]  Confidence: 87%   ││
│ │                                     ││
│ │ 2. "Areas lack accessible..."       ││
│ │    [Problem ▼]  Confidence: 92%     ││
│ └─────────────────────────────────────┘│
└─────────────────────────────────────────┘
```

---

## Files Changed

### Core Application:
- ✅ `app/models/models.py` - New SubmissionSentence model
- ✅ `app/analyzer.py` - Sentence analysis method
- ✅ `app/routes/admin.py` - API endpoints
- ✅ `app/templates/admin/submissions.html` - UI updates
- ✅ `requirements.txt` - Added nltk

### New Files:
- ✅ `app/utils/text_processor.py` - Sentence segmentation
- ✅ `migrations/migrate_to_sentence_level.py` - Migration script
- ✅ `SENTENCE_LEVEL_FEATURE_README.md` - Documentation

---

## Troubleshooting

### Issue: Migration fails
**Solution**: Make sure you're in the venv and app is not running
```bash
pkill -f run.py
source venv/bin/activate
python migrations/migrate_to_sentence_level.py
```

### Issue: NLTK error
**Solution**: NLTK will auto-download punkt tokenizer on first use. If it fails:
```python
import nltk
nltk.download('punkt')
```

### Issue: Sentences not appearing
**Solution**: 
1. Check submission has `sentence_analysis_done = True`
2. Re-run analysis: Click "Analyze All"
3. Check browser console for errors

### Issue: Can't push to GitHub
**Solution**: 
1. Check git credentials: `git config --list`
2. Use GitHub Desktop instead
3. Or manually upload via GitHub web interface

---

## What's Pending (Optional Enhancements)

The core feature is **complete and functional**. These are nice-to-haves:

### ⏸️ Phase 5: Dashboard Dual-Mode
- Show aggregation by submissions OR sentences
- Category charts with sentence-level option
- **Status**: Can be added later

### ⏸️ Phase 6: Training Data Integration
- Already works! Training examples link to sentences
- Just needs testing
- **Status**: Functional, untested

### ⏸️ Phase 8: Automated Testing
- Unit tests for text processor
- Integration tests for API
- **Status**: Not started

---

## Quick Command Reference

```bash
# Navigate to project
cd /home/thadillo/MyProjects/participatory_planner

# Activate environment
source venv/bin/activate

# Run migration
python migrations/migrate_to_sentence_level.py

# Start app
python run.py

# Push to GitHub
git push origin feature/sentence-level-categorization

# Merge to main
git checkout main
git merge feature/sentence-level-categorization
git push origin main
```

---

## Summary

✅ **What's Done**:
- Database schema with sentence support
- Sentence segmentation (NLTK)
- AI analysis per sentence
- UI with collapsible sentences
- Category editing per sentence
- Training data integration
- Migration script
- Full documentation

🎯 **What to Do Next**:
1. Push branch to GitHub (see Step 1)
2. Run migration (see Step 2)
3. Test feature (see Step 4)
4. Merge to main when satisfied (see Step 5)

📚 **Documentation**:
- This file (deployment guide)
- `SENTENCE_LEVEL_FEATURE_README.md` (feature docs)
- `SENTENCE_LEVEL_CATEGORIZATION_PLAN.md` (technical plan)

---

**You're all set! The feature is ready to test and deploy!** 🚀

Your observation was spot-on, and this implementation solves it completely.