HUGGINGFACE_OLLAMA_DEPLOY.md

# Deploying AI API Service to Hugging Face Spaces with Ollama

This guide shows you how to deploy the AI API service to Hugging Face Spaces using Ollama as your LLM backend (no API keys needed!).

## Why Ollama on Hugging Face Spaces?

✅ **No API costs** - Run models locally in your Space  
✅ **Privacy** - Data stays within your Space  
✅ **Model choice** - Use Llama 2, Llama 3, Mistral, Phi, Gemma, etc.  
✅ **No rate limits** - Only limited by Space hardware  
✅ **Full control** - Customize models and parameters  

## Prerequisites

- Hugging Face account (free)
- Basic knowledge of Git

## Step-by-Step Deployment

### 1. Create a New Space

1. Go to https://huggingface.co/new-space
2. Choose:
   - **Name**: `ai-api-ollama` (or your preferred name)
   - **License**: MIT
   - **SDK**: Docker
   - **Hardware**: 
     - **CPU Basic (free)**: Works for small models (phi, gemma:2b)
     - **CPU Upgrade ($0.60/hr)**: Better for medium models (llama2, mistral)
     - **GPU T4 ($0.60/hr)**: Recommended for fast inference
     - **GPU A10G ($3.15/hr)**: For large models (llama3:70b)
3. Click **Create Space**

### 2. Clone Your Space Repository

```bash
git clone https://huggingface.co/spaces/YOUR_USERNAME/ai-api-ollama
cd ai-api-ollama
```

### 3. Copy Project Files

Copy all files from this project to your Space directory:

```bash
# From the ai-api-service directory
cp -r backend examples tests *.md *.json *.yml .dockerignore .env.example ../ai-api-ollama/
```

### 4. Create Hugging Face Space Dockerfile

Create a new `Dockerfile` optimized for Hugging Face Spaces with Ollama:

```dockerfile
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package*.json ./
RUN npm ci

# Copy source code
COPY . .

# Build the application
RUN npm run build || echo "Build step skipped - Encore will build on startup"

# Production stage with Ollama
FROM node:18

WORKDIR /app

# Install Ollama
RUN curl -fsSL https://ollama.com/install.sh | sh

# Copy built application
COPY --from=builder /app ./

# Install production dependencies
RUN npm ci --only=production

# Set environment variables for Hugging Face Spaces
ENV PORT=7860
ENV OLLAMA_BASE_URL=http://localhost:11434
ENV OLLAMA_MODEL=llama2
ENV OLLAMA_EMBEDDING_MODEL=nomic-embed-text
ENV API_KEYS=demo-key-1,demo-key-2
ENV RATE_LIMIT_DEFAULT=60
ENV RATE_LIMIT_ADMIN=1000
ENV LOG_LEVEL=info
ENV ENABLE_BACKGROUND_WORKERS=true

EXPOSE 7860

# Create startup script
RUN echo '#!/bin/bash\n\
# Start Ollama in background\n\
ollama serve &\n\
OLLAMA_PID=$!\n\
\n\
# Wait for Ollama to start\n\
echo "Waiting for Ollama to start..."\n\
sleep 5\n\
\n\
# Pull the model\n\
echo "Pulling Ollama model: $OLLAMA_MODEL"\n\
ollama pull $OLLAMA_MODEL || echo "Model pull failed, will try on first request"\n\
\n\
# Pull embedding model if different\n\
if [ "$OLLAMA_EMBEDDING_MODEL" != "$OLLAMA_MODEL" ]; then\n\
  echo "Pulling embedding model: $OLLAMA_EMBEDDING_MODEL"\n\
  ollama pull $OLLAMA_EMBEDDING_MODEL || echo "Embedding model pull failed"\n\
fi\n\
\n\
# Start the API service\n\
echo "Starting AI API Service on port $PORT..."\n\
node .encore/build/backend/main.js || npm start\n\
' > /app/start.sh && chmod +x /app/start.sh

CMD ["/app/start.sh"]
```

### 5. Configure Environment Variables in Space Settings

In your Space settings on Hugging Face:

1. Go to **Settings** → **Variables and secrets**
2. Add these environment variables:

| Variable | Value | Description |
|----------|-------|-------------|
| `API_KEYS` | `your-secret-key-here` | Comma-separated API keys for authentication |
| `ADMIN_API_KEYS` | `admin-key-here` | Admin-level API keys (optional) |
| `OLLAMA_MODEL` | `llama2` | Default: llama2, or use llama3, mistral, phi, gemma |
| `OLLAMA_EMBEDDING_MODEL` | `nomic-embed-text` | Embedding model for RAG |
| `RATE_LIMIT_DEFAULT` | `100` | Requests per minute for default users |

**Recommended Models by Hardware:**

| Hardware | Recommended Model | Speed | Quality |
|----------|------------------|-------|---------|
| CPU Basic | `phi:latest` or `gemma:2b` | Fast | Good |
| CPU Upgrade | `llama2:latest` or `mistral:latest` | Medium | Better |
| GPU T4 | `llama3:latest` | Fast | Excellent |
| GPU A10G | `llama3:70b` | Medium | Best |

### 6. Create README.md for Your Space

Create a `README.md` in your Space root:

```markdown
---
title: AI API Service with Ollama
emoji: 🤖
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
---

# AI API Service with Ollama

Production-ready AI API with chat, RAG, image generation, and voice synthesis.

## Features

- 💬 Multi-turn chat conversations
- 📚 RAG (Retrieval-Augmented Generation)
- 🖼️ Image generation
- 🎙️ Voice synthesis
- 📄 Document ingestion
- 🔒 API key authentication
- ⚡ Rate limiting

## Quick Start

### API Documentation

Base URL: `https://YOUR_USERNAME-ai-api-ollama.hf.space`

### Example Request

```bash
curl -X POST https://YOUR_USERNAME-ai-api-ollama.hf.space/ai/chat \
  -H "Authorization: Bearer demo-key-1" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation": [
      {"role": "user", "content": "Hello! How are you?"}
    ]
  }'
```

### Available Endpoints

- `GET /health` - Health check
- `POST /ai/chat` - Chat conversation
- `POST /rag/query` - Query with retrieval
- `POST /image/generate` - Generate images
- `POST /voice/synthesize` - Text to speech
- `POST /upload` - Upload documents

See full API documentation in the repository.

## Using Your Own API Key

Replace `demo-key-1` with your configured API key from Space settings.

## Local Development

See [QUICKSTART.md](QUICKSTART.md) for local setup instructions.
```

### 7. Push to Hugging Face

```bash
git add .
git commit -m "Initial deployment with Ollama"
git push
```

### 8. Wait for Build

- Hugging Face will automatically build your Docker image
- This takes 5-10 minutes for first build
- Watch the **Logs** tab for progress
- Initial startup will download the Ollama model (2-5 minutes depending on model size)

### 9. Test Your Deployment

Once the Space is running:

```bash
# Replace YOUR_USERNAME with your Hugging Face username
SPACE_URL="https://YOUR_USERNAME-ai-api-ollama.hf.space"

# Health check
curl $SPACE_URL/health

# Chat request
curl -X POST $SPACE_URL/ai/chat \
  -H "Authorization: Bearer demo-key-1" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation": [
      {"role": "user", "content": "Tell me a joke about AI"}
    ]
  }'
```

## Optimizations for Hugging Face Spaces

### 1. Reduce Model Download Time

Pre-download models in Dockerfile:

```dockerfile
RUN ollama pull llama2 && \
    ollama pull nomic-embed-text
```

### 2. Use Smaller Models for Free Tier

```env
OLLAMA_MODEL=phi:latest
```

Phi is only 1.3GB vs Llama2's 4GB.

### 3. Enable Persistent Storage

Hugging Face Spaces have persistent storage in `/data`:

```dockerfile
# Add to Dockerfile
VOLUME /data
ENV OLLAMA_MODELS=/data/ollama-models
```

This prevents re-downloading models on restart.

### 4. Optimize for Cold Starts

Add model warmup in startup script:

```bash
# Add to start.sh
echo "Warming up model..."
ollama run $OLLAMA_MODEL "Hello" --timeout 10s
```

## Cost Comparison

| Option | Cost | Pros | Cons |
|--------|------|------|------|
| **Free CPU** | $0 | Free! | Slow inference, small models only |
| **CPU Upgrade** | $0.60/hr (~$432/mo) | Better performance | Still slower than GPU |
| **GPU T4** | $0.60/hr (~$432/mo) | Fast inference | Limited for huge models |
| **OpenAI API** | Pay per token | No hosting, fast | Ongoing costs, data sent to OpenAI |
| **Self-hosted** | VPS costs | Full control | Maintenance required |

**Recommendation**: Start with **Free CPU + Phi** for testing, upgrade to **GPU T4 + Llama3** for production.

## Troubleshooting

### Space won't start

**Check logs for**:
- Ollama installation errors → Use official Ollama install script
- Model download timeout → Use smaller model or upgrade hardware
- Port conflicts → Ensure PORT=7860

### "No LLM adapter available"

**Solution**: Ollama adapter is now always initialized. Check Ollama is running:
```bash
# In Space terminal
curl http://localhost:11434/api/tags
```

### Slow responses

**Solutions**:
- Use smaller model (phi instead of llama2)
- Upgrade to GPU hardware
- Reduce max_tokens in requests

### Model not found

**Solution**: Pull model manually:
```bash
# In Space terminal or startup script
ollama pull llama2
```

## Advanced Configuration

### Use Multiple Models

```env
# In Space settings
OLLAMA_MODEL=llama3:latest
```

Then specify model in API requests:
```json
{
  "conversation": [...],
  "model": "llama3"
}
```

### Custom System Prompts

```bash
curl -X POST $SPACE_URL/ai/chat \
  -H "Authorization: Bearer your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation": [
      {"role": "system", "content": "You are a helpful coding assistant."},
      {"role": "user", "content": "Explain Python decorators"}
    ]
  }'
```

### Enable RAG with Documents

```bash
# Upload a document
curl -X POST $SPACE_URL/upload \
  -H "Authorization: Bearer your-key" \
  -F "[email protected]"

# Query with RAG
curl -X POST $SPACE_URL/rag/query \
  -H "Authorization: Bearer your-key" \
  -H "Content-Type: application/json" \
  -d '{"query": "What does the document say about X?"}'
```

## Monitoring

### Check Space Health

```bash
curl https://YOUR_USERNAME-ai-api-ollama.hf.space/health
```

### View Metrics

```bash
curl https://YOUR_USERNAME-ai-api-ollama.hf.space/metrics \
  -H "Authorization: Bearer your-key"
```

## Scaling

### Horizontal Scaling

Hugging Face Spaces don't support horizontal scaling. For high traffic:

1. **Use multiple Spaces** with load balancer
2. **Deploy to cloud** (AWS ECS, GCP Cloud Run) with auto-scaling
3. **Use managed API** (OpenAI, Anthropic) for high volume

### Vertical Scaling

Upgrade hardware in Space settings:
- Free CPU → CPU Upgrade (2x faster)
- CPU → GPU T4 (10x faster)
- GPU T4 → GPU A10G (2x faster, larger models)

## Support

- [GitHub Issues](https://github.com/your-org/ai-api-service/issues)
- [Hugging Face Discussions](https://huggingface.co/spaces/YOUR_USERNAME/ai-api-ollama/discussions)
- [Documentation](https://github.com/your-org/ai-api-service)

## License

MIT License - see LICENSE file