---
title: Reachy Mini OpenAI Voice
emoji: 🎤
colorFrom: red
colorTo: blue
sdk: static
pinned: false
short_description: Talk with Reachy Mini via OpenAI Realtime and camera.
tags:
  - reachy_mini
  - reachy_mini_python_app
---

# Reachy Mini OpenAI Voice

Live speech-to-speech with [OpenAI Realtime](https://developers.openai.com/api/docs/guides/realtime-conversations) on Reachy Mini. Ask it to look around and it captures a camera frame and sends that image into the same conversation.

## What it does

- Streams Reachy's mic to OpenAI (`gpt-realtime-2.1` by default) and plays the reply on Reachy's speaker.
- Interrupts itself when you start talking.
- Calls a `camera` tool when you say things like “look at this”, “take a picture”, or “what do you see?”, then attaches the JPEG so the model can describe it.
- Optionally plays short emotion clips and speech-reactive head wobble.

## Setup

1. Install [Reachy Mini's SDK](https://github.com/pollen-robotics/reachy_mini) and start **Reachy Mini Control** with camera and microphone permission enabled.
2. In this repo:

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
```

3. Put your OpenAI key in `.env` (this file is gitignored):

```bash
cp .env.example .env
# then edit .env and set OPENAI_API_KEY
```

Or export it:

```bash
export OPENAI_API_KEY="sk-..."
```

## Run

Make sure the Reachy Mini daemon is running, then:

```bash
.venv/bin/python -m reachy_mini_openai
```

Talk to the robot. On start it takes a photo and describes it, then listens. Before each reply it attaches a fresh camera frame for context. Say **“look at this”** or **“what do you see?”** anytime. Ask for news, weather, or current facts and it will search the web, then speak the answer. Ctrl+C stops the app.

If the robot mic streams silence (common on audio firmware 2.1.2 after a USB reset), the app reboots the XMOS chip once. Firmware 2.1.4 is the lasting fix.

## Keep it running

The process only answers while it is running **and** **Reachy Mini Control** is open (the daemon on `127.0.0.1:8000`). To start at login and auto-restart after a crash:

```bash
scripts/install-always-on.sh
```

Add **Reachy Mini Control.app** as a Login Item (System Settings → General → Login Items) so the robot daemon is up when the voice app starts.

If the OpenAI connection drops, the app reconnects after 2 seconds without taking another startup photo.

Stop the LaunchAgent:

```bash
launchctl bootout gui/$(id -u) "$HOME/Library/LaunchAgents/ai.reachy.mini.openai.plist"
```

Logs: `.agent_tmp/always-on.log` and `.agent_tmp/realtime.log`.

Voice only:

```bash
.venv/bin/python -m reachy_mini_openai --no-camera
```

## Config

| Variable | Default | Meaning |
|----------|---------|---------|
| `OPENAI_API_KEY` | required | OpenAI API key |
| `OPENAI_REALTIME_MODEL` | `gpt-realtime-2.1` | Realtime model. Fallbacks: `gpt-realtime-2`, `gpt-realtime` |
| `OPENAI_REALTIME_VOICE` | `marin` | Voice (`marin`, `cedar`, `alloy`, …) |
| `REACHY_MIC_DEVICE` | auto | Optional sounddevice input index if the SDK mic is silent |
| `OPENAI_SEARCH_MODEL` | `gpt-4.1-mini` | Model used for the `web_search` tool (Responses API) |

## Local MLX agent

The older on-device VLM/STT/TTS service still lives under `agent/` (`python -m agent.server`). This OpenAI realtime app is the voice + camera path.