README.md

metadata

title: Reachy Mini OpenAI Voice
emoji: 🎤
colorFrom: red
colorTo: blue
sdk: static
pinned: false
short_description: Talk with Reachy Mini via OpenAI Realtime and camera.
tags:
  - reachy_mini
  - reachy_mini_python_app

Reachy Mini OpenAI Voice

Live speech-to-speech with OpenAI Realtime on Reachy Mini. Ask it to look around and it captures a camera frame and sends that image into the same conversation.

What it does

• Streams Reachy's mic to OpenAI (gpt-realtime-2.1 by default) and plays the reply on Reachy's speaker.
• Interrupts itself when you start talking.
• Calls a camera tool when you say things like “look at this”, “take a picture”, or “what do you see?”, then attaches the JPEG so the model can describe it.
• Optionally plays short emotion clips and speech-reactive head wobble.

Setup

1. Install Reachy Mini's SDK and start Reachy Mini Control with camera and microphone permission enabled.
2. In this repo:

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

3. Put your OpenAI key in .env (this file is gitignored):

cp .env.example .env
# then edit .env and set OPENAI_API_KEY

Or export it:

export OPENAI_API_KEY="sk-..."

Run

Make sure the Reachy Mini daemon is running, then:

.venv/bin/python -m reachy_mini_openai

Talk to the robot. On start it takes a photo and describes it, then listens. Before each reply it attaches a fresh camera frame for context. Say “look at this” or “what do you see?” anytime. Ask for news, weather, or current facts and it will search the web, then speak the answer. Ctrl+C stops the app.

If the robot mic streams silence (common on audio firmware 2.1.2 after a USB reset), the app reboots the XMOS chip once. Firmware 2.1.4 is the lasting fix.

Keep it running

The process only answers while it is running and Reachy Mini Control is open (the daemon on 127.0.0.1:8000). To start at login and auto-restart after a crash:

scripts/install-always-on.sh

Add Reachy Mini Control.app as a Login Item (System Settings → General → Login Items) so the robot daemon is up when the voice app starts.

If the OpenAI connection drops, the app reconnects after 2 seconds without taking another startup photo.

Stop the LaunchAgent:

launchctl bootout gui/$(id -u) "$HOME/Library/LaunchAgents/ai.reachy.mini.openai.plist"

Logs: .agent_tmp/always-on.log and .agent_tmp/realtime.log.

Voice only:

.venv/bin/python -m reachy_mini_openai --no-camera

Config

Variable	Default	Meaning
OPENAI_API_KEY	required	OpenAI API key
OPENAI_REALTIME_MODEL	gpt-realtime-2.1	Realtime model. Fallbacks: gpt-realtime-2, gpt-realtime
OPENAI_REALTIME_VOICE	marin	Voice (marin, cedar, alloy, …)
REACHY_MIC_DEVICE	auto	Optional sounddevice input index if the SDK mic is silent
OPENAI_SEARCH_MODEL	gpt-4.1-mini	Model used for the web_search tool (Responses API)

Local MLX agent

The older on-device VLM/STT/TTS service still lives under agent/ (python -m agent.server). This OpenAI realtime app is the voice + camera path.