Voice Mode

Typing is fine when you're at a keyboard. It's less fine when you're driving, cooking, or walking the dog. Voice mode turns Hermes into a two-way audio agent: you send voice messages through the gateway platforms, Hermes transcribes them with speech-to-text, processes the request through the same agent core, and responds with text-to-speech audio. You never touch the keyboard.

This lesson covers the STT and TTS providers, the /voice command family, and the specific configuration that makes voice work across gateway platforms.

What you'll learn

• STT (speech-to-text) auto-detects the best available provider, local faster-whisper is free and works offline, Groq Whisper has a generous free tier, OpenAI and Mistral are paid options
• TTS (text-to-speech) defaults to Edge TTS which is free and requires no API key, ElevenLabs, OpenAI, MiniMax, Mistral, and NeuTTS are available for higher quality or specific voices
• /voice on enables full voice-to-voice mode; /voice tts gives you spoken responses while you type; /voice off returns to text-only
• Voice mode works on every gateway platform that supports audio messages, Telegram voice notes and Discord voice channels are the two most common paths

The problem

You've built a gateway that lets you reach Hermes from anywhere. The next friction point is input. When you're on your phone and need to fire off a quick task , "check if the backup completed and tell me the size" , typing 50 characters on a phone keyboard is annoying. Voice mode eliminates the typing. You press the mic button, speak your request, and hear the answer spoken back. The entire interaction happens without a keyboard.

Options & when to use each

STT providers (voice → text)

Provider	What it uses	Cost	When to pick it
Local faster-whisper	Your machine's CPU (or GPU, if available)	Free, no API key	Offline use, privacy-sensitive environments, the default if you have Python and pip available
Groq Whisper	Groq's cloud Whisper endpoint	Free tier, generous limits	Fast transcription without local compute, usually quicker than local whisper on CPU
OpenAI Whisper	OpenAI's hosted Whisper API	Paid, requires API key	High accuracy, multiple language support, already set up if you use OpenAI for your model
Mistral Voxtral	Mistral's speech API	Paid, requires API key	Multi-language European speech, good for non-English use cases

Hermes auto-detects available providers in the order above. Install local faster-whisper and it uses that. Set GROQ_API_KEY and it prefers Groq. Both can coexist, Hermes picks the best one it has credentials and dependencies for.

To use local faster-whisper:

pip install faster-whisper

That's the only dependency. The base model is small enough to run on CPU. For better accuracy, you can configure a larger model:

hermes config set stt.local.model "small"

Options: tiny, base, small, medium, large-v3. Larger models are more accurate but slower. small is the sweet spot for most use, accurate enough for natural language requests without noticeable latency on modern hardware.

TTS providers (text → voice)

Provider	What it uses	Cost	When to pick it
Edge TTS	Microsoft's free text-to-speech service	Free, no API key, no setup	Default, works out of the box. Natural-sounding voices, decent quality for everyday use
ElevenLabs	ElevenLabs voice synthesis	Free tier, then paid	Best voice quality, voice cloning, emotional range. If audio quality matters, this is the one
OpenAI TTS	OpenAI's text-to-speech API	Paid, requires API key	Good quality, already set up if you use OpenAI for your model
MiniMax	MiniMax speech synthesis	Paid, requires API key	Strong Chinese and multi-language support
Mistral Voxtral	Mistral's speech API	Paid, requires API key	European language support, pairs with Voxtral for STT
NeuTTS	Local neural TTS	Free, needs pip install neutts[all] and espeak-ng	Fully offline, no API calls, good for air-gapped environments

Edge TTS is the default and the right choice for most people getting started. It requires no setup. If you want higher quality later, switch to ElevenLabs by setting ELEVENLABS_API_KEY and updating the config:

hermes config set tts.provider elevenlabs

Voice modes

Command	What it does	When to use it
/voice on	Full voice-to-voice, Hermes listens to your voice messages and responds with spoken audio	Hands-free use, driving, walking, cooking
/voice tts	TTS only, you type, Hermes responds with spoken audio	When you want to hear responses but can't or don't want to speak (reading code output, long responses while you work on something else)
/voice off	Text-only, no audio in either direction	The default. Disables voice mode

Voice mode is per-session. Enable it when you want audio, disable it when you're done. It doesn't persist across sessions unless you configure it as the default in config.yaml:

hermes config set voice.enabled true

Build it

Step 1: Install STT

Install the local transcription dependency. The faster-whisper package provides offline speech-to-text capabilities directly on your machine.

pip install faster-whisper

Confirm the installation was successful by importing the module in Python.

python3 -c "from faster_whisper import WhisperModel; print('STT ready')"

If you prefer Groq (no local compute needed):

# Set your Groq API key in ~/.hermes/.env
echo "GROQ_API_KEY=gsk_your_key_here" >> ~/.hermes/.env
hermes config set stt.provider groq

Step 2: Verify TTS is available

Edge TTS is the default and requires no setup. Verify it works:

hermes config | grep -A3 "tts:"

You should see provider: edge in the output. That's all the setup Edge TTS needs.

If you want ElevenLabs instead:

echo "ELEVENLABS_API_KEY=your_key_here" >> ~/.hermes/.env
hermes config set tts.provider elevenlabs

Step 3: Restart the gateway

Voice provider changes require a restart:

hermes gateway restart

Check the logs for voice initialization:

grep -i "stt\|tts\|voice" ~/.hermes/logs/gateway.log | tail -5

Step 4: Enable voice mode from a gateway platform

Send this message to your bot on Telegram or Discord:

/voice on

Hermes confirms voice mode is active. Now send a voice message, on Telegram, hold the mic button and speak; on Discord, join a voice channel and speak (requires the bot to have Connect and Speak permissions).

Hermes transcribes the audio with STT, processes the request, and responds with a voice message using TTS. The entire interaction is voice-in, voice-out.

Step 5: Test TTS-only mode

If you want spoken responses but prefer to type your requests:

/voice tts

Send a text message. Hermes responds with a voice message instead of text. This is useful for long responses you want to listen to while doing something else.

To return to text-only:

/voice off

What goes wrong

Mistake	How you notice it	The fix
Voice messages are ignored	You send a voice note on Telegram, Hermes doesn't respond or responds with text	STT isn't installed or configured. Check that faster-whisper is installed (`pip list
STT transcription is gibberish	Hermes responds to something completely unrelated to what you said	The whisper model is too small for your audio quality or accent. Upgrade the model: hermes config set stt.local.model "small" or "medium". Larger models handle accents and background noise better
TTS responses have no audio	Hermes responds with text only even though voice mode is on	TTS provider isn't configured. Edge TTS should work by default, but if it doesn't, check your internet connection, Edge TTS makes API calls even though it's free. For offline TTS, install NeuTTS
Voice mode not available on Discord	/voice on works but Discord voice messages don't get transcribed	Discord voice channel support requires the bot to have Connect and Speak permissions, plus STT configured. Make sure you granted these permissions when generating the invite URL. Re-invite the bot with the correct permissions if needed
Local whisper is too slow	Voice messages take 5+ seconds to transcribe before Hermes even starts processing	CPU transcription of large models is slow. Either switch to Groq (hermes config set stt.provider groq + GROQ_API_KEY), use a smaller model ("base" or "tiny"), or accept the latency for the privacy benefit of local processing

Confirm it worked

Run these checks to verify that both speech-to-text and text-to-speech are functioning correctly. Test the voice capabilities directly from your connected gateway platform.

# 1. Verify voice components are initialized
grep -i "voice\|stt\|tts" ~/.hermes/logs/gateway.log | tail -5

# 2. Enable voice mode from a gateway platform
# Send: /voice on

# 3. Send a voice message on Telegram
# Hold the mic button, say:
# "What time is it and what's the current system uptime?"

# 4. Hermes should respond with a spoken voice message
#    that includes the actual time and uptime from your system

# 5. Test TTS-only mode
# Send: /voice tts
# Then send: "Summarize the last 5 lines of the gateway log"
# Hermes should respond with audio

# 6. Turn voice off
# Send: /voice off
# Hermes should confirm and return to text-only responses

If step 3 produces a spoken response with actual system data (not a canned answer), voice mode is working end to end. You can now talk to your agent from anywhere, on any platform, without a keyboard.

Next: Day 4 , Delegation, Cron & the Background Agent