Table of contents
Description
tts-audiobook-tool is an audiobook creation tool focused on high-quality output, supporting a growing list of text-to-speech models.
Audio generation is performed locally (via dedicated virtual environments) or through a remote server: SGL-Omni or audio.cpp.
| TTS model | Local | SGL-Omni | audio.cpp | | --- | :---: | :---: | :---: | | AuK (Base, Flash) | | ✅ | | | Breeze TTS 2 | ✅ | | ✅ | | Chatterbox (Multilingual V3, Turbo) | ✅ | ✅ | ✅ | | dots.tts | ✅ | | ✅ | | Echo-TTS | | | ✅ | | FireRedTTS3 | | | ✅ | | Fish Audio Drama 3 | | | ✅ | | Fish Speech S2-Pro | ✅ | ✅ | ✅ | | Fish Speech S1-mini | ✅ | | | | Fun-CosyVoice3 | | ✅ | ✅ | | GLM-TTS | ✅ | | ✅ | | Higgs Audio V2 | ✅ | | | | Higgs Audio V3 | | ✅ | ✅ | | IndexTTS2 (2, 2.5) | ✅ | | ✅ | | MiraTTS | ✅ | | | | MOSS-TTS v1.5 (Delay, Local) | ✅ | ✅ | ✅ | | OmniVoice | ✅ | | ✅ | | Pocket TTS | ✅ | | | | Qwen3-TTS | ✅ | ✅ | ✅ | | VibeVoice | ✅ | | | | ZONOS2 | | ✅ | |
By themselves, these models are unreliable at book scale. The app wraps them in the tooling needed for effective long-form generation:
- Error detection using speech-to-text verification, with retry logic that keeps the most accurate take
- Rational text segmentation at paragraph/sentence/phrase boundaries
- Silence trimming and reduction of excessive pauses, plus semantically-aware pause modulation at segment boundaries to improve prosody
- Per-segment loudness leveling, so voice clones and lines match in loudness (ACX-style or louder target)
- Optional post-processing: 48 kHz LavaSR v2 upsampler, per-voice stereo positioning, room reverb
It also includes a realtime audiobook playback mode and a voice chat feature, useful for voice/model testing.
The app uses a plain-text, hotkey-based console interface.
How to create an audiobook (quick summary)
- Assign a working project directory.
- Select the source EPUB or text file.
- Select a short reference audio clip for the voice clone.
- Generate. Wait.
- Concatenate the generated sound segments to create the finished audiobook file.
- Use the optional web browser player to play and read your audiobook.
Browser player
An optional browser player displays text highlighted in sync with the generated audio — similar to the Kindle Whispersync or Google Play Books read-along experience. This works because the app embeds the text and word-level timing data directly into the metadata of the FLAC and M4B files it produces.
The player is a static HTML page, which can be run locally without the use of a web server. Open browser_player/index.html directly in any browser, or use the hosted version.
Sample outputs
These examples use the same source text, 15-second voice clone sample, and default model settings.
| | | | | --- | --- | --- | | AuK | Chatterbox | | CosyVoice3 | dots.tts | Echo-TTS (audio.cpp q8) | | FireRedTTS3 (audio.cpp q8) | Fish S2-Pro | GLM-TTS | | Higgs Audio V3 (sgl-omni)| IndexTTS2 (plus emotional guidance sample) | MOSS-TTS v1.5 | | OmniVoice | Pocket TTS | Pocket TTS (LavaSRv2 48k upsample) | | Qwen3-TTS-1.7B-Base | VibeVoice 1.5B (LoRA example) | |
| | | ------ | | Multi-character dialog (four voices, LLM auto-attributed lines, Breeze TTS 2)|
Enhance existing audiobooks
This feature extends the browser player to work with pre-existing, professionally produced audiobooks (not just ones generated with this tool). It uses speech-to-text to align the existing audio with the corresponding book text, then embeds the same timing metadata the app normally produces during generation.
Select Tools > Enhance a pre-existing audiobook, and choose your source audiobook file (M4A or M4B) and the corresponding book text.
tts-server-tool
The project also includes an optional stand-alone REST service that runs the app's full configured TTS pipeline, and plays generated audio through the default sound device or serves it as an HTTP audio stream. This is aimed at developers who want to integrate the app's TTS capabilities into their own tools and workflows — for example, triggering spoken audio from a script, a home automation system, a chatbot, or any other application that can issue HTTP requests. See server readme for setup and API details.
Installation
Step 1
Check that your platform is supported:
| OS | Architecture | Notes |
| --- | --- | --- |
| Windows | x86_64 | |
| Linux | x86_64, aarch64 (glibc 2.28 or later) | Audio playback needs the system PortAudio library (e.g. sudo apt install libportaudio2) |
| macOS | Apple Silicon, macOS 14 (Sonoma) or later | |
Step 2
Clone the repository and cd into it:
git clone https://github.com/zeropointnine/tts-audiobook-tool cd tts-audiobook-tool
Step 3
Create a Python virtual environment using the instructions below.
- Local inference: Follow the section for each model you want to use.
- SGL-Omni or audio.cpp: Follow “Virtual environment for remote TTS servers.”
--only-binary=av,soundfile: PyAV and soundfile are installed only from their prebuilt wheels, which bundle FFmpeg and libsndfile, and pip fails with a clear error rather than compiling them from source on an unsupported platform. If you install or upgrade these packages by hand, pass the same option, e.g. pip install --only-binary=av,soundfile av==18.1.0 soundfile==0.13.1.
Then return here.
Step 4 (Windows only)
To enable torch CUDA acceleration on Windows, run the following commands (The project uses the same version of torch for each TTS model's virtual environments unless otherwise noted - v2.8.0/cu128).
pip uninstall -y torch torchaudio pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
Step 5
Run the app by entering:
python -m tts_audiobook_tool
Or, if you have multiple virtual environments installed, you can use launch.py to pick a venv and start the app:
python launch.py # searches the project root for venv subdirectories by default python launch.py path/to/venvs-parent-dir # searches for venv subdirectories from the given path
For how the app chooses a project's TTS model and exposes model-specific settings, refer to TTS model selection rules. Project storage and migration are documented in Project Spec v5.
Virtual environment for Breeze TTS 2 (local inference)
ℹ️ Requires CUDA
Initialize a Python v3.12 virtual environment named venv-breeze:
python -m venv venv-breeze # linux/mac C:\path\to\python3.12\python.exe -m venv venv-breeze # windows
Activate the virtual environment:
source venv-breeze/bin/activate # linux/mac venv-breeze\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-breeze.txt
Note that the above requirements file draws from a fork of the breeze-tts library, zeropointnine/breeze-tts-packaged, refactored for use as an installable package.
Virtual environment for Chatterbox (local inference)
Initialize a Python v3.11 virtual environment named "venv-chatterbox":
python -m venv venv-chatterbox # linux/mac C:\path\to\python3.11\python.exe -m venv venv-chatterbox # windows
Activate the virtual environment:
source venv-chatterbox/bin/activate # linux/mac venv-chatterbox\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-chatterbox.txt
Windows CUDA support (unlike the other models, we must use torch 2.6/cu124 here)
pip uninstall -y torch torchaudio pip install torch==2.6.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
Note: To upgrade an environment with an old version of the chatterbox-tts package, do the following:
pip uninstall chatterbox-tts pip install -r requirements-chatterbox.txt
Virtual environment for dots.tts (local inference)
ℹ️ Note:
The SOAR model variant requires 16+GB VRAM
Initialize a Python v3.12 virtual environment named venv-dots:
python -m venv venv-dots # linux/mac C:\path\to\python3.12\python.exe -m venv venv-dots # windows
Activate the virtual environment:
source venv-dots/bin/activate # linux/mac venv-dots\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-dots.txt
Note that the above requirements file draws from a fork of the dots.tts library, zeropointnine/dots.tts-for-tat, refactored for use as an installable package.
Virtual environment for Fish S1-mini (local inference)
Initialize a Python v3.12 virtual environment named "venv-fish-s1":
python -m venv venv-fish-s1 # linux/mac C:\path\to\python3.12\python.exe -m venv venv-fish-s1 # windows
Activate the virtual environment:
source venv-fish-s1/bin/activate # linux/mac venv-fish-s1\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-fish-s1.txt
Authenticate the model on HuggingFace:
- Accept the license terms on the HuggingFace Fish model page.
- Authenticate locally using your access token by running
hf auth login
Virtual environment for Fish S2-Pro (local inference)
ℹ️ Note:
Requires 24GB VRAM
Initialize a Python v3.12 virtual environment named "venv-fish-s2":
python -m venv venv-fish-s2 # linux/mac C:\path\to\python3.12\python.exe -m venv venv-fish-s2 # windows
Activate the virtual environment:
source venv-fish-s2/bin/activate # linux/mac venv-fish-s2\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-fish-s2.txt
Authenticate the model on HuggingFace:
- Accept the license terms on the HuggingFace Fish model page.
- Authenticate locally using your access token by running
hf auth login
Virtual environment for GLM-TTS (local inference)
ℹ️ Requires CUDA
Linux
Initialize a Python v3.11 virtual environment named venv-glm:
- Linux
Activate the virtual environment:
- Linux/Mac
Install dependencies:
pip install -r requirements-glm.txt
Windows
Requires setting up a "hybrid conda environment" due to a transitive dependency which is difficult to install on Windows using only pip.
Install Miniconda if not already.
Initialize the conda environment:
conda create --prefix "c:\path\to\conda-glm" python=3.11 -y
Activate the conda environment:
conda activate "c:\path\to\conda-glm"
Install this dependency using conda:
conda install pynini==2.1.6 -c conda-forge
Install the rest of the project dependencies using pip like normal:
pip install -r requirements-glm.txt
Note that we pull from a fork of glm-tts that has been refactored for use as an installable package.
Virtual environment for Higgs Audio V2 (local inference)
ℹ️ Note:
24GB VRAM recommended
On Linux and macOS, portaudio must be installed (eg, on Mac, brew install portaudio)
Initialize a Python v3.12 virtual environment named venv-higgs-v2:
python -m venv venv-higgs-v2 # linux/mac C:\path\to\python3.12\python.exe -m venv venv-higgs-v2 # windows
Activate the virtual environment:
source venv-higgs/bin/activate # linux/mac venv-higgs\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-higgs-v2.txt
Note that the above requirements file draws from a personal fork of the higgs-audio library due to the fact that the higgs repo is missing __init__.py files required for module use.
Virtual environment for IndexTTS2 (local inference)
Initialize a Python v3.11 virtual environment named venv-indextts2:
python -m venv venv-indextts2 # linux/mac C:\path\to\python3.11\python.exe -m venv venv-indextts2 # windows
Activate the virtual environment:
source venv-indextts2/bin/activate # linux/mac venv-indextts2\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-indextts2.txt
Virtual environment for MiraTTS (local inference)
ℹ️ Requires CUDA
Initialize a Python v3.12 virtual environment named venv-mira:
python -m venv venv-mira # linux/mac C:\path\to\python3.12\python.exe -m venv venv-mira # windows
Activate the virtual environment:
source venv-mira/bin/activate # linux/mac venv-mira\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-mira.txt
Virtual environment for MOSS-TTS v1.5 (local inference)
ℹ️ Note:
9B model requires 24GB+ VRAM
Initialize a Python v3.12 virtual environment named venv-moss:
python -m venv venv-moss # linux/mac C:\path\to\python3.12\python.exe -m venv venv-moss # windows
Activate the virtual environment:
source venv-moss/bin/activate # linux/mac venv-moss\Scripts\activate.bat # windows
Update pip:
python -m pip install --upgrade pip setuptools wheel
Install dependencies:
pip install -r requirements-moss.txt
For CUDA on Windows, uninstall the vanilla version of torch, and install the CUDA version of torch 2.9.1:
pip uninstall -y torch torchaudio pip install torch==2.9.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu128
Finally, install Flash Attention (optional but recommended).
Virtual environment for OmniVoice (local inference)
Initialize a Python v3.12 virtual environment named venv-omnivoice:
python -m venv venv-omnivoice # linux/mac C:\path\to\python3.12\python.exe -m venv venv-omnivoice # windows
Activate the virtual environment:
source venv-omnivoice/bin/activate # linux/mac venv-omnivoice\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-omnivoice.txt
Virtual environment for Pocket TTS (local inference)
Initialize a Python v3.12 virtual environment named "venv-pocket":
python -m venv venv-pocket # linux/mac C:\path\to\python3.12\python.exe -m venv venv-pocket # windows
Activate the virtual environment:
source venv-pocket/bin/activate # linux/mac venv-pocket\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-pocket.txt
Authenticate the model on HuggingFace:
Accept the license terms on the HuggingFace Pocket TTS model page.
Authenticate locally using your access token by running hf auth login
Virtual environment for Qwen3-TTS (local inference)
Initialize a Python v3.12 virtual environment named venv-qwen3tts:
python -m venv venv-qwen3-tts # linux/mac C:\path\to\python3.12\python.exe -m venv venv-qwen3tts # windows
Activate the virtual environment:
source venv-qwent3tts/bin/activate # linux/mac venv-qwen3tts\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-qwen3tts.txt
Install Flash Attention if using CUDA (optional but recommended).
Virtual environment for VibeVoice (local inference)
Initialize a Python v3.11 virtual environment named venv-vibevoice:
python -m venv venv-vibevoice # linux/mac C:\path\to\python3.11\python.exe -m venv venv-vibevoice # windows
Activate the virtual environment:
source venv-vibevoice/bin/activate # linux/mac venv-vibevoice\Scripts\activate.bat # windows
Install dependencies:
pip install -r requirements-vibevoice.txt
Install Flash Attention if using CUDA (optional but recommended).
Note that because Microsoft famously removed the source code from their github repository, we pull from an archived, third-party fork, vibevoice-community.
Virtual environment for remote TTS servers (SGL-Omni + audio.cpp)
Initialize a Python v3.12 virtual environment named venv-remote-tts:
python -m venv venv-remote-tts # linux/mac C:\path\to\python3.12\python.exe -m venv venv-remote-tts # windows
Activate the virtual environment:
source venv-remote-tts/bin/activate # linux/mac venv-remote-tts\Scripts\activate.bat # windows
Install the dependencies:
pip install -r requirements-remote.txt
Run the app and set the remote TTS server URL under Options (eg, http://127.0.0.1:8080).
For server installation guidance and tips on running the server alongside this app, see:
Installing Flash Attention
If instructed to do so, perform this step after successfully running pip install requirements-[model].txt without errors.
On Linux, enter:
pip install flash-attn==2.8.3 --no-build-isolation
On Windows, download and install a pre-compiled wheel from a trustworthy source (for example, mjun0812). The filename should look something like this, depending on Python version and torch version:
- flash_attn-2.7.4+cu128torch2.8-cp311-cp311-win_amd64.whl
- flash_attn-2.7.4+cu128torch2.8-cp312-cp312-win_amd64.whl
Supported features by TTS model
Zero-shot voice cloning is a first-class feature, supported for all models.
Locally generated using dedicated venv
Breeze TTS 2
- LoRA support, with strength setting (Train with instavar/breeze-tts2-finetuning).
- Voice clone, voice direction (voice sample + instructions) and voice design (instructions only)
- Batch processing support
- Audio tags
- Temperature, top_p, top_k, repetition_penalty, CFG, seed
- Multilingual V2, Multilingual V3, and Turbo model variants
- Exaggeration, CFG, temperature, top_p, top_k, repetition_penalty, seed
- Model variants:
dots.tts-soar (default; full flow-matching)
- dots.tts-mf (MeanFlow-distilled, few-step)
- dots.tts-mf-2steps (fixed 2-step)
- dots.tts-mf-1step (fixed 1-step)
- Torch compile toggle
- Streaming (for the stand-alone server and LLM chat mode)
- Speaker scale, num steps, guidance scale, seed
- Torch compile (togglable)
- Temperature, top_p, repetition penalty, seed
- Torch compile (togglable)
- Experimental rolling continuation mode
- Temperature, top_p, top_k, seed
- Emotion tags
- Temperature, top_p, top_k, seed
- Samplerate selection (24khz or 32 khz)
- Seed
- FP16 / FP8
- Emotion voice sample, emotion vector, emotion alpha
- Temperature, top_p, top_k, seed
- Batch processing support
- Temperature, top_p, top_k, repetition_penalty, seed
- MOSS-TTS v1.5 9B and MOSS-TTS-Local-Transformer v1.5 1.7B models
- Batch processing
- Experimental rolling continuation mode
- Temperature, top_p, top_k, seed
- Music detection/rejection (for MOSS-TTS-Local-Transformer)
- LoRA support (Train with OmniVoice's LoRA finetuning recipe).
- Voice design
- Steps, speed, CFG, seed
- CustomVoice and VoiceDesign model variants
- Support for Darwin-TTS-1.7B-Cross finetune
- Batch processing support
- Experimental rolling continuation mode
- Temperature, top_p, top_k, repetition_penalty, seed
- Selectable models: English (incl. 24-layer and small 18M/7M/2M variants), French, German, Spanish, Italian, Portuguese, Dutch
- Predefined voices (alba, anna, azelma, etc, plus per-language voices giovanni, lola, juergen, rafael, estelle, daan)
- Streaming support (for the stand-alone server and LLM chat mode)
- Supports CUDA acceleration (this is not mentioned in the project page)
- Temperature, seed
- LoRA support (Train with voicepowered-ai/VibeVoice-finetuning).
- Alternate VibeVoice models (eg, VibeVoice-7B or custom finetunes)
- Batch processing support
- Streaming support (for the stand-alone server and LLM chat mode)
- Music detection/rejection
- CFG, steps, seed
SGL-Omni
AuK / AuK-Flash
- Base and Flash models
- Speech speed
- Seed
- Concurrent requests
- Streaming support (for the stand-alone server and LLM chat mode)
- Temperature, top_p, top_k, repetition_penalty
- Streaming support (for the stand-alone server and LLM chat mode)
- Concurrent requests
- Experimental rolling continuation mode
- Temperature, top_p, top_k, seed
- Emotion tags
- Streaming support (for the stand-alone server and LLM chat mode)
- Concurrent requests
- Temperature, top_p, top_k
- MOSS-TTS v1.5 9B and MOSS-TTS-Local-Transformer 1.7B models
- Concurrent requests
- Music detection/rejection (for MOSS-TTS-Local-Transformer)
- Temperature, top_p, top_k, seed
- Streaming support (for the stand-alone server and LLM chat mode)
- Concurrent requests
- Temperature, top_k, repetition_penalty
audio.cpp
Breeze TTS 2
- Voice design
- Audio tags
- Temperature, top_p, top_k, guidance scale, seed
- Multilingual V2, Multilingual V3, and English model variants
- Exaggeration, CFG, temperature, top_p, top_k, repetition_penalty, seed
- Modes: zero-shot, cross-lingual, or instruct
- Top_k, steps, seed
- SOAR or MeanFlow (whichever the server entry is configured with)
- Steps, guidance scale, speaker scale, seed
- Steps, text guidance scale, speaker guidance scale, seed
- Temperature,top_p, top_k, steps, flow guidance scale, seed
- Steps, guidance scale, stop threshold, seed
- Experimental community port
- Temperature, top_p, top_k, seed
- Event tags (eg
[sighs])
- Temperature, top_p, top_k, seed
- Emotion tags
- Temperature, top_p, top_k, seed
- Temperature, top_p, top_k, duration factor (speech rate), seed
- MOSS-TTS v1.5 8B Delay and MOSS-TTS-Local v1.5
- Temperature, top_p, top_k, seed
- Music detection/rejection (MOSS-TTS-Local)
- Voice design
- Steps, speed, CFG, seed
- Base, CustomVoice and VoiceDesign model types
- Temperature, top_p, top_k; repetition penalty, seed
- See Qwen3-TTS setup notes
Usage notes
The app uses sane, conservative defaults, which should allow you to start generating your first audiobook in just a couple minutes. From there, you can explore and adjust the app's various knobs and settings to your preference.
The app saves its state between sessions, so you can interrupt the program at any time and resume later.
Setting "chapter cut points" can be useful to generate and export a long work in manageable chunks over time, allowing you to to use early chapter files before the full job is completed.
Voice cloning
Voice samples and their transcripts belong to one shared list per project. Switching TTS models keeps the list and its order. Transcripts remain editable and are retained even for models that do not use them; each model still has its own clone support and transcript requirements. Pocket's selected predefined voice takes precedence without erasing the shared samples, and IndexTTS2's secondary emotion sample remains model-specific.
When prepping reference audio for voice cloning, it's worthwhile to prepare a few different sound samples from a given source (not just one), and then test each one out in turn on a short passage of the intended text, as the quality, characteristics, and word error rate resulting from each sample from the same source can vary quite a bit. Some TTS models are especially sensitive to this.
VRAM considerations
The app ideally wants to use ~2-4 GB extra VRAM for the Whisper speech-to-text model, which needs to runs concurrently with the currently active TTS model to validate its output. If you are getting out of memory errors, try one of the following:
- Use large-v3-turbo instead of large-v3 (saves a gig or two of VRAM) (
Options>Whisper model>large-v3-turbo) - Force Whisper to use system memory instead of running on the GPU (runs much slower, ofc) (
Options>Whisper device>CPU) - Disable Whisper altogether (last resort) (
Options>Whisper model>Disabled)
Inference speeds, expectations
Listed below are some anecdotal TTS inference speeds (locally hosted models only). The app adopts each respective model's reference implementation logic as much as possible. Note how CUDA inference speeds on Linux are typically faster than on Windows, sometimes dramatically so.
| TTS Model | Setup | Speed | Notes | | ----------------------- | -------------------- | --------------- | --------------------------- | | Breeze TTS 2 | GTX 3080 Ti, Linux | ~100% realtime | batch size=1 | Breeze TTS 2 | GTX 3080 Ti, Linux | 600% realtime | batch size=10 | Chatterbox Multilingual | GTX 3080 Ti, Linux | 250% realtime | | Chatterbox Multilingual | Macbook Pro M1 (MPS) | 20-35% realtime | | Chatterbox Turbo | GTX 3080 Ti, Linux | 500%+ realtime | | Chatterbox Turbo | Macbook Pro M1 | ~70% realtime | | dots.tts (SOAR) | RTX 4090, Windows | 400% realtime | default steps (10) | dots.tts (MF) | GTX 3080 Ti, Linux | 550% realtime | default steps (4) | Fish S2-Pro | RTX 4090, Windows | 150% realtime | | Fish S1-mini | GTX 3080 Ti, Windows | 500%+ realtime | | Higgs V2 | RTX 4090, Windows | ~200% realtime | | IndexTTS2 | RTX 4090, Windows | ~150% realtime | | IndexTTS2 | GTX 3080 Ti, Windows | ~90% realtime | | IndexTTS2 | Macbook Pro M1 (MPS) | ~20% realtime | | MOSS-TTS v1.5 | GTS 4090, Windows | ~50% realtime | batch size=1, flash attn | MOSS-TTS-Local-Transformer-v1.5 | GTS 4090, Windows | 1000% realtime | batch size=20, flash attn | OmniVoice TTS | GTX 3080 Ti, Linux | 300% realtime | default steps | OmniVoice TTS | Macbook Pro M1 (MPS) | 20% realtime | default steps | Pocket TTS | GTX 3080 Ti, Linux | 1300% realtime | | Pocket TTS | Ryzen 7 7700, Linux | ~200% realtime | CPU mode | Pocket TTS | Macbook Pro M1 | 350% realtime | | Qwen3-TTS 1.6B | GTX 3080 Ti, Linux | 300% realtime | batch size=5 | Qwen3-TTS 1.6B | GTX 3080 Ti, Linux | 100% realtime | batch size=1 | GLM-TTS | GTX 3080 Ti, Linux | 200%+ realtime | | MiraTTS | GTX 3080 Ti, Linux | 3000% realtime (yes rly) | batch size=10 | MiraTTS | GTX 3080 Ti, Linux | 800% realtime | batch size=1 | VibeVoice-Large 7B | RTX 4090, Windows | 600%+ realtime | batch size=10, default steps | VibeVoice 1.5B | GTX 3080 Ti, Linux | 1000% realtime | batch size=10, default steps | VibeVoice 1.5B | GTX 3080 Ti, Linux | 200%+ realtime | batch size=1, default steps | VibeVoice 1.5B | Macbook Pro M1 | ~40% realtime |
Update highlights
2026-11-21
- Added LoRA support for Breeze TTS 2 (See instavar/breeze-tts2-finetuning).
- Updated Pocket TTS to v4.0.1, which adds new and retrained models:
english_2026-10(recommended), some very small English language variants, Dutch, and 6-layer French, German and Spanish.
- Added LoRA support for OmniVoice.
- "Enhance-an-audiobook" feature is now much more accurate, and forced alignment step is now much faster.
- FFmpeg is no longer required as a system dependency after a broad refactor.
- Rich(er) text formatting in the browser player (heading text, boldface, italice, underline). Applies to new projects post-update, or by re-importing EPUB files into existing project and re-generating audio.
- Improved loudness leveling in exports and realtime playback. Loudness is now normalized segment by segment. Removed the now-redundant, time-consuming whole-file loudness normalization pass.
- Added per-voice volume setting. Useful for when a certain voice clone tends to overpower the mix, etc.
- Added Auto-identify characters (
Voices > Auto-identify characters). Uses an LLM to determine which character speaks what dialog, which can then be assigned to your configured voices. Love this feature, personally...
- Added support for Breeze TTS 2 (local). Supports batch mode.
- Added support for Qwen3-TTS for audio.cpp (all varianets).
- The Voices menu has been revamped. Various previuosly global controls are now configurable on a per-voice basis:
Realtime playbacknow displays currently playing waveform
- Added support for Fish Drama 3 for audio.cpp
- Added Room reverb under
Create > Post-processing. Places the narration in a subtle small room (light or moderate level) or a larger, more audible space (Larger room). Off by default. - Added Room reverb stereo effect. Spreads the reverb across left and right while the voice stays centered; creates stereo files. Off by default.
- Added Room tone under
Create > Post-processing. Adds low-level background noise, prevents awkward digital (zero amplitude) silences. Off by default. - Added Minimum sentence silence inside segments under
Create > Pause timing rules. Extends overly short sentence pauses in the audio. Off by default.
- "Quick generate" (within
Generate > Review sound segments) now allows for A/B playback comparisons before saving new version.
- Added support for dots.tts for audio.cpp (SOAR or MeanFlow)
- Added support for Fish S2-Pro for audio.cpp
- Added support for GLM-TTS for audio.cpp
- Added support for IndexTTS2 / IndexTTS2.5 for audio.cpp
- Added support for CosyVoice3 for audio.cpp
- Added support for FireRedTTS3 for audio.cpp (Base)
- The
Voice clone and model settingsmenu has been split intoVoice cloneandModel settingsmenus. - Projects now use a unified voice clone list instead of a separate list for each TTS model. This makes it more convenient to switch between models mid-project.
- Added trim voice sample audio full-screen interface. This can be useful when your clip is, say, 15s, but the TTS model you're using has a lower recommended duration (eg, Echo TTS recommends 10s).
- Added support for audio.cpp as an alternative TTS backend, along with the following models:
- Added support for CosyVoice3 via SGL-Omni
- Added support for MOSS-TTS-Local-Transformer v1.5 update (local version)
2026-09-27
- Expanded menu for the "enhance-an-audiobook" feature.
- Added support for AuK TTS models (base and Flash) via SGL-Omni.
- Reworked LLM Chat feature, which is now uses a full-screen interface. Also added "echo mode" and then removed the custom text source option from "Realtime playback".
- Removed Oute TTS model. The last commit with Oute support is 121029d.
- Added support for Chatterbox Multilingual V3 which addresses the serious shortcomings of the V2 model. After updating, the app may prompt you to update the virtual environment.
- New full-screen interface for
Text > Word substitutions - Player now highlights both the full TTS prompt and the subdivided phrase currently playing within it
- Added support for dots.tts.
- New full-screen interface for
GenerateandRealtime audiobook playback - New option:
Options > Unload modelsnow frees memory reliably, since model execution is now handled through a separate worker process. - New option:
Options > Save log files - The app now requires an ANSI-capable terminal; ‘dumb’ terminals are no longer supported.
- Menus now always use the full-screen interface. The legacy “Menu does not clear screen” option has been removed.
- Batch mode is now supported when using
Voice selection mode: user-defined. - Higgs V3 and Fish S2-Pro running through SGL-Omni now use the standard voice-clone workflow. Add local voice sample files as usual, with support for multiple voice clones.
- Added optional dialog segmentation at
Text > Segmentation settings > Dialog segmentation. Detected dialog is separated from surrounding narration and preassigned to voice sample 2, simplifying multi-voice narration. Short sample
Create > Section markers > Edit: Full-screen interface for defining section markers / file split points.
- Replaced Sidon with LavaSR v2 upscaler, which is better suited to TTS and produces more tonally faithful, consistent output.
Generate > Select lines / review sound segments: Full-screen interface for selecting lines to be queued for generation and for reviewing already-generated sound segments for deletion/quick-regeneration.
- Added support for ZONOS2 using SGL-Omni.
- Added submenu
Create > Pauses between segmentswith options forNormalandShorter.
- Added voice selections editor. When using multiple voice samples, voice samples can now be assigned on a line-by-line basis.
Voice > Edit voice selections.
- Added review-and-delete sound segments editor. Useful for reviewing and selectively deleting unwanted sound generations (bad takes, etc).
Generate > Review/delete segments.
- Added text lines editor. Allows for manually deleting unwanted lines from imported text (eg, table of contents, publisher info, etc) as well as splitting lines at inner phrase boundaries. The latter can be useful when manually assigning voice samples for direct quotes/dialog.
Text > View/edit text
- Added detection for potentially cut-off words at the end of generated audio
- Added detection for pathologically long audio generations
- Added ability to add multiple voice clone samples, which get rotated after each generation.Applies to most of the supported TTS models.
- Added support for using SGL-Omni as the backend inference engine. Currently supported TTS models using SGL-Omni are:
Also added LLM chat custom system prompts for Higgs V3 and Fish S2 Pro, which elicit "tagged output" from the LLM. Fun way to test the expressiveness of models' respective custom tag features.
2026-06-03
- Added experimental rolling continuation mode for Fish S2 Pro and Qwen3-TTS Base (this is already available for MOSS-TTS)
- Added support for MOSS-TTS (MOSS-TTS v1.5 9B and MOSS-TTS-Local-Transformer 1.7B)
- Added support for Darwin-TTS (use Qwen3-TTS virtual environment and select custom hf repo id:
zeropointnine/Darwin-TTS-1.7B-Cross-Qwen3Tokenizer)
- Added support for EPUB text import
- Added ability to import project settings from app-created m4b/flac files
- Added first-class language support for Spanish. This results in more accurate dictionary-based audio word error validation (less false positives), alongside English, and makes using "Strictness level: High" more useful and more efficient.
- Added support for OmniVoice TTS (thanks @GNBS711)
- Added streaming option for LLM chat feature and stand-alone server, which minimizes "time to first audio". Currently supported for Vibevoice and Pocket TTS.
- Added side feature: Realtime LLM chat
- Added mlx-whisper for Mac Apple Silicon (MLX acceleration for Whisper)
- Added multi-venv convenience launcher (
python launch.py)
- Added Sidon 48khz upsampler post-processing option.
- Added
Concat>Limit silence gaps(prevents long silences in sound generations, keeps narration flowing)
- Added high shelf filter post-processing option
- Added strictness level "intolerant"
- Added AAC bitrate setting
- Added support for Pocket TTS, lightweight model suitable for non-GPU systems.
- Added side feature: tts-server-tool. View server readme.
- Added configurable top-p, top-k, and repetition penalty hyperparameters to models where applicable given their architecture.
- Added support for Fish S2-Pro
- Updated torch, ctranslate2, and flash-attention dependencies for all virtual environments.
- Fish S1-mini and S2-Pro - Added torch compile toggle
- Added support for Qwen3-TTS (Base, CustomVoice, and VoiceDesign models)
- Added new text segmentation method "multiple sentences" (
Text>Text segmentation strategy). Probably the ideal way to chunk text in combination withmax_words_per_segment = 80when the TTS model can handle long prompts with high accuracy and without speeding up.
- Added support for VibeVoice LoRAs (recommended trainer)
- Batching optimization improves net TTS inference speed by up to 25% (VibeVoice and Mira).
- VibeVoice batch support (drastically speeds up inference).
- VibeVoice seed support
- Music detection - Validator will now reject audio with music hallucinations (VibeVoice)
- Added submenu:
Concat>Open audiobook file in the player app
- Improved per-phrase text segmentation (ie,
Concat>Subdivide into phrases). For pre-existing projects, requires reimporting the source text.
- Prevent some TTS models from mangling the output of prompts starting with all-caps phrases (relevant for MiraTTS in particular, plus some others).
- "Section markers" can now be used for creating M4B chapters and player bookmarks (
Concatenate>Section markers>Mode)
- Note, app now saves AAC files using
.m4bsuffix instead of.m4a.
- Added option to add section markers using regular expressions (thanks @AntitrustEnthusiast).
- Added support for Chatterbox-Turbo.
- Improved logic for trimming audio generations with spurious words and noises at the beginning or end (important for Chatterbox).
- Added support for MiraTTS, including batch mode. The model is exceptionally fast when running in batch mode.
- Added option:
Generate>Transcript validation strictness
- Option to control seed value now available for: Chatterbox, Fish, GLM
- Added option:
Project>Word substitutions. This allows you to replace words in the prompt that the TTS model may mispronounce. Requires some experimentation, as you might imagine.
- Relatedly, added:
Project>Word substitutions>Inspect project text for uncommon words.
- Also added option:
Generate>Transcript validation max retries
- Simplified and much-improved speech-to-text validation logic, which is now able to calculate the number of word errors for an audio generation (rather than simply adjudging "pass" vs "fail" using a patchwork of bespoke functions).
- When a prompt is re-generated due to excessive word errors, the program is able to cherrypick the generation with the least number of errors.
- Added support for GLM-TTS.
- Added option:
Realtime audio generation>Save output.
- Added option
Concatenate>Subdivide into phrases. This allows the player app to highlight text on a "per-phrase" basis by leveraging the STT word-level timing data (Note, on pre-existing projects, all audio would first need be re-generated).
- Added option to clone project (load an existing project > make new project > "Do you want to carry over the current project's settings?" > "Yes")
- Added option to force use of CPU torch device even when CUDA or MPS is available.
- Added max text segment word length option, allowing for up to 80 words per text segment prompt. Can be useful with VibeVoice and Fish (other models are best left at the default of 40 words).
- Resolved Linux CUDA crashes related to faster-whisper library by changing Linux torch requirements. If you were experiencing this problem, get latest and update your virtual environment (
pip install -r requirements-[modelname].txt).
- Updated Chatterbox requirement to v0.1.4, which is multilanguage capable, and added language code project setting. Thanks to @JuMGameN, co-author for this update.
- For existing users, the Chatterbox venv must be recreated using Python v3.11.
- Added option to keep the Whisper model on the CPU.
- Player/reader: Added bookmarks feature, ability to pin the audio player widget in desktop mode, plus.
- Added option to disable transcription validation (
Options>Whisper model type>Disabled). Doing so is only advisable while VRAM is at a premium or when using reliably accurate and stable TTS models (eg, IndexTTS2; definitely not VoiceVoice 1.5B or Chatterbox).
- Added option to insert a "page turn" sound effect at section breaks (2+ blank lines in the text). I use this or text files converted from epub files using Calibre, which tends to insert two or more blank lines at chapter breaks, etc.
- Added support for IndexTTS2, including secondary voice reference sample guiding emotion and "emotion vectors".
- Added option to specify Whisper model (large-v3 or large-v3-turbo)
- Added support for VibeVoice alternative models (ie, large/7B model)
- Added support for VibeVoice.
- Migrated from openai-whisper to faster-whisper (faster, less memory, equivalent accuracy).
- A