Skip to content

TTS improvements.#4145

Merged
filipi87 merged 9 commits intomainfrom
filipi/tts_improvements_remove_reset
Mar 27, 2026
Merged

TTS improvements.#4145
filipi87 merged 9 commits intomainfrom
filipi/tts_improvements_remove_reset

Conversation

@filipi87
Copy link
Copy Markdown
Contributor

@filipi87 filipi87 commented Mar 25, 2026

Summary

  • Removed "Reset" and "TTSStoppedFrame" string sentinels from add_word_timestamps
  • Services that previously called add_word_timestamps([("TTSStoppedFrame", 0), ("Reset", 0)], ctx_id) now call append_to_audio_context(ctx_id, TTSStoppedFrame(...)) directly, and reset logic is handled uniformly at the end of _handle_audio_context
  • append_to_audio_context signature broadened to accept Frame | _WordTimestampEntry | None, and remove_audio_context now routes through it for consistency
  • All on_audio_context_interrupted and on_audio_context_completed overrides across TTS services now call super(), ensuring base class lifecycle hooks are consistently invoked
  • Fixed a race condition in InterruptibleTTSService where, if run_tts had been invoked but BotStartedSpeakingFrame had not yet been received, a user interruption could allow stale audio to leak through.
  • Fixed websocket TTS word timestamps so interrupted contexts cannot leak stale words or backward PTS values into later turns.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 25, 2026

Codecov Report

❌ Patch coverage is 21.27660% with 37 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/pipecat/services/tts_service.py 37.03% 17 Missing ⚠️
src/pipecat/services/cartesia/tts.py 0.00% 3 Missing ⚠️
src/pipecat/services/gradium/tts.py 0.00% 3 Missing ⚠️
src/pipecat/services/resembleai/tts.py 0.00% 3 Missing ⚠️
src/pipecat/services/asyncai/tts.py 0.00% 2 Missing ⚠️
src/pipecat/services/elevenlabs/tts.py 0.00% 2 Missing ⚠️
src/pipecat/services/inworld/tts.py 0.00% 2 Missing ⚠️
src/pipecat/services/rime/tts.py 0.00% 2 Missing ⚠️
src/pipecat/services/deepgram/sagemaker/tts.py 0.00% 1 Missing ⚠️
src/pipecat/services/deepgram/tts.py 0.00% 1 Missing ⚠️
... and 1 more
Files with missing lines Coverage Δ
src/pipecat/services/azure/tts.py 0.00% <ø> (ø)
src/pipecat/services/hume/tts.py 11.94% <ø> (+0.17%) ⬆️
src/pipecat/services/deepgram/sagemaker/tts.py 30.95% <0.00%> (-0.25%) ⬇️
src/pipecat/services/deepgram/tts.py 29.94% <0.00%> (-0.16%) ⬇️
src/pipecat/services/fish/tts.py 7.69% <0.00%> (-0.04%) ⬇️
src/pipecat/services/asyncai/tts.py 33.46% <0.00%> (-0.26%) ⬇️
src/pipecat/services/elevenlabs/tts.py 28.30% <0.00%> (+0.07%) ⬆️
src/pipecat/services/inworld/tts.py 25.36% <0.00%> (+0.15%) ⬆️
src/pipecat/services/rime/tts.py 32.92% <0.00%> (-0.01%) ⬇️
src/pipecat/services/cartesia/tts.py 47.07% <0.00%> (-0.14%) ⬇️
... and 3 more

... and 8 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@filipi87 filipi87 marked this pull request as ready for review March 26, 2026 13:41
@filipi87 filipi87 force-pushed the filipi/tts_improvements_remove_reset branch from 8c16cfa to c4466ba Compare March 26, 2026 14:38
@markbackman
Copy link
Copy Markdown
Contributor

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Copy link
Copy Markdown
Contributor

@markbackman markbackman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change looks good.

I tested the services modified and all sound good.

I noticed issues with word timestamp / audio alignment for Inworld and Rime. I think we can follow up on those separately as they may be service related.

Nice cleanup! 👏

@filipi87
Copy link
Copy Markdown
Contributor Author

Hi @markbackman, yes, I had noticed that for Inworld as well. I just pushed a PR with some fixes:

I will take a look on what is happening for Rime.

Thank you for testing this. 🙌

@filipi87 filipi87 merged commit 95c55ec into main Mar 27, 2026
6 checks passed
@filipi87 filipi87 deleted the filipi/tts_improvements_remove_reset branch March 27, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants