← Return

Pocket TTS Deno

Fast, secure, high quality TTS on the CPU via Deno ·

Port of a port

Pocket TTS Deno takes the very easy to use browser implementation pocket-tts-onnx of the efficient Kyutai Pocket TTS project. I borrowed the idea for the OpenAI-like API from another port, pocket-tts-server. In this case, I pointed Claude to those projects and guided it towards a functional port.

Accents

I wanted to explore the use of voice cloning in some of my projects. Many of the voices I've tried this with sound more realistic compared to the state of the art from before 2022. However, the model definitely has difficulty with replicating the Australian accent. Using my own voice, it seems to make it sound very American. "Very Australian" accents1 can't seem to get replicated nicely at all, they sound robotic and the American bias comes through and breaks the coherence.2 My best experiments come from using Eastern European type voices.

This model seems to be very "community-centric" with many ports, and also new models that support languages other than the core languages by the Kyutai team.3 I've also experimented with another port optimized for my phone, which on that hardware works much better.4

Jargon

Other than the accent issue it struggles with certain kinds of words, specifically jargon. However, for things like symbols and abbreviations it just detects those and converts them into full words. It would be useful to use a local model to convert the user's input (or a program's output) into exactly what the model can handle.5

I will continue to explore using this as part of a personal assistant tool.

  1. My accent isn't a classic, strong Austrlian accent due to growing up in Sydney, and living in Europe for many years. ↩
  2. Perhaps the solution is to train a model specifically on voices that sound more Australian (or even, British and Irish). ↩
  3. I'm not sure they're planning to support all languages. ↩
  4. Although the onnx / wasm models work on my phone, they're really slow, and seem to use a lot of power. This other port is instant, and also allows me to use a pre-generated voice embedding, which loads faster than having to rely on a wav file for the voice. ↩
  5. I experimented a bit with my own mappings and also trying to formulate the right context, and it worked OK, even with some phonetic approaches. However, I'm sure there's even better approaches regarding how tokenization works so I'll need to research all of this further. ↩