Automating Immersive Reading

[−]smoores · 2026-08-30 Sun 11:46 UTC · link

I took a week of from work recently to reimplement Storyteller's forced alignment algorithm. Storyteller[1] is an open source, self hosted platform for creating, managing, and reading/listening to "readaloud" books — books that have audiobook narration built in and can highlight each sentence (and/or word, with this new algorithm!) as it's read aloud. Forced alignment is the process of determining where each piece of text starts and ends in the audiobook.

Anyway, I am really pleased with how the new algorithm turned out! Hopefully someone else finds it interesting as well.

[1] https://storyteller-platform.dev

[−]orsenthil · 2026-08-30 Sun 13:37 UTC · link

This is excellent! I wanted this. I had noticed that Kindle had limited feature like this for certain books that support audiobooks. I had tried this style of reading with audiobook and ebook without a any automated sync, but hadn't pursued it. This looks cool. :)

[−]smoores · 2026-08-30 Sun 13:47 UTC · link

Thanks! We're improving it all the time — we're working on big new releases ("v3") for the web app and mobile apps. There's a Discord server linked in the docs if you ever need any help out want to chat!

[−]zackify · 2026-08-30 Sun 14:58 UTC · link

This is great. Recently got an xteink x3 and jailbroke my kindle, making custom stuff for it.

I couldn't find it exactly in the docs. Sorry if I missed it.

If you align these and then pull onto koreader via opds or other way, will audio work over Bluetooth there too? Along with the highlighting.

[−]smoores · 2026-08-30 Sun 15:03 UTC · link

You can use a KOReader plugin, https://github.com/stradichenko/audiobook.koplugin, which has work-in-progress support for Media Overlays (the EPUB spec that Storyteller uses for readaloud)!

[−]zackify · 2026-08-30 Sun 15:34 UTC · link

Wow I didn't even think about just using on device AI.

I'm going to set this one up and try it with kokoro which has the most natural for small size that I've seen. Wonder if the paperwhite 12th Gen can handle it.

And also setup the main repo for when I have the real audio book. Thanks!

[−]mobeets · 2026-08-30 Sun 13:35 UTC · link

This is really impressive! One thing I’ve often wanted is a way to listen to an ebook (say, while cooking), and then later be able to sit down and read from where I left off. This essentially requires the same functionality that Storyteller offers (syncing progress through an ebook and audiobook), though arguably overkill for this use case (since I could handle a more coarse alignment). In any case, is this something Storyteller would be good for? Or are there other tools for the job?

[−]smoores · 2026-08-30 Sun 13:39 UTC · link

That is actually the exact use case that I originally built Storyteller for! I wanted to listen to my books on long runs, and then switch to reading when I got back.

The Storyteller mobile apps have great support for this, I think. They have fully fledged audiobook players and ebook readers, and you can switch between the two with one tap. And then of course you can also double tap on a sentence and start playing from there, with the app highlighting the currently read sentence.

[−]Xerox9213 · 2026-08-30 Sun 13:54 UTC · link

What about reading on a koreader? Is bookbridge still the best bet?

[−]tefkah · 2026-08-30 Sun 14:55 UTC · link

i think smoores response got flagged for some reason, but there’s a StorytellerSync plugin https://github.com/Sirozha1337/storytellersync.koplugin

[−]tcoff91 · 2026-08-30 Sun 13:51 UTC · link

Storyteller works great for this. It even syncs your position across devices. So when I really want to settle in to read, I go ahead and switch from my iPhone where I was listening to my Boox e-ink reader which syncs right to where I left off.

[−]usern20260720 · 2026-08-30 Sun 14:22 UTC · link

I generated an immersive reading for Athenaze in a few afternoons with Claude

[−]smoores · 2026-08-30 Sun 14:27 UTC · link

... Okay!

[−]cainxinth · 2026-08-30 Sun 15:12 UTC · link

This is how I proofread. I play a screen reader while I read my work. The brain does some autocorrecting that you don’t even notice and occasionally skips over a typo when you read it without the audio.