Building an interruptible voice agent in Expo

Your user gets impatient halfway through your app's answer and just starts talking. Now the software has maybe a hundred milliseconds to drop every queued audio buffer, keep the microphone open, and make sure the phone doesn't hear its own voice coming out of the speaker. On a phone. In JavaScript, mostly.

That last part is the trap. Echo cancellation on iOS and Android lives in the operating system's voice-processing audio path, not in anything you can reach from JS. So does the low-latency playback queue that has to start the moment the first bytes arrive. A voice conversation app that only has the microphone, or only has the speaker, is easy. Having both live at once, on one audio session, is the part that eats weekends.

We built a chat app, and we hit this. The existing Expo answer was @speechmatics/expo-two-way-audio, a well-made module that does the duplex plumbing: echo-cancelled microphone capture as raw PCM, low-latency PCM playback, and interruption. But it stopped receiving changes in January 2025, and our production app was finding the rough edges the hard way. So in October we forked it. The fork is @equationalapplications/expo-two-way-audio, MIT licensed like the original, with Speechmatics credited as the author, and it's on npm now at 0.2.1.

What the module actually does

The design rule is that JavaScript stays thin. The JS layer is types, event helpers, and React hooks. Everything that cares about audio timing lives in Swift and Kotlin, and Expo autolinks it into your dev build (this is a native module: it works in development builds and EAS builds, not Expo Go).

The whole loop fits in one screen:

import {
  bargeIn,
  initialize,
  playPCMData,
  tearDown,
  toggleRecording,
  useExpoTwoWayAudioEventListener,
} from '@equationalapplications/expo-two-way-audio'

// Once, on mount:
initialize()
// later, on unmount:
tearDown()

// Mic, as 16 kHz 16-bit mono PCM, chunk by chunk:
useExpoTwoWayAudioEventListener('onMicrophoneData', (event) => {
  // send event.data to your speech service
})
toggleRecording(true) // unmute

// Playback: hand it bytes as they arrive, it queues and starts on the first one:
playPCMData(pcmChunk)

// The user talked over the assistant. Drop the queue, keep the mic live:
bargeIn()

One format, documented, both directions: 16 kHz, 16-bit, mono PCM. The microphone emits it and the playback queue eats it, so wiring the module to a streaming speech service is a pipe, not a translation layer.

bargeIn() deserves its own sentence. It flushes everything queued for playback immediately and does not touch the microphone. That is the whole interruptible-agent trick: you never mute the mic during playback, so the model can be cut off mid-word and hear what came next.

What the fork changed

The upstream project was good enough that the fork's value is mostly in the bugs we hit in production. In 0.2.0:

• The iOS crash where a media-services reset (iOS restarting its audio subsystem under a running app, say for a phone call) re-ran the audio tap setup and crashed on a duplicate tap. setup() now removes any existing tap before installing one, and the reset path refreshes the audio session.
• Android playback got thread-safe and teardown became repeatable. An interrupted reply or a fast settings change used to be able to wedge the playback thread.
• A lost-wakeup race on Android: a chunk queued at the exact moment playback ended could sit unplayed until the next chunk arrived. If your client chunks slowly, that read as seconds of dead air.
• Everything builds on current Expo SDK 55+ tooling again.

0.2.1 was a republish through a GitHub Actions workflow with npm trusted publishing, so every release from now on carries provenance naming the exact commit and workflow that built it. Releases are a version-bump PR, a merge, and a signed tag; nothing is published from a laptop.

What we have proven, and what we have not

We put the npm package through a device smoke run before calling any of this done. Two builds went out to real phones: one built from the vendored module copy, and one, the interesting one, built from the published npm package itself, so what passed is exactly what you'd install. On a physical iPhone and a physical Android phone, driving a live voice conversation through Gemini: microphone in, replies out, barge-in cutting off the assistant mid-reply with the mic staying live, and on Android specifically no echo of the phone's own voice coming back through the microphone, stop and restart of the call, background and return, and a Siri interruption on iOS. Logs stayed clean, and we checked the binary was the npm-sourced build before trusting any of it.

That is the plumbing working. What we have not formally measured is the two numbers a serious voice app eventually wants: the smallest chunk size that plays cleanly on each platform without choppiness, and a dedicated echo cancellation probe on Android 13, where an upstream user reported incomplete cancellation and crackling at playback start. The fork shipped code that should address both, and the smoke run heard nothing wrong, but "heard nothing wrong during normal conversation" is not the same as a measurement. The sweep is next; the recommended chunk sizes will land in the README when it's done, and if the echo cancellation still misbehaves on Android 13 under a deliberate probe, we'll file it as a bug instead of a feature.

If you try the module, that's the report we'd want from you too: which phone, which Android or iOS version, and what your ears heard. The npm package is @equationalapplications/expo-two-way-audio, the repo has a migration guide at the top of the README, and switching from the upstream package is a one line swap.