Mochi — Voice AI Screening Agent

Summary

Mochi is a voice agent built at Maki People. It calls a job candidate, runs a structured screening interview in one of thirteen languages, and hands the recruiter a transcript with the answers extracted. It has been in production for over a year.

Very little of the difficulty was in making it talk. The hard parts were keeping track of where it is in the interview, working out when the candidate has finished a sentence, handling interruptions, staying fast enough to feel like a conversation, failing without hanging up on anyone, and being able to tell whether a change made things better or worse.

Context and problem

A screening interview has to reach the end. The recruiter needs every question answered, in order, inside a time budget. That rules out the obvious approach — hand the whole interview to a language model and let it work through the questions — because a model that loses its place produces a transcript nobody can act on. Being mostly on track counts for nothing here.

The second constraint is that a phone call cannot be retried. When something goes wrong, a real person is holding a phone in silence, and they hang up.

My role and ownership

I wrote most of the runtime: the conversation engine, the turn-taking and interruption handling, the in-call reasoning pipeline, the inference layer, and the simulation and evaluation harness. Architectural decisions were made as a team.

This case study describes the problems and the shape of the solutions. Prompts, model configuration, tuning values and interview content belong to Maki People and are left out.

The problems, and what we did about them

Knowing where the interview is

A language model is good at saying the next thing and unreliable at knowing where it is. Asked to both conduct an interview and navigate it, it drifts — re-asking a question it just covered, skipping one, or deciding the conversation is over. The drift is also hard to debug, because the model's sense of position exists only inside the words it happened to generate.

So we split the two jobs. A state machine owns the position in the interview; the model owns the wording. Whether to move on is a separate decision, made by its own small model call that answers one question and nothing else. That answer is restricted to a short list of valid options at the point of generation, so it cannot return something the system doesn't understand.

The condition it evaluates is written as an ordinary sentence by whoever designs the interview, rather than as code. Interview design became a thing a non-engineer can do, and behaviours that would otherwise need a developer — including detecting that you have reached an answering machine — became a sentence in a configuration file.

The position itself is never stored, only derived: the system keeps a log of everything that happened and replays it to work out where it is. That reads like an implementation detail, and three useful things fall out of it. A dropped call resumes at the right question, every move the interview made is attributable to the moment that caused it, and the whole engine can be tested in milliseconds with no model involved at all, which is what makes it safe to change.

How we got here

The design above is the third attempt at the control layer in about eighteen months.

The first version did the obvious thing: one model per phase of the interview, each deciding when its part was finished and handing over to the next. It drifted — handing over is a decision, decisions drift, and with the logic spread across several sets of instructions there was no single place that knew where the interview was.

The second put a state machine in charge and had one large model call decide every move. That fixed the drift, but the call was asked to reason out loud before answering, to judge several unrelated questions at once, and to reply with a number identifying a position in a list. It worked, slowly, and a recorded call was nearly unreadable afterwards.

The third is the split described above, and it makes more model calls per turn than either of its predecessors while being faster than both.

That last transition ran in production behind a per-interview switch for four months before becoming the default. Changing the control layer of a live product is closer to a migration than a deploy, and it only works if the previous engine stays selectable call by call. Every rewrite also produced its own class of bugs, each one in code that depended on a guarantee the new design had removed. When you take a guarantee away, you have to go and find everything that was relying on it.

Being interrupted without corrupting the interview

A turn can be abandoned at any moment: the candidate interrupts, the model takes too long, something errors. If the interview had already advanced halfway through that turn, the result is a question skipped or asked twice.

Each turn therefore runs against a copy of the conversation state and is merged back only if it completes — the same idea as a database transaction. An abandoned turn throws the copy away. Interruption stops being a special case and becomes a normal outcome — which is what it should be, since in a real conversation it is one of the most common things that happens.

Knowing when the candidate has finished speaking

There is no single signal for "they stopped talking." Independent detectors disagree with each other, and which one fires first depends on the phone line. Answer too early and you talk over the candidate; answer too late and the call feels dead. Neither is recoverable — people decide in the first thirty seconds whether the thing on the other end is worth cooperating with.

We race several signals and reconcile them, and most of the work went into three things.

A fixed amount of silence is wrong for everybody. Someone answering "Yes" wants a quick reply; someone telling you about their last three jobs pauses mid-thought and wants patience. So the wait grows with how long they have already been speaking.

A pause is also not an ending. "My notice period is... three months." If the system treats the gap as the end of the answer, the recruiter reads a wrong notice period. Nearly every piece of state in that part of the code exists because of a bug of that kind, and the history contains an obvious-looking fix followed immediately by its own revert, because the obvious fix broke the pause.

The transcription provider can be asked to finish up, and it is, but there is always a deadline after which the system proceeds with what it has. A slow supplier should cost a fraction of a second, not the call.

Interrupting well

"Mm-hmm" while the agent is speaking is not an interruption. "Wait, can you repeat that?" is. Treat the two the same and the agent is either deaf or impossible to interrupt.

Interruption therefore happens in two stages. The agent's voice drops immediately to a low volume — enough that the candidate can hear themselves and knows they have been heard — and only speech that continues past a threshold actually cancels the turn. Coming back out is slower than going in, because returning to full volume as abruptly as it dropped sounds like a fault.

When a turn is cancelled, whatever the agent managed to say is recorded and explicitly marked as cut off. A half-finished sentence that pretends to be complete makes the next turn either repeat itself or skip ahead.

Staying fast enough to feel like a conversation

Silence is the failure mode, so every stage of the pipeline sits on a clock the candidate can hear.

The speaking model streams, so the candidate hears the first words while the rest is still being produced. The decision-making calls were made deliberately small: we replaced one large call that reasoned out loud with several narrow ones that don't reason at all, and the system got faster despite making more calls. The large call had been spending its time producing analysis the candidate waits for and never hears.

We run our own inference rather than renting it. That started as a latency decision, and it also let us constrain what the model is allowed to produce, which deleted a whole layer of parsing, retrying and repairing malformed answers. Cost came into it in a specific way: with per-token pricing a more thorough interview costs more, so the pricing penalises the product for being good at its job. Owning the capacity removes that pressure.

There is also a watchdog. If the agent has not started speaking after a few seconds, it says something short and human while the retry runs, then picks up in character rather than starting the sentence again. The first version said its filler phrase on every tick, which was its own kind of comedy — and the fix was a rate limit that should have been in the original change.

Failing without dropping the call

We catalogued every way the system can fail and decided, for each one, whether it is allowed to end a call. Almost none are, and a handful of rules cover most of them.

Restarting a subsystem rather than retrying inside it, because reconnecting inside a component means reasoning about all the state it was holding, and tearing it down and rebuilding it means reasoning about none.
Bounding every retry and logging the surrender, since an unbounded retry against something permanently broken is a candidate listening to nothing.
Failing open on anything that only observes, and closed on the product itself. Losing telemetry or one saved transcript is bad; dropping the call is worse. The interview definition is the exception — without it there is nothing to conduct, so that one stops.
Separating expected errors from real ones, because an error that fires on every call trains the team to ignore errors, and then the real one arrives unnoticed.
Degrading the conversation rather than dropping it. Any unhandled error becomes a spoken apology in the candidate's own language and a clean ending — so they hear a sentence rather than a dead line.

A candidate on a bad line gets a graceful explanation after a few unusable exchanges rather than fifteen minutes of "sorry, could you repeat that?" And when the time budget runs out, whatever was collected is submitted with the gaps marked, because seven answers out of ten is a usable screening and zero is not.

Fairness, as an engineering constraint

The agent must never praise, evaluate, accept or reject a candidate. This is harder than it sounds, because a language model's reflex is to be encouraging, and an agent that says "that's great" to one candidate and "okay" to another has created a fairness problem in a hiring process. The agent's instructions forbid it, recorded calls are checked for it automatically, and a simulated candidate who demands to be told whether they got the job is used to stress-test it.

Candidates who want to stop, want a human, or need the interview adapted to a disability each have an explicit path instead of being left to improvisation. Ending early requires the candidate to confirm it after being asked, so one frustrated sentence does not end an interview.

Knowing whether a change made it better

You cannot ship a voice agent by listening to it, and a change that improves one thing will break another somewhere you were not looking. Four layers of checking answer different questions.

The conversation engine is unit-tested with no model in the loop, which is only possible because the interview is data rather than an instruction buried in a prompt.

Above that is a simulator that reuses the production system unchanged, replacing only the parts that reach the outside world, and drives it with simulated candidates. There is a set of personas, each one a named way a real call goes wrong: the one who cannot hear you, the one who switches language halfway through, the one who rambles, the one who wants to leave, the one who answers every question with a question. The hardest part was stopping the simulated candidate from behaving like a chat assistant — a real candidate does not ask whether there is anything else they can help with.

Then a set of written behavioural rules, scored automatically against recorded conversations. Scoring has three values rather than two: compliant, non-compliant, and not applicable. Most evaluation harnesses have two, and rules that never applied in the first place end up poisoning the average.

The automated judges are themselves measured against human labels before being trusted — a judge is a model too. That closes the loop: real conversations get sampled into a labelled dataset, people confirm the labels, and the next change is evaluated against it offline before anybody gets a phone call.

Stack

Language: Python
Real-time: LiveKit for audio transport and telephony, with the turn-taking, interruption and audio handling written directly against the low-level primitives rather than a prepackaged agent loop
Voice: third-party speech-to-text, voice activity detection and speech synthesis
Reasoning: a self-hosted open-weights model for the in-call decisions and dialogue; a hosted model for structured extraction of the answers
Observability: conversation-level tracing plus system and call-quality metrics
Thirteen spoken languages in production

Outcome and evidence

In production at Maki People for over a year, conducting screening interviews by phone and in the browser across thirteen languages.

What is verified rather than asserted: the conversation engine is tested without a model in the loop; the full system is exercised end to end by simulated candidates before release; behavioural compliance is scored automatically by judges whose own accuracy has been measured against human labels; and whether an interview actually finished is checked with no model at all. Production latency, completion rates and quality metrics are measured internally and are not published here.