Open a transcript from almost any free meeting recorder and you'll see it: a wall of text tagged "Speaker 1," "Speaker 2," "Speaker 1" again, with no idea who actually said what. Somewhere in there is the sentence "I'll own the contract redline by Friday" — and if the tool can't tell you which voice belongs to which person, that sentence is functionally useless. That gap between raw transcription and a transcript you can actually act on is what speaker diarization is supposed to close.

The short definition

Speaker diarization is the process of splitting an audio recording into segments and labeling each segment with which speaker said it — answering "who spoke when," not "what was said." It's a separate step from speech-to-text. Transcription converts audio to words; diarization figures out which words belong to which voice. A meeting tool can nail one and botch the other, which is why some transcripts read cleanly sentence-by-sentence but scramble the speaker attribution the moment two people talk over each other.

It's worth being precise about a second distinction too: diarization is not the same as speaker identification. Diarization tells you "Speaker A talked for these three stretches and Speaker B talked for these two" — it clusters voices, but by default it doesn't know their names. Speaker identification is the extra step of mapping "Speaker A" to "Priya Shah" using a voice profile, calendar invite, or meeting participant list. A transcript can have flawless diarization and still show up as "Speaker 1, Speaker 2, Speaker 3" if there's no identification layer attached. Most of the frustration people report with AI notetakers — generic labels instead of real names — is actually an identification failure sitting on top of working diarization.

How it actually works, without the math

You don't need to understand embedding vectors to evaluate whether a tool's diarization is any good, but knowing the three-step shape helps explain why it sometimes fails:

1. Segmentation — the audio is chopped into small chunks wherever the system detects a change in speaker or a shift from silence to speech.
2. Voice embedding — each chunk gets converted into a numerical fingerprint of vocal characteristics (pitch, tone, cadence) that's more stable than the actual words.
3. Clustering — chunks with similar fingerprints get grouped and labeled as the same speaker, across the whole recording.

Every failure mode traces back to one of those three steps. Crosstalk breaks segmentation, because two overlapping voices look like one chunk. Two colleagues with similar vocal ranges break clustering, because their fingerprints sit too close together. Bad audio — a laptop mic three feet from someone's face, a dial-in over spotty Wi-Fi — degrades the fingerprint itself and makes every downstream step less reliable.

Why this matters more in meetings than in podcasts

Diarization exists across a lot of use cases — podcast editing, call center QA, legal transcription — but meetings are an unusually hard case for it, for reasons specific to how meetings actually run:

• More speakers, less structure. A podcast has two or three fixed voices. A sales call, a standup, or a client review can have five to eight people, several of whom say only a sentence or two the entire call.
• Constant interruption. Meetings are full of "yeah, exactly," "sorry, go ahead," and people finishing each other's sentences — the crosstalk pattern that diarization struggles with most.
• The stakes of misattribution are higher than annoyance. In a podcast, mislabeling doesn't matter much. In a meeting, "I'll get the SOW over by end of week" attributed to the wrong person means the wrong person gets chased for it, or nobody does.

This is also where diarization quietly determines whether the rest of an AI meeting tool's output is trustworthy. Action item extraction, decision logs, and searchable summaries are all downstream of diarization — a report generator can only tell you what your champion committed to on the call if the transcript correctly separated your champion's voice from everyone else's in the first place. Bad diarization doesn't just mangle the transcript; it quietly corrupts every AI-generated artifact built on top of it.

What good diarization actually looks like in practice

A few concrete signals separate diarization that holds up in real meetings from diarization that only looks good in a demo:

• It survives crosstalk. Two people talking over each other for two seconds shouldn't blend into a garbled third "speaker."
• It holds up past 15–20 minutes. Some systems drift — a person correctly labeled for the first ten minutes starts getting mislabeled as the call goes on and voice patterns shift with fatigue, distance from the mic, or background noise.
• It scales past four or five speakers. This is where clustering quality gets tested hardest. A tool that's accurate on a 1:1 can fall apart on an eight-person cross-functional review.
• It's consistent across sessions, not just within one recording. If the same person joins three separate meetings, does the tool recognize them as the same speaker each time, or does every meeting start from a blank slate?

That last point is the difference between diarization and something more useful: persistent speaker identity. Meetbook runs transcription and diarization through AssemblyAI, then layers speaker identification on top so recurring participants — teammates, repeat clients, recurring stakeholders — get resolved to their actual names across meetings, not just within a single call. That matters less for reading one transcript and more for what you can do with a hundred of them: when speaker identity is stable across a meeting history, you can ask Meetbook's chat what a specific person has said about a topic across your last several syncs and get an answer that's actually scoped to that one person's voice, not a search across an undifferentiated wall of text. That's the practical payoff of diarization done right — it's the thing that makes semantic search over your meeting history possible at all, and it's a core part of how Meetbook's Second Brain memory stitches recurring conversations together over time.

How to evaluate it when you're choosing a tool

If you're comparing AI notetakers, don't take "speaker labeling" as a checkbox feature — test it on your actual meetings, specifically:

1. Run a call with 5+ participants and at least one deliberate interruption. Check if the transcript correctly separates the interruption.
2. Check whether generic "Speaker 1" labels get resolved to real names automatically, or require manual tagging every single meeting.
3. Re-run the same recurring meeting a week later and confirm the same person is still labeled correctly — that's the identity-persistence test most demos skip.
4. Ask whether the tool's action-item and summary generation cites who said what, or just produces an unattributed list. If it can't name the owner, the diarization behind it probably isn't feeding into anything downstream.

This is the same evaluation lens worth applying across any AI meeting notetaker you're choosing between — diarization accuracy is one of the few features that's genuinely hard to fake in a sales demo, because vendors control which clip they show you. Run it on your own messy, overlapping, five-person meeting instead. It's also a useful lens for reading through comparison pieces like Otter alternatives, Fireflies alternatives, or a direct tl;dv comparison — most of these tools will claim "speaker identification" in their marketing copy, but the real differentiator is whether that identification survives crosstalk, scales past a handful of speakers, and holds up meeting after meeting rather than resetting every time.

Diarization is one of those pieces of infrastructure nobody thinks about until it's wrong — and then it's the only thing you notice, because the whole transcript stops being trustworthy. Worth testing before you commit to a tool, not after your third meeting with someone else's commitment attributed to your name.