Capture · Transcription
Local Whisper vs cloud note takers: accuracy and speaker diarization compared
Whisper on its own does not label speakers: OpenAI's README lists six model sizes and no speaker diarization, so local apps add pyannote-audio, often through WhisperX. Cloud note takers label speakers in their own service, mostly with names from the meeting. Barside labels each line You or Others by audio channel.
If you are comparing Whisper local transcription accuracy with a cloud note taker's speaker diarization, start with one fact: Whisper on its own does not say who spoke. OpenAI's Whisper README lists six model sizes, from tiny to turbo, and does not mention speaker diarization, so local apps that label speakers add a second model such as pyannote-audio, often through WhisperX. Cloud note takers label speakers inside their own service, and on the help pages we read, the names mostly come from the meeting app. We found no page that tests both kinds on the same recordings, so every accuracy figure below comes with the dataset its publisher used.
Barside, the free desktop app we make for Windows, Linux and Mac, transcribes calls with Whisper on your computer and labels each line You or Others by audio channel. It does not tell the other people apart by voice. Everything here about other projects and products comes from their own README files, help pages and pricing pages, read on November 6, 2026. We ran no hands-on tests and took no screenshots, so treat each line as the publisher's own statement and open the linked page before you decide.
Local Whisper vs cloud note takers: accuracy and speaker diarization at a glance
A Whisper local transcription vs cloud note taker comparison comes down to two separate jobs: writing the words down, and tying each line to the right speaker. One model often does the first and another model does the second, and a publisher can measure one without the other. The table sums up what each page says, as of November 2026.
| Tool | Kind | Speaker labels, as its own page describes them | Accuracy figure on that page |
|---|---|---|---|
| OpenAI Whisper | Open-source speech model you run yourself | None mentioned in the README | Word and character error rates by language for large-v3 and large-v2, on Common Voice 15 and Fleurs |
| pyannote.audio | Open-source speaker diarization toolkit | Speaker diarization is its whole job | Diarization error rate on 12 datasets, last updated in 2025-09 |
| WhisperX | Open-source tool that pairs Whisper with pyannote-audio | Speaker ID labels from pyannote-audio | We found none; it says "Diarization is far from perfect" |
| MacWhisper | Mac app | Automatic Speaker Recognition | We found none on its site |
| Vibe | App for macOS, Windows and Linux | Speaker diarization | We found none in its README |
| Buzz | App for Windows, Mac and Linux | Speaker identification in transcribed media | We found none in its README |
| Granola | Note taker with desktop and phone apps | Participants' display names on Google Meet, Zoom and Microsoft Teams; Me and Them otherwise | We found none on its transcription page |
| Otter.ai | Note-taking service | Speaker identification, listed on the free Basic plan | We found none on its pricing page |
| Fireflies.ai | Note-taking service | Speaker labels when its bot joins; Speaker 1 and Speaker 2 in system audio mode and for uploaded files | We found none on the pages we read |
| Barside | Desktop app, Whisper on your computer | You (microphone) and Others (computer sound) | Its guides give none; it uses Whisper's compact base models |
The pattern is worth keeping in mind. A low word error rate tells you the words are right, and it says nothing about whether each line went to the right person. Check both before you trust a transcript of a meeting with several voices.
Does Whisper tell speakers apart on its own?
On speaker identification, local Whisper transcription and cloud note takers start from different places: Whisper has none built in, while every note taker we read lists some kind of speaker label. OpenAI's Whisper README calls Whisper "a general-purpose speech recognition model" and a multitasking model for multilingual speech recognition, speech translation and language identification. It says nothing about speaker diarization, speaker identification or speaker labels.
The README does publish the trade-offs between its six model sizes:
| Model | Parameters | Required VRAM | Relative speed |
|---|---|---|---|
| tiny | 39 M | ~1 GB | ~10x |
| base | 74 M | ~1 GB | ~7x |
| small | 244 M | ~2 GB | ~4x |
| medium | 769 M | ~5 GB | ~2x |
| large | 1550 M | ~10 GB | 1x |
| turbo | 809 M | ~6 GB | ~8x |
OpenAI says it measured those speeds by transcribing English speech on an A100, and that real-world speed may vary significantly, including by language. On accuracy, the README states: "Whisper's performance varies widely depending on the language." It charts word and character error rates for large-v3 and large-v2 on the Common Voice 15 and Fleurs datasets, and points to the appendices of its paper for the other models. We found no figure in the README for meetings or for crosstalk, so the honest answer to "how accurate is local Whisper" is that it depends on the model size, the language and the recording.
How local apps add speaker diarization to Whisper
Speaker diarization is the step that splits a recording into stretches of speech and groups them by voice. Local setups usually get it from pyannote.audio, which describes itself as "an open-source toolkit written in Python for speaker diarization" under the MIT license. Its community-1 pipeline runs locally once you accept the user conditions for pyannote/speaker-diarization-community-1 on Hugging Face and create an access token, and it needs ffmpeg to read audio. A graphics card is optional: the README's example moves the pipeline to CUDA when one is available. Its premium precision-2 pipeline runs on pyannoteAI's servers with an API key, and the README also mentions a self-hosted version.
WhisperX joins the two halves. Its README lists batched inference at "70x realtime transcription using whisper large-v2", word-level timestamps from wav2vec2 alignment, and "Multispeaker ASR using speaker diarization from pyannote-audio (speaker ID labels)", using the community-1 model. It is released under the BSD-2-Clause license and gives a command for running on a CPU, which it also names for Mac: --compute_type int8 --device cpu.
A do-it-yourself setup with WhisperX takes five steps:
- Install the basics. You need Python, and the README says you may also need
ffmpeg. - Unlock the diarization model. Accept the user agreement for
speaker-diarization-community-1on Hugging Face and create a token with read access. - Run WhisperX with diarization. Pass the token with
--hf_token, and if you know how many people spoke, set--min_speakersand--max_speakers, as the README suggests. - Put names on the labels. Diarization gives speaker IDs, so you match each one to a person by listening to a few lines.
- Check the hard spots. WhisperX lists its own limits: "Overlapping speech is not handled particularly well by whisper nor whisperx", "Diarization is far from perfect", and words such as "£13.60" that the alignment model cannot handle get no timing.
How accurate is Whisper speaker diarization in 2025?
For Whisper speaker diarization accuracy in 2025, the clearest published numbers come from pyannote.audio, the toolkit WhisperX uses, whose benchmark was last updated in 2025-09. It reports the diarization error rate in percent, where lower is better, for three pipelines on 12 datasets. Six of the rows:
| Dataset | legacy 3.1 | community-1 | precision-2 |
|---|---|---|---|
| AMI (IHM) | 18.8 | 17.0 | 12.9 |
| AMI (SDM) | 22.7 | 19.9 | 15.6 |
| AliMeeting (channel 1) | 24.5 | 20.3 | 15.2 |
| CALLHOME (part 2) | 28.5 | 26.7 | 16.6 |
| DIHARD 3 (full) | 21.4 | 20.2 | 14.7 |
| VoxConverse (v0.3) | 11.2 | 11.2 | 8.5 |
Source: pyannote.audio benchmark, last updated in 2025-09. We found no note on that README about the scoring setup beyond the datasets and the date.
Three things follow from the publisher's own numbers. WhisperX uses community-1, so that column is the closest published figure for a WhisperX setup, although WhisperX itself prints no diarization score. The kind of recording matters as much as the pipeline: in the full table, precision-2 ranges from 7.4 on REPERE (phase2) to 39.0 on Ego4D (dev.). And these numbers rate only who spoke when, so the words still carry Whisper's own error rate for your language.
Which speaker diarization is most accurate on long audio?
If you want the most accurate speaker diarization for AI transcription of long audio, the lowest published figures we found are for pyannoteAI's precision-2, which beats pyannote's two other pipelines in every row of pyannote's own benchmark. On AMI (IHM), whose files run about an hour, it scores 12.9 against 17.0 for community-1. The README also gives speed on a self-hosted NVIDIA H100 80GB HBM3: on those hour-long files, community-1 takes 31 seconds per hour of audio and precision-2 takes 14 seconds. Your own computer will differ.
Keep two limits in mind. These are the publisher's own measurements of its own pipelines, and we found no diarization error rate at all on the Granola, Otter and Fireflies pages we read, so there is no published basis for ranking those services against pyannote. For long recordings of your own, set the number of speakers when you know it, give each person a microphone close to them when you can, and check overlapping speech against the audio.
What cloud note takers say about speaker labels
The cloud note takers' help pages show a pattern: speaker names mostly come from the meeting itself, and a plain mix of audio gets generic labels.
- Granola. Its transcription page says "speaker tags can label who said what using each participant's display name", on Google Meet, Zoom and Microsoft Teams on macOS and Windows. Otherwise you see Me and Them, for your microphone and your system audio, and the page adds that "our AI usually does a good job of inferring speakers from contextual clues in the transcript". The phone apps can recognize different speakers in face-to-face meetings. Granola passes audio to its transcription provider and says it does not record or save audio or video.
- Otter.ai. Otter's pricing page lists "Speaker identification" on the free Basic plan and "Team vocabulary & taggable speakers" on Pro. As of November 2026, Basic is free with 300 transcription minutes per user a month, shared across transcription types, and at most 30 minutes per conversation. We found no accuracy figure on that page.
- Fireflies.ai. Its guide to getting started with the desktop app says speaker labels are available when its notetaker bot joins, and that in system audio mode "Speaker labels are not available (you'll see Speaker 1 / Speaker 2)". Its recording FAQ says uploaded files get generic labels such as Speaker 1 and Speaker 2, which you can relabel, and that speaker name detection works only when Fireflies joins a video meeting or you use the Chrome extension.
So before you pay for a cloud note taker to get names, check how you will record. Granola's Me and Them, like Barside's You and Others, splits lines by audio source, and the named labels depend on a supported meeting app or a bot in the call.
Which local transcription apps with speaker diarization run on Mac and Windows?
The local transcription apps with speaker diarization for Mac and Windows that we found documented are MacWhisper on a Mac, Vibe and Buzz on Windows, Mac and Linux, and WhisperX for people at home in a terminal. The table sums up their own pages, as of November 2026.
| App | Systems on its page | Speaker feature on its page | What it transcribes | Price or license |
|---|---|---|---|---|
| MacWhisper | Mac | Speaker recognition, Automatic Speaker Recognition | Dropped files (mp3, wav, m4a, ogg, opus, mov, mp4), meetings, audio from any app | Free; Pro €64 per license, with lifetime updates (as of November 2026) |
| Vibe | macOS, Windows, Linux | Speaker diarization | Files in batches, microphone, system audio | Free, MIT license |
| Buzz | Windows, Mac with Apple silicon, Linux | Speaker identification in transcribed media | Audio and video files, YouTube links, live microphone | Free, MIT license |
| WhisperX | Runs on a CPU, Mac included, by its README | Speaker ID labels from pyannote-audio | Audio files, from the command line | Free, BSD-2-Clause license |
| Barside | Windows, Linux and Mac | You and Others, by audio channel | Live calls only: microphone and computer sound | Free |
MacWhisper's site says it transcribes "audio, video, meetings, and system audio locally on your Mac" with Whisper, Parakeet and other models, and that it records meetings in Zoom, Teams, Webex, Skype, Chime, Discord and more. It also lets you switch to models hosted remotely, and the page we read does not say which plan includes speaker recognition. Vibe's README promises "fully offline transcription, no data ever leaves your device" and also offers summaries through the Claude API, which is a cloud step. Buzz's README notes that its Windows installer is unsigned, so Windows shows a warning, and that the last version for Intel Macs is 1.4.5.
What Barside does: You and Others, by audio channel
Barside records a call as two-channel audio: your microphone on one side and the sound your computer plays on the other. Every 30 seconds, Whisper turns the new piece into text on your computer, Barside compares how loud each side was, and it adds a line with the time from the start of the call and a label. On a group call, everyone except you is Others, and in a room with one microphone, every line is You. Headphones give the cleanest labels, because without them a quiet reply right at a change of speaker can get the other label.
- Install speech once. Open the assistant with
Ctrl+Shift+A, select Assistant settings, and choose Install speech. It downloads Whisper and its English model, about 150 MB, and needs no AI key. - Add languages if you need them. Under Speech language, choose Install more languages · 141 MB for 34 more languages, pick a language or Detect automatically, then choose Save assistant settings.
- Put on headphones, so your microphone hears only you.
- Open the call setup. Choose Transcribe a call in the assistant, click the microphone on the desktop bar, or press
Alt+Shift+Spaceand typecall. - Tell the people on the call, then press Start recording. A banner reads Recording call · mic + computer.
- Press Stop & save when the call ends, and the Call transcript note is done.
Each piece with sound is also kept in the note as a Call audio clip, so you can check a line against the recording. Barside captures the computer's sound through the system's audio loopback on Windows, through PulseAudio or PipeWire on Linux, and on a Mac with macOS 14.2 or later. The full guide is transcribe calls on your computer.
What Barside does not do matters just as much here. It does not tell the other people apart by voice. It transcribes calls live and cannot transcribe a long audio file you already have. It uses Whisper's compact base models (base.en for English), which can mishear names, numbers and crosstalk. When you hand out tasks, say the person's name out loud, because Meeting follow-through gives a task an owner only when that name appears in the transcript. For your own calls of up to 6 people, a Barside video meeting with captions on writes each person's words on their own computer, so the meeting note shows who said what, although a guest who joins from a browser needs the app for captions of their own words.
Examples: which setup fits which job
These are made-up examples that show how the trade-offs play out.
- Example: Priya, a podcast producer, has hour-long interviews with three voices already recorded as files. Barside does not transcribe files, so she looks at MacWhisper on her Mac, or Vibe or Buzz on Windows, or WhisperX with the speaker count set to three.
- Example: Daniel, a consultant on Windows, takes one-on-one client calls in Zoom. With headphones on, Barside's You and Others already separate him from the client, and he says the client's name when they agree on a task.
- Example: Ana, a team lead, runs a weekly sync with five people. She holds it as a Barside video meeting with captions on, so the note shows who said what, while a colleague who meets in Google Meet uses a note taker that reads display names.
- Example: Lukas, a researcher, needs speaker turns across many long recordings. He runs pyannote's community-1 locally on a sample, compares it with precision-2 on pyannoteAI's servers if his data may leave the computer, and checks a few minutes of each by ear.
Privacy: where the audio goes
Where speaker labels are made decides where your audio travels. Whisper, WhisperX and pyannote's community-1 run on your own computer, while pyannote's README marks precision-2 as running on pyannoteAI servers. Granola passes audio to its transcription provider, and Fireflies' bot joins the meeting as a participant, so read each vendor's own privacy pages before you record a confidential call.
In Barside, the call audio and the transcript stay in the note on your computer. Apart from Barside Plus cloud backup and sync when you turn them on, the transcript leaves only when you give it to an AI: Meeting follow-through sends the text to the AI company you connected in Settings → AI, whose data policy applies. Barside does count which features are used, sending barside.space the names of the features used that day with the system and the Barside version, never notes, files or who you are, and a switch in Settings turns it off. Read how recordings and transcripts work for the details, and always tell people before you record.
How it compares
If you need named speakers in recordings you already have, a local diarization setup or an app that lists it, such as MacWhisper, Vibe or Buzz, fits better than Barside. If you need names on Zoom, Google Meet or Teams calls and accept a cloud service, the note takers above describe how they get them. Barside fits when you want live calls transcribed on your computer into your notes, with your own lines marked You and everyone else marked Others.
- For recorded files and cloud services side by side, read the best transcription apps.
- For a setup where the summary stays local too, see fully offline AI meeting transcription with local Whisper.
- For a feature-by-feature comparison with one note taker at a time, read the Otter.ai alternative and Granola alternative guides.
- For tools that record without a bot in the call, see AI note takers without a bot.
Works well with
Once a transcript is in your notes, these guides show what to do with it next.
FAQ
Questions and answers
Can Whisper identify different speakers?
Not on its own. OpenAI's Whisper README describes speech recognition, speech translation and language identification, lists six model sizes, and does not mention speaker diarization. To label speakers locally, add a diarization model such as pyannote-audio, for example through WhisperX, or use an app whose page lists the feature, such as MacWhisper, Vibe or Buzz.
How accurate is WhisperX speaker diarization?
WhisperX prints no diarization score of its own, and its README says plainly that "Diarization is far from perfect" and that overlapping speech is not handled particularly well. It uses pyannote's community-1 pipeline, which pyannote's benchmark, last updated in 2025-09, puts at a diarization error rate of 17.0% on AMI (IHM) and 20.2% on DIHARD 3 (full), where lower is better.
What is the most accurate speaker diarization for long recordings?
Among the published figures we found, pyannoteAI's precision-2 has the lowest diarization error rate of pyannote's three pipelines in every row of pyannote's own benchmark, for example 12.9% on AMI (IHM), whose files run about an hour. It runs on pyannoteAI's servers with an API key, and the README mentions a self-hosted version. We found no diarization figures on the Granola, Otter and Fireflies pages we read.
Do cloud note takers recognize speakers by voice?
Their help pages say the names mostly come from the meeting. Granola labels speakers with each participant's display name on Google Meet, Zoom and Microsoft Teams and shows Me and Them otherwise. Fireflies says speaker name detection works only when it joins a video meeting or you use its Chrome extension, and that uploaded files get labels such as Speaker 1 and Speaker 2.
Which local transcription apps label speakers on Mac and Windows?
Going by their own pages in November 2026, MacWhisper (Mac) lists Automatic Speaker Recognition, Vibe (macOS, Windows and Linux) lists speaker diarization, and Buzz (Windows, Mac with Apple silicon and Linux) lists speaker identification in transcribed media. WhisperX adds diarization from the command line. Barside labels lines You and Others by audio channel on Windows, Linux and Mac.
Can Barside tell who is speaking?
Partly. Barside labels each line You (your microphone) or Others (the sound your computer plays), with its time from the start of the call. It does not tell the other people apart by voice, so everyone except you is Others, and in a room with one microphone every line is You. Headphones give the cleanest labels. In Barside's own video meetings, the meeting note shows who said what.
Can Barside transcribe an audio file I already have?
No. Barside transcribes a call live, as it records it, and does not transcribe an audio file you already have. MacWhisper, Vibe and Buzz say on their own pages that they transcribe files. For videos in your notes, Barside's Clips editor writes down every word on your computer.
Is Barside free?
Yes. Barside is a free desktop app for Windows, Linux and Mac, and call transcription needs no subscription and no AI key. Summaries use the AI account you connect, which bills you for what it uses, or your own Claude or ChatGPT plan through My plan. Barside Plus is optional, at US$20 a month or US$200 a year plus sales tax, for cloud backup, sync and cloud agents.
Sources
Where the facts come from
Checked on November 12, 2026. Products change: if something here is out of date, tell us at support@barside.space.
- OpenAI Whisper on GitHub: README
- pyannote.audio on GitHub: README and benchmark
- WhisperX on GitHub: README
- Granola docs: how transcription works
- Otter.ai pricing
- Fireflies Knowledge Base: Getting Started with the Fireflies Desktop App
- Fireflies Knowledge Base: How Fireflies joins and records your meetings - FAQs
- MacWhisper
- Vibe on GitHub: README
- Buzz on GitHub: README