Barside
Download
Barside/Content/Capture · Transcription

Capture · Transcription

Local Whisper vs cloud note takers: accuracy and speaker diarization compared

Whisper on its own does not label speakers: OpenAI's README lists six model sizes and no speaker diarization, so local apps add pyannote-audio, often through WhisperX. Cloud note takers label speakers in their own service, mostly with names from the meeting. Barside labels each line You or Others by audio channel.

Feature guideUpdated 16 min read

If you are comparing Whisper local transcription accuracy with a cloud note taker's speaker diarization, start with one fact: Whisper on its own does not say who spoke. OpenAI's Whisper README lists six model sizes, from tiny to turbo, and does not mention speaker diarization, so local apps that label speakers add a second model such as pyannote-audio, often through WhisperX. Cloud note takers label speakers inside their own service, and on the help pages we read, the names mostly come from the meeting app. We found no page that tests both kinds on the same recordings, so every accuracy figure below comes with the dataset its publisher used.

Barside, the free desktop app we make for Windows, Linux and Mac, transcribes calls with Whisper on your computer and labels each line You or Others by audio channel. It does not tell the other people apart by voice. Everything here about other projects and products comes from their own README files, help pages and pricing pages, read on November 6, 2026. We ran no hands-on tests and took no screenshots, so treat each line as the publisher's own statement and open the linked page before you decide.

Local Whisper vs cloud note takers: accuracy and speaker diarization at a glance

A Whisper local transcription vs cloud note taker comparison comes down to two separate jobs: writing the words down, and tying each line to the right speaker. One model often does the first and another model does the second, and a publisher can measure one without the other. The table sums up what each page says, as of November 2026.

ToolKindSpeaker labels, as its own page describes themAccuracy figure on that page
OpenAI WhisperOpen-source speech model you run yourselfNone mentioned in the READMEWord and character error rates by language for large-v3 and large-v2, on Common Voice 15 and Fleurs
pyannote.audioOpen-source speaker diarization toolkitSpeaker diarization is its whole jobDiarization error rate on 12 datasets, last updated in 2025-09
WhisperXOpen-source tool that pairs Whisper with pyannote-audioSpeaker ID labels from pyannote-audioWe found none; it says "Diarization is far from perfect"
MacWhisperMac appAutomatic Speaker RecognitionWe found none on its site
VibeApp for macOS, Windows and LinuxSpeaker diarizationWe found none in its README
BuzzApp for Windows, Mac and LinuxSpeaker identification in transcribed mediaWe found none in its README
GranolaNote taker with desktop and phone appsParticipants' display names on Google Meet, Zoom and Microsoft Teams; Me and Them otherwiseWe found none on its transcription page
Otter.aiNote-taking serviceSpeaker identification, listed on the free Basic planWe found none on its pricing page
Fireflies.aiNote-taking serviceSpeaker labels when its bot joins; Speaker 1 and Speaker 2 in system audio mode and for uploaded filesWe found none on the pages we read
BarsideDesktop app, Whisper on your computerYou (microphone) and Others (computer sound)Its guides give none; it uses Whisper's compact base models

The pattern is worth keeping in mind. A low word error rate tells you the words are right, and it says nothing about whether each line went to the right person. Check both before you trust a transcript of a meeting with several voices.

Does Whisper tell speakers apart on its own?

On speaker identification, local Whisper transcription and cloud note takers start from different places: Whisper has none built in, while every note taker we read lists some kind of speaker label. OpenAI's Whisper README calls Whisper "a general-purpose speech recognition model" and a multitasking model for multilingual speech recognition, speech translation and language identification. It says nothing about speaker diarization, speaker identification or speaker labels.

The README does publish the trade-offs between its six model sizes:

ModelParametersRequired VRAMRelative speed
tiny39 M~1 GB~10x
base74 M~1 GB~7x
small244 M~2 GB~4x
medium769 M~5 GB~2x
large1550 M~10 GB1x
turbo809 M~6 GB~8x

OpenAI says it measured those speeds by transcribing English speech on an A100, and that real-world speed may vary significantly, including by language. On accuracy, the README states: "Whisper's performance varies widely depending on the language." It charts word and character error rates for large-v3 and large-v2 on the Common Voice 15 and Fleurs datasets, and points to the appendices of its paper for the other models. We found no figure in the README for meetings or for crosstalk, so the honest answer to "how accurate is local Whisper" is that it depends on the model size, the language and the recording.

Every call, captured · Open the video

How local apps add speaker diarization to Whisper

Speaker diarization is the step that splits a recording into stretches of speech and groups them by voice. Local setups usually get it from pyannote.audio, which describes itself as "an open-source toolkit written in Python for speaker diarization" under the MIT license. Its community-1 pipeline runs locally once you accept the user conditions for pyannote/speaker-diarization-community-1 on Hugging Face and create an access token, and it needs ffmpeg to read audio. A graphics card is optional: the README's example moves the pipeline to CUDA when one is available. Its premium precision-2 pipeline runs on pyannoteAI's servers with an API key, and the README also mentions a self-hosted version.

WhisperX joins the two halves. Its README lists batched inference at "70x realtime transcription using whisper large-v2", word-level timestamps from wav2vec2 alignment, and "Multispeaker ASR using speaker diarization from pyannote-audio (speaker ID labels)", using the community-1 model. It is released under the BSD-2-Clause license and gives a command for running on a CPU, which it also names for Mac: --compute_type int8 --device cpu.

A do-it-yourself setup with WhisperX takes five steps:

  1. Install the basics. You need Python, and the README says you may also need ffmpeg.
  2. Unlock the diarization model. Accept the user agreement for speaker-diarization-community-1 on Hugging Face and create a token with read access.
  3. Run WhisperX with diarization. Pass the token with --hf_token, and if you know how many people spoke, set --min_speakers and --max_speakers, as the README suggests.
  4. Put names on the labels. Diarization gives speaker IDs, so you match each one to a person by listening to a few lines.
  5. Check the hard spots. WhisperX lists its own limits: "Overlapping speech is not handled particularly well by whisper nor whisperx", "Diarization is far from perfect", and words such as "£13.60" that the alignment model cannot handle get no timing.

How accurate is Whisper speaker diarization in 2025?

For Whisper speaker diarization accuracy in 2025, the clearest published numbers come from pyannote.audio, the toolkit WhisperX uses, whose benchmark was last updated in 2025-09. It reports the diarization error rate in percent, where lower is better, for three pipelines on 12 datasets. Six of the rows:

Datasetlegacy 3.1community-1precision-2
AMI (IHM)18.817.012.9
AMI (SDM)22.719.915.6
AliMeeting (channel 1)24.520.315.2
CALLHOME (part 2)28.526.716.6
DIHARD 3 (full)21.420.214.7
VoxConverse (v0.3)11.211.28.5

Source: pyannote.audio benchmark, last updated in 2025-09. We found no note on that README about the scoring setup beyond the datasets and the date.

Three things follow from the publisher's own numbers. WhisperX uses community-1, so that column is the closest published figure for a WhisperX setup, although WhisperX itself prints no diarization score. The kind of recording matters as much as the pipeline: in the full table, precision-2 ranges from 7.4 on REPERE (phase2) to 39.0 on Ego4D (dev.). And these numbers rate only who spoke when, so the words still carry Whisper's own error rate for your language.

Which speaker diarization is most accurate on long audio?

If you want the most accurate speaker diarization for AI transcription of long audio, the lowest published figures we found are for pyannoteAI's precision-2, which beats pyannote's two other pipelines in every row of pyannote's own benchmark. On AMI (IHM), whose files run about an hour, it scores 12.9 against 17.0 for community-1. The README also gives speed on a self-hosted NVIDIA H100 80GB HBM3: on those hour-long files, community-1 takes 31 seconds per hour of audio and precision-2 takes 14 seconds. Your own computer will differ.

Keep two limits in mind. These are the publisher's own measurements of its own pipelines, and we found no diarization error rate at all on the Granola, Otter and Fireflies pages we read, so there is no published basis for ranking those services against pyannote. For long recordings of your own, set the number of speakers when you know it, give each person a microphone close to them when you can, and check overlapping speech against the audio.

What cloud note takers say about speaker labels

The cloud note takers' help pages show a pattern: speaker names mostly come from the meeting itself, and a plain mix of audio gets generic labels.

  • Granola. Its transcription page says "speaker tags can label who said what using each participant's display name", on Google Meet, Zoom and Microsoft Teams on macOS and Windows. Otherwise you see Me and Them, for your microphone and your system audio, and the page adds that "our AI usually does a good job of inferring speakers from contextual clues in the transcript". The phone apps can recognize different speakers in face-to-face meetings. Granola passes audio to its transcription provider and says it does not record or save audio or video.
  • Otter.ai. Otter's pricing page lists "Speaker identification" on the free Basic plan and "Team vocabulary & taggable speakers" on Pro. As of November 2026, Basic is free with 300 transcription minutes per user a month, shared across transcription types, and at most 30 minutes per conversation. We found no accuracy figure on that page.
  • Fireflies.ai. Its guide to getting started with the desktop app says speaker labels are available when its notetaker bot joins, and that in system audio mode "Speaker labels are not available (you'll see Speaker 1 / Speaker 2)". Its recording FAQ says uploaded files get generic labels such as Speaker 1 and Speaker 2, which you can relabel, and that speaker name detection works only when Fireflies joins a video meeting or you use the Chrome extension.

So before you pay for a cloud note taker to get names, check how you will record. Granola's Me and Them, like Barside's You and Others, splits lines by audio source, and the named labels depend on a supported meeting app or a bot in the call.

Which local transcription apps with speaker diarization run on Mac and Windows?

The local transcription apps with speaker diarization for Mac and Windows that we found documented are MacWhisper on a Mac, Vibe and Buzz on Windows, Mac and Linux, and WhisperX for people at home in a terminal. The table sums up their own pages, as of November 2026.

AppSystems on its pageSpeaker feature on its pageWhat it transcribesPrice or license
MacWhisperMacSpeaker recognition, Automatic Speaker RecognitionDropped files (mp3, wav, m4a, ogg, opus, mov, mp4), meetings, audio from any appFree; Pro €64 per license, with lifetime updates (as of November 2026)
VibemacOS, Windows, LinuxSpeaker diarizationFiles in batches, microphone, system audioFree, MIT license
BuzzWindows, Mac with Apple silicon, LinuxSpeaker identification in transcribed mediaAudio and video files, YouTube links, live microphoneFree, MIT license
WhisperXRuns on a CPU, Mac included, by its READMESpeaker ID labels from pyannote-audioAudio files, from the command lineFree, BSD-2-Clause license
BarsideWindows, Linux and MacYou and Others, by audio channelLive calls only: microphone and computer soundFree

MacWhisper's site says it transcribes "audio, video, meetings, and system audio locally on your Mac" with Whisper, Parakeet and other models, and that it records meetings in Zoom, Teams, Webex, Skype, Chime, Discord and more. It also lets you switch to models hosted remotely, and the page we read does not say which plan includes speaker recognition. Vibe's README promises "fully offline transcription, no data ever leaves your device" and also offers summaries through the Claude API, which is a cloud step. Buzz's README notes that its Windows installer is unsigned, so Windows shows a warning, and that the last version for Intel Macs is 1.4.5.

What Barside does: You and Others, by audio channel

Barside records a call as two-channel audio: your microphone on one side and the sound your computer plays on the other. Every 30 seconds, Whisper turns the new piece into text on your computer, Barside compares how loud each side was, and it adds a line with the time from the start of the call and a label. On a group call, everyone except you is Others, and in a room with one microphone, every line is You. Headphones give the cleanest labels, because without them a quiet reply right at a change of speaker can get the other label.

  1. Install speech once. Open the assistant with Ctrl+Shift+A, select Assistant settings, and choose Install speech. It downloads Whisper and its English model, about 150 MB, and needs no AI key.
  2. Add languages if you need them. Under Speech language, choose Install more languages · 141 MB for 34 more languages, pick a language or Detect automatically, then choose Save assistant settings.
  3. Put on headphones, so your microphone hears only you.
  4. Open the call setup. Choose Transcribe a call in the assistant, click the microphone on the desktop bar, or press Alt+Shift+Space and type call.
  5. Tell the people on the call, then press Start recording. A banner reads Recording call · mic + computer.
  6. Press Stop & save when the call ends, and the Call transcript note is done.

Each piece with sound is also kept in the note as a Call audio clip, so you can check a line against the recording. Barside captures the computer's sound through the system's audio loopback on Windows, through PulseAudio or PipeWire on Linux, and on a Mac with macOS 14.2 or later. The full guide is transcribe calls on your computer.

What Barside does not do matters just as much here. It does not tell the other people apart by voice. It transcribes calls live and cannot transcribe a long audio file you already have. It uses Whisper's compact base models (base.en for English), which can mishear names, numbers and crosstalk. When you hand out tasks, say the person's name out loud, because Meeting follow-through gives a task an owner only when that name appears in the transcript. For your own calls of up to 6 people, a Barside video meeting with captions on writes each person's words on their own computer, so the meeting note shows who said what, although a guest who joins from a browser needs the app for captions of their own words.

Examples: which setup fits which job

These are made-up examples that show how the trade-offs play out.

  • Example: Priya, a podcast producer, has hour-long interviews with three voices already recorded as files. Barside does not transcribe files, so she looks at MacWhisper on her Mac, or Vibe or Buzz on Windows, or WhisperX with the speaker count set to three.
  • Example: Daniel, a consultant on Windows, takes one-on-one client calls in Zoom. With headphones on, Barside's You and Others already separate him from the client, and he says the client's name when they agree on a task.
  • Example: Ana, a team lead, runs a weekly sync with five people. She holds it as a Barside video meeting with captions on, so the note shows who said what, while a colleague who meets in Google Meet uses a note taker that reads display names.
  • Example: Lukas, a researcher, needs speaker turns across many long recordings. He runs pyannote's community-1 locally on a sample, compares it with precision-2 on pyannoteAI's servers if his data may leave the computer, and checks a few minutes of each by ear.

Privacy: where the audio goes

Where speaker labels are made decides where your audio travels. Whisper, WhisperX and pyannote's community-1 run on your own computer, while pyannote's README marks precision-2 as running on pyannoteAI servers. Granola passes audio to its transcription provider, and Fireflies' bot joins the meeting as a participant, so read each vendor's own privacy pages before you record a confidential call.

In Barside, the call audio and the transcript stay in the note on your computer. Apart from Barside Plus cloud backup and sync when you turn them on, the transcript leaves only when you give it to an AI: Meeting follow-through sends the text to the AI company you connected in Settings → AI, whose data policy applies. Barside does count which features are used, sending barside.space the names of the features used that day with the system and the Barside version, never notes, files or who you are, and a switch in Settings turns it off. Read how recordings and transcripts work for the details, and always tell people before you record.

How it compares

If you need named speakers in recordings you already have, a local diarization setup or an app that lists it, such as MacWhisper, Vibe or Buzz, fits better than Barside. If you need names on Zoom, Google Meet or Teams calls and accept a cloud service, the note takers above describe how they get them. Barside fits when you want live calls transcribed on your computer into your notes, with your own lines marked You and everyone else marked Others.

Works well with

Once a transcript is in your notes, these guides show what to do with it next.

FAQ

Questions and answers

Can Whisper identify different speakers?

Not on its own. OpenAI's Whisper README describes speech recognition, speech translation and language identification, lists six model sizes, and does not mention speaker diarization. To label speakers locally, add a diarization model such as pyannote-audio, for example through WhisperX, or use an app whose page lists the feature, such as MacWhisper, Vibe or Buzz.

How accurate is WhisperX speaker diarization?

WhisperX prints no diarization score of its own, and its README says plainly that "Diarization is far from perfect" and that overlapping speech is not handled particularly well. It uses pyannote's community-1 pipeline, which pyannote's benchmark, last updated in 2025-09, puts at a diarization error rate of 17.0% on AMI (IHM) and 20.2% on DIHARD 3 (full), where lower is better.

What is the most accurate speaker diarization for long recordings?

Among the published figures we found, pyannoteAI's precision-2 has the lowest diarization error rate of pyannote's three pipelines in every row of pyannote's own benchmark, for example 12.9% on AMI (IHM), whose files run about an hour. It runs on pyannoteAI's servers with an API key, and the README mentions a self-hosted version. We found no diarization figures on the Granola, Otter and Fireflies pages we read.

Do cloud note takers recognize speakers by voice?

Their help pages say the names mostly come from the meeting. Granola labels speakers with each participant's display name on Google Meet, Zoom and Microsoft Teams and shows Me and Them otherwise. Fireflies says speaker name detection works only when it joins a video meeting or you use its Chrome extension, and that uploaded files get labels such as Speaker 1 and Speaker 2.

Which local transcription apps label speakers on Mac and Windows?

Going by their own pages in November 2026, MacWhisper (Mac) lists Automatic Speaker Recognition, Vibe (macOS, Windows and Linux) lists speaker diarization, and Buzz (Windows, Mac with Apple silicon and Linux) lists speaker identification in transcribed media. WhisperX adds diarization from the command line. Barside labels lines You and Others by audio channel on Windows, Linux and Mac.

Can Barside tell who is speaking?

Partly. Barside labels each line You (your microphone) or Others (the sound your computer plays), with its time from the start of the call. It does not tell the other people apart by voice, so everyone except you is Others, and in a room with one microphone every line is You. Headphones give the cleanest labels. In Barside's own video meetings, the meeting note shows who said what.

Can Barside transcribe an audio file I already have?

No. Barside transcribes a call live, as it records it, and does not transcribe an audio file you already have. MacWhisper, Vibe and Buzz say on their own pages that they transcribe files. For videos in your notes, Barside's Clips editor writes down every word on your computer.

Is Barside free?

Yes. Barside is a free desktop app for Windows, Linux and Mac, and call transcription needs no subscription and no AI key. Summaries use the AI account you connect, which bills you for what it uses, or your own Claude or ChatGPT plan through My plan. Barside Plus is optional, at US$20 a month or US$200 a year plus sales tax, for cloud backup, sync and cloud agents.

Sources

Where the facts come from

Checked on November 12, 2026. Products change: if something here is out of date, tell us at support@barside.space.

  1. OpenAI Whisper on GitHub: README
  2. pyannote.audio on GitHub: README and benchmark
  3. WhisperX on GitHub: README
  4. Granola docs: how transcription works
  5. Otter.ai pricing
  6. Fireflies Knowledge Base: Getting Started with the Fireflies Desktop App
  7. Fireflies Knowledge Base: How Fireflies joins and records your meetings - FAQs
  8. MacWhisper
  9. Vibe on GitHub: README
  10. Buzz on GitHub: README