Recorded subtitles · Accuracy
How Baian’s caption tool compares with professional translators and with other AI.
Checking a scholarly lecture01 · Against translators
Even with speech recognition as an extra step, Baian scored above the professional translations in Hausa, Tamil, Swahili and Urdu. The judge found no clear difference in 21 other languages.
Baian’s subtitles were preferred:
The line marks 50%: an even split.
02 · Against an edited translation
On the WMT26 test, Baian was preferred over the professionally edited translation on 91 of 133 clips.
English to Egyptian Arabic
The line marks 50%: an even split.
03 · Against other AI
Out of 15 or 16 engines in each language.
Recorded speech: transcription rankingsErrors in words, or characters for Korean and Japanese. Lower is better.
SpanishRecorded
RussianRecorded
HindiRecorded
FrenchRecorded
JapaneseRecorded
For scholarly lectures
Baian uses nearby lines as context when translating a passage. For a scholarly lecture, the final check is whether the English preserves the speaker’s meaning, quotations and distinctions.
Speech-recognition scores measure the transcript. Translation scores assess the move into another language. The FLoRes comparison above starts with English speech; its Urdu result does not measure Urdu-to-English translation. The WMT result is English to Egyptian Arabic and uses an automatic judge.
Read the test methodsHave someone who understands both languages check names, specialist terms, negation, who is speaking or being quoted, and any Qur’an or hadith references. Check the subtitles against the recording, not just whether the English reads smoothly.
How Baian uses contextFor Arabic recordings, Baian can retrieve Qur’an text and a stored translation when it recognises a quotation. That is a separate step from ordinary translation. Short or unclear quotations can be missed, and the tests above do not establish a zero-error rate for scholarly lectures.
Upload a recording, choose your languages and download the subtitles.
Upload a recordingCaption Tool 9.5% of words wrong, Baian’s live interpreter 15.6%, ElevenLabs 21.3%, OpenAI 43.7%, Azure 50.6%, Google 62.4%.
300 recordings of Saudi speakers switching between Arabic and English. Measures words wrong and meaning kept.
Perle’s authors tested ElevenLabs, OpenAI, Microsoft Azure and Google. We ran both Baian products on the same recordings.
Scored with Perle’s own rules and meaning model. The Caption Tool ran in transcription-only mode; Live used default settings with no language hint.
1st in Spanish, Russian and Hindi; joint 1st in French and Japanese.
200 FLEURS sentences per language. Measures words wrong.
Verli tested 15 or 16 engines per language. We ran Baian’s caption tool on the same test set.
Verli tested each engine as it was in July 2026 on its own 200-sentence FLEURS sample. We used the same test set and scoring rule on a different 200 sentences. Verli’s scoring rule splits Hindi words at vowel signs, which roughly halves everyone’s error rate, so we publish Hindi as a rank only.
Caption Tool scored above the translators in Hausa, Swahili, Tamil and Urdu (preferred on 67% to 75% of sentences). Baian’s live interpreter had about the same overall quality in Arabic, Hindi and Korean.
148 English sentences that also have professional human translations. Measures translation quality with Google’s MetricX-24 automatic judge.
Baian ran the test. The human translations come from the public FLoRes-200 set.
Baian worked from the spoken audio; the translators worked from text. We compared scores sentence by sentence and counted a result only when the 95% range did not cross zero and two runs agreed. “About the same” means the test found no clear difference; it does not prove the translations were identical.
2nd of 30 systems, and preferred over the professional post-edited translation on 91 of 133 clips (68%).
133 English YouTube clips, the speech test from the 2026 Conference on Machine Translation.
Baian scored every published system’s output, and its own, with Google’s MetricX-24 automatic judge.
Baian received the same automatic transcript as the other systems. WMT’s own human raters rank systems differently, so we always describe this as an automatic score.
13.2% of words wrong, the lowest in the comparison. OpenAI GPT-4o Transcribe got 19.7%.
FLEURS Urdu recordings. Measures words wrong.
The Qwen3.5-Omni authors published scores for several systems; we ran the Caption Tool on the same test set.
Scored with the standard Whisper rule: lower case, punctuation removed.