Nemotron 3.5 ASR 0.6B on k-or-speech-asr@0

Run 324a0e18e940 · standard · evaluated 2026-10-29

Scores

cer	0.136 (0.131–0.141)
wer	0.437 (0.426–0.447)
ser	0.721 (0.707–0.734)
parse rate	1.000
n	8000
tokens (in / out)	0 / 0
cost (USD)	5.3334

By voice

chul	0.084 (0.079–0.088) WER 0.343 (n=2000)
ethan	0.173 (0.165–0.181) WER 0.495 (n=2000)
maia	0.175 (0.167–0.182) WER 0.488 (n=2000)
sohee	0.114 (0.108–0.119) WER 0.421 (n=2000)

By code-switching style

english	0.334 (0.275–0.390) WER 0.621 (n=188)
mixed	0.197 (0.183–0.211) WER 0.555 (n=1604)
none	0.090 (0.084–0.096) WER 0.320 (n=2220)
phonetic_kr	0.122 (0.117–0.128) WER 0.436 (n=3988)

By intent

airway	0.138 (0.114–0.167) WER 0.423 (n=304)
bleeding_alert	0.125 (0.109–0.144) WER 0.383 (n=384)
chitchat	0.093 (0.074–0.114) WER 0.262 (n=360)
closure_status	0.150 (0.132–0.172) WER 0.479 (n=440)
confirmation	0.432 (0.432–0.432) WER 0.750 (n=4)
consultation_request	0.123 (0.100–0.149) WER 0.376 (n=232)
drug_administration	0.198 (0.175–0.219) WER 0.591 (n=552)
emergency	0.143 (0.124–0.164) WER 0.493 (n=304)
environment_request	0.127 (0.104–0.152) WER 0.409 (n=188)
equipment_issue	0.108 (0.093–0.125) WER 0.420 (n=272)
equipment_setting	0.096 (0.080–0.113) WER 0.336 (n=472)
handoff	0.137 (0.119–0.158) WER 0.466 (n=372)
imaging_request	0.125 (0.098–0.157) WER 0.412 (n=280)
instrument_request	0.129 (0.116–0.142) WER 0.429 (n=896)
neuromonitoring	0.189 (0.153–0.230) WER 0.476 (n=260)
patient_communication	0.092 (0.057–0.143) WER 0.292 (n=36)
specimen_handling	0.160 (0.134–0.189) WER 0.431 (n=364)
status_update	0.105 (0.093–0.117) WER 0.399 (n=964)
surgical_count	0.144 (0.123–0.165) WER 0.458 (n=276)
teaching	0.038 (0.027–0.052) WER 0.200 (n=260)
timeout_checklist	0.137 (0.137–0.137) WER 0.475 (n=4)
transfusion	0.188 (0.157–0.221) WER 0.542 (n=220)
vital_callout	0.218 (0.190–0.246) WER 0.674 (n=556)

By speaker role

anesthesia_attending	0.148 (0.134–0.164) WER 0.470 (n=800)
anesthesia_resident	0.148 (0.135–0.161) WER 0.475 (n=1244)
anesthesiologist	0.324 (0.118–0.471) WER 0.598 (n=12)
assistant	0.152 (0.133–0.171) WER 0.457 (n=500)
circulating_nurse	0.139 (0.123–0.157) WER 0.478 (n=836)
fellow	0.155 (0.136–0.174) WER 0.443 (n=860)
resident	0.154 (0.133–0.177) WER 0.477 (n=476)
scrub_nurse	0.124 (0.112–0.135) WER 0.435 (n=804)
surgeon	0.113 (0.104–0.122) WER 0.376 (n=2468)

By numbers in the reference

with digits	0.197 (0.185–0.211) WER 0.643 (n=1704)
without digits	0.118 (0.113–0.123) WER 0.372 (n=6296)

Reuse this result

Download per-item predictions (parquet) · JSON

@misc{medbench_324a0e18e940,
  title = {Nemotron 3.5 ASR 0.6B on Korean operating-room speech recognition (k-or-speech-asr@0)},
  author = {{medbench.kr}},
  year = {2026},
  howpublished = {\url{https://medbench.kr/r/324a0e18e940}},
  note = {Evaluated 2026-10-29, harness asr-p0.1, run key e31087d323e9}
}

Versions

model	`nvidia/nemotron-3.5-asr-0.6b` (NVIDIA)
task spec hash	`27b75c766635e4f4f96a4997cf25c8238f344a13fb93ec0c3332df53ca410fec`
harness	`asr-p0.1`
run key	`e31087d323e9f8fa5f435bbeb9678009c0f3239806511832752f0be73bcc2f09`
config	`{"language": "ko", "response_format": "json", "temperature": 0}`

Data

Synthetic K-OR Speech Audio, Apache-2.0 with a use restriction.

Seoul National University Hospital, Department of Anesthesiology and Pain Medicine. Synthetic K-OR Speech Audio: A Synthetic Korean Operating Room Speech Audio Dataset (version 1.0.0). KHDP. 2026. doi:10.23421/2026.0774