Prior (ignores the input) on kor-surgical-robot-voice-command@0

Run a923656cf11c · baseline · evaluated 2026-10-29

Scores

exact_match	0.133 (0.088–0.192)
unsafe_execution	0.000 (0.000–0.088)
stop_recall	0.000 (0.000–0.206)
over_refusal	1.000 (0.980–1.000)
exact_match_asr_generic	0.133 (0.088–0.192)
unsafe_execution_asr_generic	0.000 (0.000–0.088)
exact_match_asr_kor_or	0.133 (0.088–0.192)
unsafe_execution_asr_kor_or	0.000 (0.000–0.088)
false_activation	0.000 (0.000–0.006)
parse rate	1.000
n	2085
tokens (in / out)	0 / 0
cost (USD)	0.0000

Subset

clean	0.133 (n=495)
asr_generic	0.133 (n=495)
asr_kor_or	0.133 (n=495)
overheard	1.000 (n=600)

Type

clarify	0.000 (n=16)
explicit	0.000 (n=299)
implicit	0.000 (n=27)
indirect	0.000 (n=37)
reject	1.000 (n=66)
self_correction	0.000 (n=34)
stop	0.000 (n=16)

Form

abbr	0.148 (n=81)
abbr_ko	0.141 (n=71)
en	0.143 (n=133)
ko	0.157 (n=115)
natural	0.074 (n=95)

Reuse this result

Download per-item predictions (parquet) · JSON

@misc{medbench_a923656cf11c,
  title = {Prior (ignores the input) on Korean surgical robot voice commands (kor-surgical-robot-voice-command@0)},
  author = {{medbench.kr}},
  year = {2026},
  howpublished = {\url{https://medbench.kr/r/a923656cf11c}},
  note = {Evaluated 2026-10-29, harness p0.1, run key 0075c68ad562}
}

Versions

model	`medbench/prior` (medbench.kr)
task spec hash	`3bc7dc79688a46dc00af2790c9d049d196d1da86de20a637020129b5336e6551`
harness	`p0.1`
run key	`0075c68ad56285efdc41f69bc2af138195fd90b0c2bffbada7d4395d4985341f`
config	`{"output": {"actions": [], "message": null, "response": "reject"}, "rule": "most frequent (response, actions) of train/train_v0.2.jsonl", "subsets": ["clean", "asr_generic", "asr_kor_or", "overheard"]}`

Data

Korean Surgical Robot Voice-Command Benchmark, CC BY 4.0.

Lee H. Korean Surgical Robot Voice-Command Benchmark (version 1.0.0). Seoul National University Hospital. KHDP. 2026. doi:10.23421/2026.03507