DeepSeek V4.1 Flash (2026-10-17) on clinicalqa@0
Run 2224a6c5f8d6 · baseline tier · evaluated 2026-10-29
Scores
| accuracy | 0.904 (0.887–0.922) |
|---|---|
| parse rate | 0.998 |
| n | 1045 |
| tokens (in / out) | 432343 / 680371 |
| cost (USD) | 0.4731 |
By category
| Allergy and Immunology | 0.917 (n=36) |
|---|---|
| Cardiology | 0.955 (n=89) |
| Dermatology | 0.833 (n=12) |
| Emergency Medicine | 1.000 (n=32) |
| Endocrinology | 0.912 (n=68) |
| Endocrinology, metabolic, and nutritional disorders | 1.000 (n=1) |
| Forensic Medicine | 1.000 (n=1) |
| Gastroenterology | 0.929 (n=112) |
| General Surgery | 0.897 (n=58) |
| Geriatric Medicine | 1.000 (n=1) |
| Gynecology | 0.829 (n=70) |
| Hematology and Oncology | 0.979 (n=48) |
| Infectious Diseases | 0.808 (n=52) |
| Musculoskeletal (e.g., costochondritis) | 1.000 (n=1) |
| Nephrology | 0.874 (n=111) |
| Neurology / Neurosurgery | 0.959 (n=49) |
| Obstetrics | 0.778 (n=18) |
| Ophthalmology | 1.000 (n=1) |
| Orthopedic Surgery | 0.958 (n=24) |
| Otolaryngology | 0.929 (n=14) |
| Otolaryngology (ENT) | 0.786 (n=14) |
| Pediatrics | 0.875 (n=56) |
| Pharmacology | 1.000 (n=1) |
| Psychiatry | 0.911 (n=45) |
| Pulmonology (Respiratory Medicine) | 0.892 (n=65) |
| Radiology | 1.000 (n=1) |
| Rheumatology | 0.918 (n=49) |
| Urology | 0.923 (n=13) |
| Vascular Surgery | 1.000 (n=2) |
| nan | 1.000 (n=1) |
Reuse this result
Download per-item predictions (parquet) · JSON
@misc{medbench_2224a6c5f8d6,
title = {DeepSeek V4.1 Flash (2026-10-17) on Korean clinical knowledge (KMLE-level MCQ) (clinicalqa@0)},
author = {{medbench.kr}},
year = {2026},
howpublished = {\url{https://medbench.kr/r/2224a6c5f8d6}},
note = {Evaluated 2026-10-29, harness p0.1, run key 4a339f14894d}
}
Versions
| model digest | openrouter/deepseek/deepseek-v4.1-flash-1017 |
|---|---|
| task spec hash | 77e37cd0ca6bded7fba03792ec1922342b3f6a7dc3ce602881ce196e6aa7377c |
| harness | p0.1 |
| run key | 4a339f14894dc4d3e0deac8f1378217632804927a48f01bd95438f10fc655610 |
| config | {"max_tokens": 16384, "prompt": "clinicalqa-p0", "temperature": 0} |
Data
ClinicalQA: Korean Medical Knowledge QA Benchmark, CC BY 4.0 (upstream Apache-2.0).
SNUH-HARI. ClinicalQA: Korean Medical Knowledge QA Benchmark. Seoul National University Hospital Healthcare AI Research Institute, 2025. doi:10.23421/2026.0800