DeepSeek V4.1 Flash 552B-A16B on clinicalqa@0
Run 70314bc6d7e9 · baseline tier · evaluated 2026-10-29
Scores
| accuracy | 0.900 (0.883–0.918) |
|---|---|
| parse rate | 0.997 |
| n | 1045 |
| tokens (in / out) | 435518 / 923636 |
| cost (USD) | 0.0000 |
By category
| Allergy and Immunology | 0.861 (n=36) |
|---|---|
| Cardiology | 0.944 (n=89) |
| Dermatology | 0.833 (n=12) |
| Emergency Medicine | 1.000 (n=32) |
| Endocrinology | 0.912 (n=68) |
| Endocrinology, metabolic, and nutritional disorders | 1.000 (n=1) |
| Forensic Medicine | 1.000 (n=1) |
| Gastroenterology | 0.929 (n=112) |
| General Surgery | 0.931 (n=58) |
| Geriatric Medicine | 1.000 (n=1) |
| Gynecology | 0.800 (n=70) |
| Hematology and Oncology | 0.958 (n=48) |
| Infectious Diseases | 0.827 (n=52) |
| Musculoskeletal (e.g., costochondritis) | 1.000 (n=1) |
| Nephrology | 0.883 (n=111) |
| Neurology / Neurosurgery | 0.918 (n=49) |
| Obstetrics | 0.778 (n=18) |
| Ophthalmology | 1.000 (n=1) |
| Orthopedic Surgery | 0.958 (n=24) |
| Otolaryngology | 0.929 (n=14) |
| Otolaryngology (ENT) | 0.857 (n=14) |
| Pediatrics | 0.893 (n=56) |
| Pharmacology | 1.000 (n=1) |
| Psychiatry | 0.911 (n=45) |
| Pulmonology (Respiratory Medicine) | 0.908 (n=65) |
| Radiology | 1.000 (n=1) |
| Rheumatology | 0.878 (n=49) |
| Urology | 0.923 (n=13) |
| Vascular Surgery | 1.000 (n=2) |
| nan | 0.000 (n=1) |
Reuse this result
Download per-item predictions (parquet) · JSON
@misc{medbench_70314bc6d7e9,
title = {DeepSeek V4.1 Flash 552B-A16B on Korean clinical knowledge (KMLE-level MCQ) (clinicalqa@0)},
author = {{medbench.kr}},
year = {2026},
howpublished = {\url{https://medbench.kr/r/70314bc6d7e9}},
note = {Evaluated 2026-10-29, harness p0.1, run key 26fc2ddc80f7}
}
Versions
| model digest | kgpu/deepseek-v4.1-flash-ax-552b-a16b |
|---|---|
| task spec hash | 77e37cd0ca6bded7fba03792ec1922342b3f6a7dc3ce602881ce196e6aa7377c |
| harness | p0.1 |
| run key | 26fc2ddc80f74524cc3e64502f7b6541425f7de2bcf614eab33fc603390a3661 |
| config | {"max_tokens": 16384, "prompt": "clinicalqa-p0", "temperature": 0} |
Data
ClinicalQA: Korean Medical Knowledge QA Benchmark, CC BY 4.0 (upstream Apache-2.0).
SNUH-HARI. ClinicalQA: Korean Medical Knowledge QA Benchmark. Seoul National University Hospital Healthcare AI Research Institute, 2025. doi:10.23421/2026.0800