Korean clinical knowledge (KMLE-level MCQ) preview

clinicalqa@0 · mcq · primary metric: accuracy · spec hash 77e37cd0ca6b

ClinicalQA has been public on Hugging Face since 2025-04-04. Models trained after that date may have seen it.

#	Model	Type	Tier	accuracy (95% CI)	n	Cost (USD)	Date
1	GLM-5.3 753B-A40B kgpu/glm-5.3-ax-753b-a40b	llm	baseline	0.909 (0.892–0.926)	1045	0.0000	2026-10-29
2	Gemini 3.8 Flash openrouter/google/gemini-3.8-flash	llm	baseline	0.908 (0.891–0.925)	1045	2.3561	2026-10-29
3	DeepSeek V4.1 Flash (2026-10-17) openrouter/deepseek/deepseek-v4.1-flash-1017	llm	baseline	0.904 (0.887–0.922)	1045	0.4731	2026-10-29
4	Qwen3.6 35B-A3B kgpu/qwen3.6-35b-a3b	llm	baseline	0.903 (0.885–0.922)	1045	2.2815	2026-10-29
5	DeepSeek V4.1 Flash 552B-A16B kgpu/deepseek-v4.1-flash-ax-552b-a16b	llm	baseline	0.900 (0.883–0.918)	1045	0.0000	2026-10-29
6	GPT-6 Luna openrouter/openai/gpt-6-luna	llm	baseline	0.900 (0.882–0.919)	1045	0.1103	2026-10-29
7	DeepSeek V4 Pro 1.6T-A49B kgpu/deepseek-v4-pro-ax-1.6t-a49b	llm	baseline	0.894 (0.876–0.911)	1045	0.0000	2026-10-29
8	Qwen3.8 Flash Next 180B-A6B kgpu/qwen3.8-flash-next-180b-a6b	llm	baseline	0.894 (0.877–0.913)	1045	0.0000	2026-10-29
9	Claude Haiku 5.5 openrouter/anthropic/claude-haiku-5.5	llm	baseline	0.885 (0.866–0.903)	1045	0.4409	2026-10-29
10	Gemma 4 26B-A4B kgpu/gemma-4-26b-a4b	llm	baseline	0.874 (0.854–0.894)	1045	0.2035	2026-10-29
11	MedGemma 1.5 4B kgpu/medgemma-1.5-4b	llm	baseline	0.765 (0.739–0.789)	1045	0.1244	2026-10-29
12	Majority class (always B) medbench/majority-b	baseline	baseline	0.343 (0.314–0.370)	1045	0.0000	2026-10-29

Data

ClinicalQA: Korean Medical Knowledge QA Benchmark (text). License: CC BY 4.0 (upstream Apache-2.0). Source: https://doi.org/10.23421/2026.0800

SNUH-HARI. ClinicalQA: Korean Medical Knowledge QA Benchmark. Seoul National University Hospital Healthcare AI Research Institute, 2025. doi:10.23421/2026.0800

Task specification

{
  "dataset": "clinicalqa",
  "kind": "mcq",
  "preview": true,
  "primary_metric": "accuracy",
  "prompt": "clinicalqa-p0",
  "slug": "clinicalqa",
  "source": {
    "file": "data/train-00000-of-00001.parquet",
    "repo": "snuh/ClinicalQA",
    "revision": "29bdc9043f3e693b7671f868e2038ec7ac288acd"
  },
  "split": "test-only (all 1,045 items)",
  "title": "Korean clinical knowledge (KMLE-level MCQ)",
  "version": 0
}

JSON: /v1/tasks/clinicalqa · /v1/leaderboard?task=clinicalqa@0