Biosignal › SNUH LYDUS ECG 50K › lydus-ecg12-dx@0

Diagnoses from a 10-second 12-lead ECG (multi-label, PhysioNet/CinC Challenge 2021 score) preview

Data: doi.org/10.23421/2025.0704 · CC BY 4.0 · multilabel · primary metric: challenge_score · spec 663e73f47199

No results yet.

Data

SNUH LYDUS ECG 50K (biosignal). License: CC BY 4.0. Source: https://doi.org/10.23421/2025.0704

Seoul National University Hospital. SNUH LYDUS ECG 50K (version 1.1.1). KHDP. 2025. doi:10.23421/2025.0704

Task specification

{
  "baselines": {
    "medbench/lydus-hgb-beat-features": "one HistGradientBoostingClassifier per class (300 iterations, learning rate 0.1, random_state 0; classes with fewer than 20 train positives always answer no) on lead II RR features, each lead's median beat (-200 to +400 ms around lead II R peaks, every 10 ms) and amplitude percentiles of the band-passed leads, with age and sex",
    "medbench/lydus-resnet1d": "the network of Ribeiro et al. 2020 (antonior92/automatic-ecg-diagnosis @ 4039348, MIT), ported to PyTorch and trained from scratch on the train split with its train.py settings; input resampled to 400 Hz and padded to 4,096 samples; the weights of the lowest validation loss",
    "medbench/prior": "sinus rhythm only as the outputs (the Challenge metric's inactive answer, 0 by construction), train prevalence as the scores",
    "reduced_lead_sets": "both trained baselines are also fitted on the 6-, 4-, 3- and 2-lead sets (report only)",
    "thresholds": "per class, the highest F-measure on the validation split; 0.5 for a class with fewer than 10 validation positives (with a handful of positives the best F-measure can sit at the lowest score and answer the class for every record)"
  },
  "counts": {
    "groups": {
      "test": 5228,
      "train": 17586,
      "validation": 2544
    },
    "hedged_only": {
      "CRBBB|RBBB": 1,
      "LQT": 2,
      "PAC|SVPB": 31,
      "TAb": 923
    },
    "positives": {
      "test": {
        "AF": 3517,
        "AFL": 492,
        "BBB": 0,
        "Brady": 1,
        "CLBBB|LBBB": 315,
        "CRBBB|RBBB": 1555,
        "IAVB": 1749,
        "IRBBB": 645,
        "LAD": 753,
        "LAnFB": 454,
        "LPR": 0,
        "LQRSV": 352,
        "LQT": 1029,
        "NSIVCB": 386,
        "NSR": 3094,
        "PAC|SVPB": 271,
        "PR": 1027,
        "PRWP": 1,
        "PVC|VPB": 864,
        "QAb": 9,
        "RAD": 77,
        "SA": 144,
        "SB": 649,
        "STach": 191,
        "TAb": 2195,
        "TInv": 34
      },
      "train": {
        "AF": 12884,
        "AFL": 1487,
        "BBB": 0,
        "Brady": 4,
        "CLBBB|LBBB": 922,
        "CRBBB|RBBB": 5696,
        "IAVB": 5960,
        "IRBBB": 2381,
        "LAD": 2906,
        "LAnFB": 1399,
        "LPR": 0,
        "LQRSV": 1200,
        "LQT": 3762,
        "NSIVCB": 1227,
        "NSR": 10668,
        "PAC|SVPB": 819,
        "PR": 3427,
        "PRWP": 4,
        "PVC|VPB": 3283,
        "QAb": 28,
        "RAD": 357,
        "SA": 473,
        "SB": 2179,
        "STach": 732,
        "TAb": 7955,
        "TInv": 103
      },
      "validation": {
        "AF": 1955,
        "AFL": 199,
        "BBB": 0,
        "Brady": 0,
        "CLBBB|LBBB": 101,
        "CRBBB|RBBB": 839,
        "IAVB": 859,
        "IRBBB": 323,
        "LAD": 416,
        "LAnFB": 205,
        "LPR": 0,
        "LQRSV": 178,
        "LQT": 565,
        "NSIVCB": 184,
        "NSR": 1483,
        "PAC|SVPB": 126,
        "PR": 407,
        "PRWP": 0,
        "PVC|VPB": 477,
        "QAb": 4,
        "RAD": 36,
        "SA": 52,
        "SB": 298,
        "STach": 127,
        "TAb": 1228,
        "TInv": 16
      }
    },
    "records": {
      "test": 9999,
      "train": 34998,
      "validation": 4998
    }
  },
  "data": {
    "files": {
      "LYDUS_SNUH_ECG_ANONYMIZED_50K_0512.csv": "841fd752cdb7420d6f1e70e39c364dd21fc89120d2506421f0a40cd896ad5045",
      "LYDUS_SNUH_ECG_ANONYMIZED_50k_500hz_1219.npz": "c53419da2e875da1db145f975915954eff9f05778acae2945ae25373b9b47154"
    },
    "signals": "50,000 records x 8 stored leads (I, II, V1-V6) x 5,000 int16 samples (500 Hz, 10 s), stored uncompressed and read by memory map. The dataset page says 12 channels; the file holds the 8 independent leads, and III, aVR, aVL and aVF are derived (III = II - I, aVR = -(I + II) / 2, aVL = I - II / 2, aVF = II - I / 2). Amplitude units are not documented; lead II's QRS spans about 215 counts (median over 2,000 records), consistent with about 4.9 uV per count. Inputs keep the stored counts."
  },
  "dataset": "lydus-snuh-ecg-50k",
  "deviations": [
    "Protocol transfer: the Challenge never used LYDUS. Its datasets carry diagnoses coded to SNOMED CT by their sources; here labels come from a medbench-made map of the electrocardiograph's interpretation statements (reviewed by two clinicians according to the dataset page, not re-read) to the Challenge's codes. Hedged statements (\"Possible\", \"Cannot rule out\", \"consider\", \"probable\", \"suspected\", \"R/O\") are not labels.",
    "Labels come from the dx text, not from the release's reviewed rhythm, premature beat, bundle branch block and AV block columns: those columns follow keywords of the same text (for example \"Possible pacing\" for 1,072 records whose text says \"with a competing junctional pacemaker\", the VPC flag for 97.6% of \"with premature ventricular or aberrantly conducted complexes\" but 7% of \"Premature ventricular complexes\", incomplete right bundle branch block counted as RBBB) and name fewer classes.",
    "Partial class set: 24 of the 26 scored classes occur; BBB (6374002) and prolonged PR (164947007) never do, and Brady (5 records), PRWP (5) and QAb (41) are rare (test: 1, 1 and 9). The scorer and weights.csv are unchanged, so every class keeps its weights; macro AUROC and macro F-measure average the classes where they are defined, as the scorer does (a class without test positives has no AUROC, and its F-measure counts only if a model predicts it).",
    "Leads: LYDUS stores 8 independent leads; III, aVR, aVL and aVF are derived from I and II, so the 6-lead set carries no more information than the 2-lead set and the 4-lead set no more than the 3-lead set. The Challenge's recordings stored all 12 (limb leads are linear in I and II there too, apart from noise).",
    "Population: LYDUS 50K was drawn from the 160K release to enrich abnormal rhythms and conduction (inclusion criteria on KHDP), so sinus rhythm (30.9% of test records) is rarer than in the Challenge's sources.",
    "Split: the Challenge tested on hidden sets, some from hospitals absent from training. Here a 70/10/20 split of LYDUS by groups of likely repeat ECGs (split.rules), because the release has no patient id. One hospital only.",
    "Intervals: 95% intervals from 1,000 bootstrap samples of groups; the Challenge reported point values.",
    "Records left out: 5 without an interpretation and 3 without signal (exclusions)."
  ],
  "exclusions": {
    "flat_signal": "all 8 stored leads constant: 3 records",
    "no_analysis": "the dx says the electrocardiograph made no interpretation (no or fewer than 4 QRS complexes): 5 records"
  },
  "input": {
    "also": [
      "age",
      "sex"
    ],
    "hz": 500,
    "lead_sets": {
      "12": [
        "I",
        "II",
        "III",
        "aVR",
        "aVL",
        "aVF",
        "V1",
        "V2",
        "V3",
        "V4",
        "V5",
        "V6"
      ],
      "2": [
        "I",
        "II"
      ],
      "3": [
        "I",
        "II",
        "V2"
      ],
      "4": [
        "I",
        "II",
        "III",
        "V2"
      ],
      "6": [
        "I",
        "II",
        "III",
        "aVR",
        "aVL",
        "aVF"
      ]
    },
    "leads": [
      "I",
      "II",
      "III",
      "aVR",
      "aVL",
      "aVF",
      "V1",
      "V2",
      "V3",
      "V4",
      "V5",
      "V6"
    ],
    "never": "new_id or any other record id, the record's position in the release, the dx text, the reviewed rhythm, premature beat, bundle branch block and AV block columns, and the electrocardiograph's measurements (rates, intervals, axes): they come from the same interpretation as the labels (leakage_checks)",
    "samples": 5000
  },
  "kind": "multilabel",
  "label": {
    "map": "registry/maps/lydus-dx-snomed-v0.csv",
    "review": "The map is medbench-made and is reviewed by three frontier models before publication (planned, not run: no paid route was used in this build). Plan: each row with a code, or a finding without one, goes to GPT, Gemini and Claude flagships with the line, its count, three example records' full dx text and the Challenge tables; the rubric asks whether the role and each code are right and whether a scored class is missing; unanimous approval keeps a row, two rejections change it, anything else is revised and reviewed again (SOP item review). About 345 rows outside the 383 explanation lines; batches of five rows.",
    "rules": "A finding's codes are labels. A hedge line (\"Possible\", \"Cannot rule out\", \"Borderline\", \"probable\", \"r/o\") makes the next finding hedged, and statements worded \"possible\", \"probable\", \"consider\", \"suspected\" or \"R/O\" carry hedged codes: hedged codes are never labels (reported in build_summary.json). Modifier, explanation, summary and quality lines carry no label (except \"ST &\", an unscored ST code). Every line of the data must be in the map and every row's n_records must match the data, or the build stops.",
    "scored_classes": [
      "AF",
      "AFL",
      "BBB",
      "Brady",
      "CLBBB|LBBB",
      "CRBBB|RBBB",
      "IAVB",
      "IRBBB",
      "LAD",
      "LAnFB",
      "LPR",
      "LQRSV",
      "LQT",
      "NSIVCB",
      "NSR",
      "PAC|SVPB",
      "PR",
      "PRWP",
      "PVC|VPB",
      "QAb",
      "RAD",
      "SA",
      "SB",
      "STach",
      "TAb",
      "TInv"
    ],
    "source": "the record's dx text (the electrocardiograph's interpretation statements, GE 12SL wording for most records, Schiller wording with \"because\" explanations for 474), one statement per line, mapped line by line with registry/maps/lydus-dx-snomed-v0.csv (medbench-made, one row per distinct line: role, SNOMED CT codes from the Challenge 2021 tables, hedged codes, note)"
  },
  "leakage_checks": {
    "best_constant": "the input-blind answer with the highest train Challenge score (classes added greedily), scored on test",
    "demographics_only": "gradient boosting on age and sex alone",
    "measurements_only": "gradient boosting on the electrocardiograph's measurements (ventricular and atrial rate, PR, QRS, QT, QTc, P, QRS and T axes) with age and sex: the labels come from the same interpretation (prolonged QT from QTc, axis deviations from the QRS axis), so this shows what those numbers are worth and why they are never inputs",
    "order_only": "gradient boosting on new_id alone: what the record's position in the release would be worth (neighbouring records are often the same patient), which is why instance ids are opaque",
    "record_level_split": "the gradient-boosting baseline trained and tested on a record-level random split of the same fractions (seed 20261030), against this group split: how much repeat ECGs across the split would inflate a score",
    "split_leakage": "split.residual_leakage (build_summary.json, leakage)"
  },
  "limitations": [
    "The data are public (KHDP, open access), so the test records and their dx text can be looked up; a model trained on LYDUS outside medbench's train split may have seen the test records.",
    "Labels are the electrocardiograph's statements, not an over-read: a model is scored on agreeing with the machine. Classes defined by the machine's measurements (prolonged QT, axis deviations, low voltage) follow its thresholds.",
    "The Challenge metric gives partial credit across classes, so an input-blind answer of many common classes scores well above the sinus-rhythm-only prior: the 16 classes chosen greedily on train (AF, TAb, RBBB, NSR, IAVB, LQT, PVC, LAD, PR, IRBBB, LAnFB, SB, NSIVCB, LQRSV, STach, PAC) for every record score 0.495 on test (leakage_checks.best_constant). The prior stays the Challenge's inactive answer (0 by construction).",
    "Rare classes move the macro values: Brady and PRWP have 1 test positive each and QAb 9, so a bootstrap resample without them averages over fewer classes; macro AUROC and macro F-measure intervals are therefore wider on the upper side. The Challenge score does not average over classes and is not affected this way."
  ],
  "metrics": {
    "bootstrap": {
      "by": "group",
      "n": 1000
    },
    "by_class": [
      "positives",
      "auroc",
      "auprc",
      "f1"
    ],
    "definitions": "as evaluate_model.py computes them: Challenge metric with weights.csv and sinus rhythm (426783006) as the inactive class; AUROC and F-measure per class, averaged over the classes where they are defined (macro); accuracy is the share of records whose 26 outputs all equal the labels. AUPRC per class is reported beside them.",
    "primary": "challenge_score",
    "secondary": [
      "macro_auroc",
      "macro_f1",
      "accuracy"
    ]
  },
  "owner_decisions": [
    "D17: confirm on /admin that LYDUS-SNUH-ECG-50K patients are SNUH patients (KHDP summary: ECG examinations at SNUH, 2010-2020).",
    "Split: keep this group split (residual repeats estimated at about 6-16% of test records), use a record-level split (about 37%), or ask the LYDUS authors for a patient map (the 160K release's rid is described as a patient identifier, yet repeats are present).",
    "Accept the Challenge 2021 metric with a partial class set (24 of 26 classes present; 2 absent, 3 rare).",
    "D1: is a protocol transfer (published scoring on a dataset its authors never used) an A1 build or provisional?",
    "Labels from the dx text rather than the reviewed columns (deviations).",
    "The three-model review of the dx map (label.review): run it, and with which budget.",
    "Reduced lead sets: report-only beside the 12-lead card (this build), or cards of their own.",
    "Input-blind reference for the headroom check: the Challenge's always-sinus-rhythm answer (0, this spec's prior) or the best constant answer (0.495 on test, limitations)."
  ],
  "prediction": "for each record and each of the 26 scored classes a binary output and a score in [0, 1], as the Challenge's output files; classes may be named by any of their equivalent codes",
  "preview": true,
  "primary_metric": "challenge_score",
  "protocol": {
    "adopted": "The PhysioNet/Computing in Cardiology Challenge 2021 task and scoring: every diagnosis of a 10-second 12-lead ECG among the Challenge's 26 scored classes (SNOMED CT codes, equivalent codes scored as one class), from the ECG with the patient's age and sex, scored with the Challenge metric (weights.csv: full credit for the right class, partial credit for a similar one, normalised so that the right labels score 1 and always answering sinus rhythm scores 0). The scorer of the Challenge's evaluation repository is vendored unchanged and checked against the original script. The Challenge's reduced lead sets (6, 4, 3 and 2 leads) are defined here and scored the same way. This is a protocol transfer: the Challenge never used LYDUS, so the labels come from a medbench-made map of the dataset's diagnosis statements to the Challenge's SNOMED CT codes, and the split is medbench's.",
    "commit": "e2a75fc01f729cb74cc4e853e054ce81e28381fc",
    "dataset_release": "LYDUS-SNUH-ECG-50K 1.1.1, KHDP, doi:10.23421/2025.0704, CC BY 4.0 (ECG examinations at Seoul National University Hospital, 2010-2020; waveforms and the electrocardiograph's interpretation, reviewed by an anesthesiologist and an intensivist; drawn from the 160K release to enrich abnormal rhythms and conduction)",
    "license": "BSD-2-Clause, Copyright (c) 2020, 2021 PhysioNet/Computing in Cardiology Challenges. evaluate_model.py, helper_code.py, weights.csv, weights_abbreviations.csv and the two dx mapping tables are vendored unchanged in third_party/evaluation-2021/; the scoring functions are ported in src/medbench/metrics_challenge2021.py.",
    "repository": "https://github.com/physionetchallenges/evaluation-2021",
    "source": "Reyna MA, Sadr N, Perez Alday EA, Gu A, Shah AJ, Robichaux C, Rad AB, Elola A, Seyedi S, Ansari S, Ghanbari H, Li Q, Sharma A, Clifford GD. Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021. Comput Cardiol. 2021;48. doi:10.23919/CinC53138.2021.9662687",
    "weights_sha256": "668cb04555a97e8977488cb4bb6088d0d2bf9e59557945deedf29d5065c14d2b"
  },
  "slug": "lydus-ecg12-dx",
  "split": {
    "by": "group",
    "evidence": "Measured 2026-10-30 on the 49,667 records with a median-beat template. Pairs of records whose templates correlate >= 0.96 and whose ages differ by at most 2 years: 25,273 of the same sex, 969 of opposite sex (who cannot be the same patient); at >= 0.98: 6,045 against 8. Records next to each other in new_id order are of the same sex and age in 39.4% of pairs one apart, against 1.2% for records 50-200 apart. So LYDUS holds repeat ECGs of one patient, although the 160K page counts 167,199 patients. Linking: 25,273 template and 37,634 index links; 25,360 groups, 30,407 records in groups of two or more, the largest 1,152 records.",
    "file": "registry/splits/lydus-snuh-ecg-50k-1.1.1-split.csv",
    "fractions": [
      0.7,
      0.1,
      0.2
    ],
    "method": "groups are the connected components of the links (named by their smallest new_id); the sorted group names are shuffled with numpy default_rng(seed) and fill train, validation and test by cumulative share of records",
    "residual_leakage": "Test records with a likely repeat outside test, estimated as test records with a same-sex non-test neighbour (ages within 2 years) at a template correlation of at least t minus those with an opposite-sex one: with this split 1,638 of 9,999 at t = 0.93 and 582 at 0.95 (none left at 0.96 or by the index rule); with a record-level random split of the same fractions 3,716, 3,783 and 3,270, and 2,568 by the index rule (build_summary.json, leakage). Repeats whose median beats correlate below 0.93 (another rhythm, years apart) are not detected.",
    "rules": {
      "index": {
        "max_age_gap": 1,
        "max_id_gap": 10
      },
      "template": {
        "max_age_gap": 2,
        "min_corr": 0.96
      }
    },
    "seed": 20261030,
    "sha256": "a7fddba5764a73d8c1c936296fa52909a822b797be321ec9448c7cc345ea3779",
    "unit": "group of records linked as likely repeat ECGs of one patient (the release has no patient id; see evidence)"
  },
  "title": "Diagnoses from a 10-second 12-lead ECG (multi-label, PhysioNet/CinC Challenge 2021 score)",
  "version": 0,
  "visibility": "hidden"
}

JSON: /v1/tasks/lydus-ecg12-dx · /v1/leaderboard?task=lydus-ecg12-dx@0