Model Validation
/model-validationNEWWhat it does
Design or audit the clinical-validation study for an engineer-built medical-imaging model: patient-level split disjointness, the data-leakage taxonomy, internal vs external validation, comparator design, and task-correct metric selection — with a deterministic split-leakage gate.
Highlights
- ✓Split-leakage gate proves patient disjointness
- ✓Leakage taxonomy + tuning-on-test checks
- ✓CLAIM 2024 / TRIPOD+AI / STARD-AI reporting fit
Install this skill
git clone https://github.com/aperivue/medsci-skills.git
cp -r medsci-skills/skills/model-validation ~/.claude/skills/Related skills
"Which architecture for which research question" decision tool: maps task, modality, data scale, and class imbalance to a paper-grounded architecture shortlist — each with the source paper, when-to-use, medical-imaging use, reference implementation, and the matching scaffold template.
Model Scaffold/model-scaffoldGenerate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation, classification, detection, synthesis, or self-supervised pretraining — with a patient-level seed-locked split, train/evaluate scripts, and a Methods stub. Integrates MONAI / nnU-Net, never reimplements them.
Model Card & Datasheet/model-cardGenerate the documentation an engineer-built model must carry — a Model Card (Mitchell et al. 2019), a Datasheet for its dataset (Gebru et al. 2021), and a METRIC data-quality pass — filled only from user-supplied facts, then verify every required section is present with a completeness gate.
Model Evaluation/model-evaluationCompute and report task-correct held-out metrics for a trained model — Dice + HD95/NSD, AUROC/AUPRC with bootstrap CIs, FROC/mAP, calibration, and subgroup slices — emitting a per-case table for analyze-stats. Numbers come only from executed code.