64 STATISTICS, EVIDENCE & INCIDENTS
# 65 STATISTICS, EVIDENCE & INCIDENTS
Statistics, evidence-based medicine and clinical governance form a small but reliable slice of MRCS Part A. The questions are not about being a statistician — they are about whether you can pick the right test, interpret a 2×2 table, recognise a study design, and know what to do when something goes wrong on the ward. Get the patterns straight and these become some of the easiest marks on the paper.
Levels of evidence
Evidence is graded by how robustly it controls bias and confounding. The hierarchy descends from synthesised randomised data to a single expert's opinion.
| Level | Source |
|---|---|
| 1a | Systematic review / meta-analysis of RCTs |
| 1b | Individual RCT (with narrow confidence interval) |
| 2 | Cohort study (or systematic review of cohorts) |
| 3 | Case-control study |
| 4 | Case series / case report |
| 5 | Expert opinion, bench research, first principles |
👩⚕️ A well-designed RCT outranks a meta-analysis of poor trials — garbage in, garbage out. Examiners usually keep it simple: meta-analysis of RCTs sits at the top.
Study designs
Each design answers a different question. Pick the design from the verb in the stem.
| Design | Direction | Best for | Key statistic |
|---|---|---|---|
| RCT | Prospective, randomised | Treatment effect | RR, ARR, NNT |
| Cohort | Prospective (usually) | Incidence, risk factors, prognosis | Relative risk |
| Case-control | Retrospective | Rare diseases, multiple exposures | Odds ratio |
| Cross-sectional | Snapshot in time | Prevalence | Prevalence (%) |
| Case series / report | Descriptive | Novel observations | None |
| Systematic review / meta-analysis | Synthesis | Pooled estimate of effect | Pooled RR / OR |
➡ Rare disease, looking back at exposures ➡ case-control (odds ratio).
➡ Following healthy people forward to see who develops disease ➡ cohort (relative risk).
➡ "How many people in Bristol have gallstones today?" ➡ cross-sectional (prevalence).
Diagnostic test statistics
Every screening/diagnostic question reduces to a 2×2 table. Memorise the layout once and the rest follows.
| Disease + | Disease − | |
|---|---|---|
| Test + | TP | FP |
| Test − | FN | TN |
| Metric | Formula | Plain English |
|---|---|---|
| Sensitivity | TP / (TP + FN) | Of those with disease, how many test positive |
| Specificity | TN / (TN + FP) | Of those without disease, how many test negative |
| PPV | TP / (TP + FP) | Of those who test positive, how many truly have it |
| NPV | TN / (TN + FN) | Of those who test negative, how many are truly disease-free |
👩⚕️ SnNOUT — a Snsitive test, when Negative, rules OUT disease (used for screening).
👩⚕️ SpPIN — a Specific test, when Positive, rules IN disease (used for confirmation).
Sensitivity and specificity are intrinsic to the test — they don't change with disease frequency. PPV and NPV depend on prevalence: in a rare disease, even a brilliant test has a low PPV because most positives are false positives. This is the most commonly tested trap in the entire topic.
Relative risk, odds ratio, NNT
- Absolute risk reduction (ARR) = risk in control − risk in treatment group.
- Relative risk (RR) = risk in exposed / risk in unexposed. Used in cohort and RCT.
- Odds ratio (OR) = odds of exposure in cases / odds in controls. Used in case-control.
- Number needed to treat (NNT) = 1 / ARR. Round up.
- Number needed to harm (NNH) = 1 / absolute risk increase of adverse event.
➡ RR or OR of 1 = no effect. >1 = increased risk. <1 = protective.
Confidence intervals and p-values
- 95% confidence interval (CI) is the range within which the true population value lies with 95% certainty.
- For RR / OR, if the 95% CI crosses 1, the result is not statistically significant.
- For a difference (e.g. mean difference), the null value is 0, not 1.
- p-value = probability that the observed result (or more extreme) occurred by chance if the null hypothesis were true. p < 0.05 is the conventional threshold.
| Error | What happened | Driven by |
|---|---|---|
| Type I (α) | False positive — rejected a true null | Multiple testing, low threshold |
| Type II (β) | False negative — missed a true effect | Underpowered study, small sample |
- Power = 1 − β, conventionally set at 80%. A study with low power risks missing a real effect.
Choosing the right statistical test
| Data | Two groups, paired | Two groups, unpaired | >2 groups |
|---|---|---|---|
| Normally distributed (parametric) | Paired t-test | Unpaired t-test | ANOVA |
| Not normally distributed (non-parametric) | Wilcoxon signed rank | Mann-Whitney U | Kruskal-Wallis |
| Categorical | McNemar | Chi-square (or Fisher's if small) | Chi-square |
👩⚕️ "Same patients before and after" = paired. "Two separate groups" = unpaired. "Some patients had very high values / skewed data" = non-parametric.
Bias and confounding
Bias is systematic error in how a study is designed or run. Confounding is a third variable distorting the apparent relationship between exposure and outcome.
| Bias | Example |
|---|---|
| Selection | Recruited only motivated patients |
| Recall | Cases remember exposures better than controls |
| Observer | Investigator influenced by knowing allocation |
| Lead-time | Earlier diagnosis falsely lengthens "survival" |
| Length | Screening preferentially picks up slow-growing disease |
| Attrition | Drop-outs differ systematically from completers |
- Confounding (e.g. smoking confounds the link between coffee and lung cancer) is controlled by randomisation, matching, stratification or multivariable regression.
- Effect modification is different — the exposure–outcome relationship genuinely differs across subgroups (e.g. a drug works in men but not women). You don't adjust it out; you report it.
Clinical governance and incident management
Clinical governance = the framework through which the NHS is accountable for continuously improving quality. The pillars (mnemonic CRISP):
- Clinical effectiveness & research
- Risk management (incidents, audit, complaints)
- Information & IT
- Staff training & education
- Patient & public involvement
Types of incident
| Term | Definition |
|---|---|
| Near miss | Error caught before it reached the patient — no harm |
| No-harm incident | Reached the patient but caused no harm |
| Adverse event | Harm caused by healthcare, not the disease |
| Never event | Wholly preventable, serious incident (e.g. wrong-site surgery, retained swab) |
Frontline tools
- WHO Surgical Safety Checklist — three phases: Sign-in (before anaesthesia), Time-out (before skin incision), Sign-out (before patient leaves theatre).
- SBAR — Situation, Background, Assessment, Recommendation. Standardised escalation.
- Datix — electronic incident reporting system.
- Root cause analysis (RCA) — structured retrospective review (often the "5 whys") used after serious incidents to identify systemic causes, not individual blame.
- Duty of candour — statutory obligation (since 2014) to be open with patients when something goes wrong: explain, apologise, document.
Types of audit
| Type | What it measures | Example |
|---|---|---|
| Structure | Resources, staffing, equipment | Are enough CT scanners available? |
| Process | Steps in care, guideline adherence | Time from clinic to GP letter; antibiotic prophylaxis given on time |
| Outcome | End results | 30-day mortality after laparotomy |
The audit cycle
Plan → Set standard → Measure practice → Compare to standard → Implement change → Re-audit. Without the second loop ("close the loop") it is not an audit — it is a survey.
👩⚕️ Audit vs research: audit measures current practice against an existing standard; research generates new knowledge. Audit does not need ethics committee approval; research does.
Research ethics
- Declaration of Helsinki (1964, repeatedly updated) — global ethical framework for research involving humans.
- Informed consent — voluntary, informed of risks/benefits/alternatives, capacity, no coercion.
- IRB / Research Ethics Committee — independent approval before any research begins.
[Image: MCQs banner]
Test yourself
Which level of evidence is a meta-analysis of randomised controlled trials?
- ((Level 1a::☑️ Top of the hierarchy; pooled RCT data with the least bias.))
- ((Level 1b::Single high-quality RCT — one step below meta-analysis.))
- ((Level 2::Cohort study — observational, not synthesised RCT data.))
- ((Level 5::Expert opinion — the bottom of the hierarchy.))
Which study design is best for investigating a rare disease?
- ((Case-control::☑️ Retrospective; cases and controls compared for past exposures; yields odds ratio.))
- ((Cohort::Needs huge numbers and long follow-up — impractical for rare disease.))
- ((Cross-sectional::Measures prevalence at a single time point, not causation.))
- ((RCT::Unethical and unfeasible to randomise rare-disease exposures.))
A cohort study reports a relative risk of 2.3 (95% CI 1.4–3.8) for smoking and pancreatic cancer. What does this mean?
- ((Smokers have 2.3× the risk and the result is statistically significant::☑️ CI does not cross 1, so significant; smokers carry 2.3-fold risk.))
- ((Result is not statistically significant::CI must cross 1 (for RR/OR) to be non-significant.))
- ((Smoking causes 2.3% more cancers::RR is a multiplier, not a percentage.))
- ((23 smokers needed to cause one cancer::That would be NNH, calculated from absolute risks.))
Which statistic is intrinsic to a test and unchanged by disease prevalence?
- ((Sensitivity::☑️ Sensitivity and specificity are properties of the test itself.))
- ((Positive predictive value::PPV falls as prevalence falls — most positives become false positives.))
- ((Negative predictive value::NPV depends on prevalence.))
- ((Number needed to treat::NNT applies to treatment effect, not diagnostic tests.))
A screening test for a rare cancer has PPV of 7%. What does this mean?
- ((7% of people who test negative have the disease::That describes 1 − NPV, not PPV.))
- ((7% of those who test positive truly have the disease::☑️ PPV = TP / (TP + FP); low PPV is expected in rare diseases.))
- ((7% of results are false positives::That is the false-positive rate, not PPV.))
- ((The test has 7% sensitivity::Sensitivity is TP/(TP+FN), independent of prevalence.))
What is the number needed to treat (NNT) if a drug reduces 30-day mortality from 10% to 6%?
- ((25::☑️ ARR = 10% − 6% = 4%; NNT = 1/0.04 = 25.))
- ((4::That is the absolute risk reduction as a percentage, not NNT.))
- ((40::Would require an ARR of 2.5%.))
- ((100::Would require an ARR of 1%.))
Which test compares blood loss (highly skewed data) between two independent surgical techniques?
- ((Paired t-test::Wrong — groups are independent, not the same patients before/after.))
- ((Unpaired t-test::Assumes normal distribution; skewed data violates this.))
- ((Mann-Whitney U::☑️ Non-parametric, two independent groups, skewed/ordinal data.))
- ((Chi-square::For categorical data, not continuous blood-loss volumes.))
Urea and creatinine measured before and after surgery in the same 87 patients, normally distributed. Best test?
- ((Paired t-test::☑️ Same patients before/after = paired; normal distribution allows parametric test.))
- ((Unpaired t-test::Data is paired, not from two independent groups.))
- ((Mann-Whitney U::Non-parametric — unnecessary when data is normally distributed.))
- ((Chi-square::For categorical, not continuous data.))
Which test compares a non-normally distributed parameter across three independent groups?
- ((Mann-Whitney U::Only compares two groups.))
- ((ANOVA::Parametric — assumes normal distribution.))
- ((Kruskal-Wallis::☑️ Non-parametric equivalent of ANOVA; >2 independent groups.))
- ((Paired t-test::For paired, normally distributed two-group data.))
A blood transfusion line is connected to the wrong patient's IV but spotted before infusion. This is:
- ((Near miss::☑️ Error did not reach the patient; no harm occurred.))
- ((Never event::Would require the transfusion to have actually been given.))
- ((Adverse event::Requires harm to have resulted.))
- ((Sentinel event::Used for events causing death or serious harm.))
Which is a "never event"?
- ((Postoperative wound infection::A complication, not a never event.))
- ((Wrong-site surgery::☑️ Classic never event — wholly preventable, serious.))
- ((Anastomotic leak::Recognised complication, not a never event.))
- ((Postoperative DVT despite prophylaxis::Adverse event, not a never event.))
A hospital measures the time taken for clinic letters to reach the GP. What kind of audit is this?
- ((Structure::Concerns resources/staffing, not workflow timeliness.))
- ((Process::☑️ Measures steps in care delivery and timelines.))
- ((Outcome::Would measure end results such as mortality or readmission.))
- ((Financial::Concerns costs and budgets, not clinical workflow.))
Which audit type evaluates a rise in mortality after a procedure?
- ((Process::Measures care steps, not end results.))
- ((Structure::Concerns resources and equipment.))
- ((Outcome::☑️ Mortality and complication rates are outcomes.))
- ((Financial::Concerns costs, not clinical outcomes.))
What is the final step of the audit cycle?
- ((Set the standard::That is the first step.))
- ((Implement change::Penultimate step, before re-audit.))
- ((Re-audit::☑️ Closes the loop — without it, it's a survey, not an audit.))
- ((Publish results::Not part of the formal cycle.))
A p-value of 0.03 means:
- ((A 3% probability the result occurred by chance if the null hypothesis were true::☑️ Standard definition; <0.05 is conventionally significant.))
- ((A 3% chance the treatment doesn't work::Common misinterpretation — p is not the probability of the hypothesis.))
- ((97% certainty the treatment works::Confuses p-value with confidence.))
- ((The study had 3% power::Power is 1 − β, unrelated to p.))
A trial fails to detect a real treatment effect because the sample size was too small. This is a:
- ((Type I error::False positive — rejecting a true null.))
- ((Type II error::☑️ False negative — missed a real effect due to low power.))
- ((Selection bias::Concerns how subjects were recruited, not statistical power.))
- ((Confounding::A third variable distorts the result, not the same as missed effect.))
Which framework is used for structured clinical handover and escalation?
- ((WHO checklist::Used in theatre, not for general escalation.))
- ((SBAR::☑️ Situation, Background, Assessment, Recommendation.))
- ((Datix::Incident reporting system, not handover.))
- ((Root cause analysis::Retrospective investigation of incidents.))
Which phase of the WHO Surgical Safety Checklist occurs immediately before skin incision?
- ((Sign-in::Performed before induction of anaesthesia.))
- ((Time-out::☑️ Final check before knife to skin — team, site, prophylaxis.))
- ((Sign-out::Performed before the patient leaves theatre.))
- ((Debrief::Not a formal phase of the WHO checklist.))
Revision summary
- Evidence hierarchy: 1a meta-analysis of RCTs → 1b RCT → 2 cohort → 3 case-control → 4 case series → 5 expert opinion.
- Study design by question: treatment ➡ RCT; prognosis/incidence ➡ cohort (RR); rare disease ➡ case-control (OR); prevalence ➡ cross-sectional.
- SnNOUT, SpPIN. Sensitivity/specificity are intrinsic; PPV/NPV depend on prevalence.
- NNT = 1/ARR. RR or OR = 1 → no effect. 95% CI crossing 1 → not significant.
- p < 0.05 = conventionally significant. Type I = false positive; Type II = false negative; power = 1 − β (usually 80%).
- Stats tests: paired/parametric ➡ paired t; unpaired/parametric ➡ unpaired t; non-parametric two groups ➡ Mann-Whitney U; non-parametric >2 groups ➡ Kruskal-Wallis; categorical ➡ chi-square.
- Never event = wholly preventable serious incident (wrong-site, retained swab). Near miss = didn't reach the patient.
- WHO checklist: sign-in → time-out → sign-out. SBAR for escalation. Datix for reporting. Duty of candour for open disclosure.
- Audit cycle closes with re-audit. Structure / process / outcome. Audit ≠ research; research needs ethics approval (Declaration of Helsinki).
- Clinical governance pillars (CRISP): Clinical effectiveness, Risk management, Information, Staff, Patient involvement.