Our own work, warts and all.

Why this page exists.

Episode 04 of Road to CEPAS 2026 applies the Episode 03 reading to my own group's paper — van den Berg et al. 2023, the largest unselected single-centre cohort in this corner of the LOS literature and the most extensive retrospective impact simulation done for it to date. The Substack post argues that the paper's central strength and its hardest limitations are not separate items to be weighed against each other — they are the same decisions seen from two sides.

This page is the working file: the two decisions laid out against the figures in the paper and its supplement, the proxy the whole impact story rests on, and the supplementary discrepancy I could not reconstruct. It is organised around the same two decisions Episode 03 identified as the field's least-documented, rather than re-running the full eight-dimension grid — that grid already exists on the Episode 03 page.

The headline, and the number beside it.

The abstract reports detection of LOS in at least 47% of patients before clinical suspicion, without exceeding an alarm-fatigue threshold of three alarms per day. That is the sentence most readers take away. At that same operating point the precision is 3.93% on the training set and 4.41% on the test set — a number that is not in the abstract.

Both figures come from one alarm configuration; they summarise different aspects of it. The detection fraction is how many LOS episodes are flagged before the culture. The precision is the positive predictive value of the alarms that policy generates. A precision of ~4% means roughly 96 of every 100 alarms do not pan out — at the operating point chosen precisely because it matched the alarm burden a NICU already tolerates.

Signal stack: one decision, two faces.

The model sees heart rate and oxygen saturation, and nothing else — low-frequency, one sample a minute, the values on every monitor on the ward. CRP, blood pressure and white-cell counts were deliberately excluded.

• The strength. A clinician orders a CRP only when they already suspect something. A model trained on CRP learns to detect clinical suspicion and then claims credit for predicting it. Excluding anything carrying that prior knowledge is, I would argue, the best decision in the paper.
• The first limitation. Because the model keys on physiological deterioration alone, it cannot say why a baby is deteriorating. The paper states it directly: the algorithm may not be specific to sepsis and could respond to other inflammatory conditions such as NEC. It is more accurately a deterioration model aimed at a sepsis question.
• The second limitation. Two physiological streams and no specific marker put a hard ceiling on how cleanly the model can separate a baby becoming septic from one briefly unstable for ordinary reasons. The supplement shows XGBoost and GAMs reach the same performance as the seven-feature logistic regression, with no significant difference. The low precision is not an architecture problem — it is the signal stack's ceiling.

Alarm policy: a convenience that became a convention.

A model produces a score every hour. A score is not an alarm. The policy that converts one into the other has three parts:

• A refractory period. Once an alarm fires, further alarms are silenced for eight hours — chosen because eight hours is the length of a nursing shift at the WKZ NICU.
• A true-positive window. An alarm counts as catching the sepsis if it falls from 24 hours before to 12 hours after the blood culture — a 36-hour span.
• A multi-threshold escalation rule. A higher-risk alarm can break through the silence imposed by a lower-risk one.

The refractory period is the revealing part. Eight hours is an operational convenience — reasonable for a working ward, but not a clinically derived parameter. No one showed that eight hours is where early detection and alarm fatigue trade off best; it is where the staff rotation happens to fall.

The supplement lets us check whether these choices matter, and they do:

The proxy at the centre of everything.

Beneath both decisions is one substitution. We do not know the moment a clinician first suspected sepsis — it is not reliably recorded — so we used the blood-culture timestamp as a stand-in for "t = 0". Every claim about detecting LOS before clinical suspicion is, more precisely, a claim about detecting it before the culture was drawn.

The paper concedes this directly: because actual suspicion could arise earlier, using culture time as the proxy can overestimate early-warning performance. If a clinician was already uneasy an hour before drawing the culture, part of our measured lead time is an artefact of when the culture was recorded, not of when the model spoke first.

There is a smaller version of the same problem. A number of cultures arrived timestamped at exactly midnight — an artefact of missing times defaulting to 00:00. We imputed those to noon, closer to morning rounds, and confirmed that dropping the contaminated records did not change the results. It is a defensible fix, but it means a portion of the t = 0 anchors are reasoned estimates of what the clock read.

One definitional point cuts the other way. We labelled only culture-positive cases as sepsis; culture-negative "clinical sepsis" — babies treated as septic who never grew an organism — sits in the control group. When the model fires on one of them it is scored as a false positive, even where a clinician might have agreed. So some unknown fraction of the 96% of "false" alarms may not be false. A muddied outcome definition is muddied whichever way it leans.

A discrepancy I cannot fully explain.

While re-reading the supplement for this episode I found something I cannot reconstruct from the published PDF. The train/test baseline table reports a CRP-above-10 mg/L rate in the control group of roughly 80%, against 8.5% in the main-text table — a figure that cannot be right for a control group, and one the page appears to print twice with differing control numbers.

It is most likely a proofing fault rather than an analytical one, and I cannot say more than that from the PDF alone. I flag it here in the same spirit as the rest of the page: a critic of his own paper should not only notice the errors that flatter it. This item is carried forward as an open accuracy note rather than a resolved finding.

Simulation is not impact.

We described our longitudinal alarm analysis as the most extensive clinical impact assessment in the LOS prediction literature to date. It is worth being exact about what kind of object that is: a retrospective simulation on historical data, the model applied to records of babies whose outcomes were already settled. No clinician saw a score. No decision changed. Nothing that happened to those babies happened because of the model.

The paper's most honest move is its last. Listing what a real evaluation would require, the authors note that counterfactuals — would this sepsis have been caught later without the model? — are not measured. That is the thesis of this whole series, stated in our own paper two years before I began writing these episodes. We demonstrated prediction. We simulated impact. We did not show utility, and we said so.