Do Clinical Frailty Scores (CFS) measured in A&E tell us anything useful?

Tiny answer: Yes.
1/7

Thank you to to the patients, clinical teams & data collectors (esp. the ED nurses!) my co-authors including @jthteo.bsky.social and @krockdoc.bsky.social and others not on this platform

What can we learn from 68 000 clinical frailty scale scores? Evaluating the utility of frailty assessment in emergency departments AbstractBackground. Emergency departments (EDs) in England are under significant strain, with increasing attendances and extended wait times, affecting fra

Long answer? Please do read the full paper, it's open access in @age-and-ageing.bsky.social

academic.oup.com/ageing/artic...

7/7

In summary: CFS strongly predicts outcomes. Recognising the variability we observed (worth checking your own ED data?) is key to refining its application. We think automated tools could offer useful support here.

6/7

It also seemed why you were in A&E played a role. Arrive confused or acutely unwell? You might be rated frailer than someone with a less alarming issue. Understandable, perhaps, but it does suggest the score might be reflecting more than just baseline frailty.
5/7

CFS scores over time for the 4 patients of 68 067 with the most frequent ED attendances. Each panel represents an individual patient. The x-axis shows years since their initial admission. The y-axis indicates the CFS score (range 1–9). Data points represent CFS scores recorded at each ED visit. This figure illustrates significant intra-patient variability in CFS scores over relatively short time periods. For example, the second panel in the first row (blue dots) shows a patient scoring first 1(‘Very Fit’) and 8 (‘Very severely frail’) within ~3 months, and then surviving for at least 2 years with scores as low as 3 (‘Managing well’). This variability seems at odds with a CFS score that reflects a relatively stable measure of baseline frailty, highlighting potential inconsistencies in CFS assessment.

The 'But'... reliability; that's where it gets interesting. We saw scores for the same patient varying rather dramatically between visits, from 'Fit' to 'Severely Frail' to 'Fit' again. These results don't fit with our understanding of frailty.
4/7

aplan–Meier survival curves illustrating survival probability over a 90-day period following ED attendance for various CFS scores. Each curve represents a different CFS score from 1.5 (very fit/fit) to 9 (terminally ill). The graph highlights a distinct trend in which the survival probability decreases as frailty increases.

The 'Yes' part: Those CFS scores do correlate strongly with important things - hospital admission, length of stay, short, medium and long-term survival. They're clearly picking up on something significant for patients.

3/7

Short answer: Yes, but... it's complicated:

While CFS screening became wider NHS guidance in 2019,
@kingscollegenhs.bsky.social EDs started way back in 2017 giving us a over 68,000 scores to analyse; the largest cohort of CFS scores in the literature:

2/7

Do Clinical Frailty Scores (CFS) measured in A&E tell us anything useful?

Tiny answer: Yes.
1/7

La paradoja de la alerta temprana en el entrenamiento de los modelos de aprendizaje automático. Imprescindible.
www.nature.com/articles/s41...
Básicamente las intervenciones con éxito pueden provocar que un factor de riesgo importante parezca poco predictivo en los datos retrospectivos
#IA

Why do some AI models struggle in real-world healthcare? A new study explores the early warning paradox—where existing alert systems can mask the true impact of machine learning predictions. Systems-level approaches to #AI evaluation are key! 🤖⚕️ nature.com

www.nature.com/articles/s41...

Hugh has been talking about this issue for a long time. It’s very gratifying to see it in print. I’m honoured to have contributed. Please do have a read and let’s discuss more about this issue.

And finally - anonymous Reviewer 2, whoever you are - thank you (yes really!). Your thoughtful feedback and the Associate Editor's guidance significantly strengthened this paper. When peer review works, it really works.

I wouldn't have written this up without Zina Ibrahim encouraging me to. @doced.bsky.social was immense, helping frame it the language of causal inference language. @jthteo.mstdn.social.ap.brid.gy and @krockdoc.bsky.social offered masterful supervision, and bulked out the COIs, and MBW made me think!

What's the solution? How do we build AI systems that can spot these cases when successful treatment makes them invisible in our data? You'll need to read the paper for that...

rdcu.be/d8r6N

And these are exactly the patients we want to catch. The ones where early treatment makes all the difference. Where you can turn someone from gripping the rails gasping for breath to sipping tea within an hour.

The "problem"? When we treat these patients successfully - GTN, oxygen, furosemide - they might not die or go to ICU. Why is this a problem? Because if we train an EWS to detect people who might die or go to ICU, they won't learn from this case.

Your bleep goes: Bed 8's observations are concerning. You find them bolt upright, gripping the rails, wheezing with each desperate gasp. BP up, sats down. Classic APO - treat it now and they'll live, wait too long and they'll die. Yet AI systems might miss these cases. Our new paper explains why...

Complete 12 parkruns. You'll probably be able to see improvement with each run, and this works as powerful motivation for me at least.