This Week In Wellbeing Measurement

Transcript 29 lines

Cold Open Stats Overview Paper Walkthrough free_promo

Cold Open

Jenny When does keeping track of yourself stop being helpful and start messing with your head?

Davis I think it's when the number stops being a map and starts acting like a tiny judge, though I'll defend a good number if it helps you look away at the right time.

Jenny See, I like a good number too, but I distrust that little buzz on my wrist that says my body failed a test I didn't know I was taking.

Davis And there's now a name for that trap, orthometria, meaning digital health metrics that create anxiety or make wellbeing and performance worse, so today we're asking what our measures really measure...welcome to This Week In Wellbeing Measurement on paperboy.fm.

Stats Overview

Davis This week we screened seven hundred seventy-five hits and ended up with eighty-six qualified papers, from about four hundred sixty authors across thirty-two countries. So the funnel is wide, but the final set is fairly tight.

Jenny And that final set is smaller than last week: down from ninety-eight to eighty-six, a twelve point two percent drop. The methods give one clue, not a proof: nineteen surveys, nineteen qualitative studies, and twelve quantitative papers, so a lot of work is close to lived experience, but may not clear a strict measurement-and-metrics bar.

Davis The weird part is the search got bigger while the keeper pile got smaller. Query hits rose from seven hundred six to seven hundred seventy-five, up nearly ten percent, which makes me ask whether broad themes like mental health, quality of life, and depression are pulling in more papers than the show can honestly call measurement work.

Jenny Country coverage moved the other way: twenty-six countries last week, thirty-two this week, up about twenty-three percent. The top contributors are still concentrated, with the UK at twelve papers, the US and China at six each, and Germany at five, and we can't say much below country because city and institution metadata are both zero.

Davis The author mix also matters for how stable these metrics feel. Of four hundred sixty-two authors, eighty-five are first-time authors, meaning their first-ever paper in this metadata, one hundred eighty-two are emerging, and one hundred ninety-five are experienced.

Jenny So the snapshot is not less activity. It's more searchable activity, spread across more countries, with fewer papers making the cut, which fits the episode's through-line: wellbeing metrics keep claiming to measure lived reality, and the first test is whether our own filters can tell that apart from just more wellbeing language.

Paper Walkthrough

Paper 1 Comparative performance of the EQ-5D-5L, EQ-HWB, and PROMIS-10 in screening for history of anxiety and depression in the general adult population in 15 countries

Jenny Alright, let's get into the papers, and the first one asks a very practical measurement question: can a generic health survey also flag mental health risk. It's called Comparative performance of the EQ-5D-5L, EQ-HWB, and PROMIS-10 in screening for history of anxiety and depression in the general adult population in fifteen countries, by Hilary Short and colleagues in Quality of Life Research.

Jenny They had sixty-eight thousand four hundred nineteen adults across Argentina, Australia, Brazil, Canada, China, the United States, and nine other countries. The plain finding is that the EQ-5D-5L did best as an extra anxiety-and-depression signal, and its anxiety or depression item hit AUROC values from zero point seven six to zero point eight eight in every country except China; AUROC just means how well a score separates people with and without the condition, where zero point five is a coin flip and one point zero is perfect.

Davis If the benchmark is self-reported diagnosis history, how confident should we be that this is actually screening current anxiety or depression, rather than remembering who once got a label from a clinician.

Jenny That's the right pressure point. They used cross-sectional EQ-DAPHNIE survey data, with planned samples of about four thousand five hundred per country and quotas for age, sex, income, and urban or rural area, then compared EQ-5D-5L, EQ-HWB, PROMIS-10, PHQ-2, and GAD-2 items against people's own reports of an anxiety or depression diagnosis. So it's big and international, but it's not a clinical interview, and China is the caution flag because performance dropped to about zero point six five to zero point seven five.

Davis For a health system, that makes the takeaway useful but modest: if you're already collecting EQ-5D-5L as a patient-reported outcome measure, meaning patients report how their own health feels and functions, the anxiety and depression item can be a smoke alarm, not the fire inspector. And it fits this patient-centred measurement thread, because the value is catching something people experience before the administrative data notices it.

Paper 2 Quality Assessment of Public Health Datasets and Their Alignment With the United States Core Data for Interoperability (USCDI) Data Elements.

Davis That smoke alarm point carries over, because the next paper asks whether the wiring behind the alarm even works: Quality Assessment of Public Health Datasets and Their Alignment With the United States Core Data for Interoperability Data Elements.

Davis Chehab, Jaafa, and colleagues looked at five national public health datasets in twenty twenty-four and twenty twenty-five, across social, economic, education, physical, and health services domains. They judged them on six basics: completeness, uniqueness, accuracy, timeliness, consistency, and conformity, meaning whether the fields are filled, non-duplicated, correct, current, stable, and in the expected format.

Davis The headline is a little sneaky: for a subset of elements mapped to USCDI, the United States Core Data for Interoperability, which is basically a shared menu of data fields that systems agree to exchange, completeness and uniqueness hit one hundred percent across most datasets. But accuracy, timeliness, and conformity couldn't be checked the same way across datasets, so the neat-looking scorecard had blank spots in the places policymakers may care about most.

Jenny If accuracy and timeliness couldn't be uniformly assessed, what does a one hundred percent completeness score actually buy us?

Davis It buys a narrower kind of confidence: the field is present, and the record isn't obviously duplicated. The team was an interdisciplinary public health informatics group, and this was a qualitative case-study audit of five datasets, plus a publicly accessible toolkit for teams trying to map their own data to USCDI. The big limitation is that some of the most policy-relevant checks were impossible because the datasets didn't have comparable gold-standard reference points.

Jenny So this is useful, but modest: it's a plumbing inspection, not proof the water is clean. In the Data before decisions thread, that's exactly the point, because before we link housing, services, education, and health data into decision systems, we need to know which quality claims we can actually test, not just which boxes are filled.

Paper 3 Retirement, Economic Vulnerability, and Subjective Wellbeing: A 13-Year Panel Study from the UK

Jenny So after that plumbing inspection, here's a paper that asks whether one of our big life-course labels actually lines up with how people feel: Retirement, Economic Vulnerability, and Subjective Wellbeing: A 13-Year Panel Study from the UK.

Jenny Jie Liu, Frank Agyemang Karikari, Seth Acquah Boateng, and Alexander Opoku use thirteen waves of the UK Household Longitudinal Study, from two thousand nine to two thousand twenty-two, covering twenty-six thousand one hundred eighteen adults aged fifty to seventy-five and one hundred eighty thousand eight hundred forty-eight person-years.

Jenny The surprise is distributional: retirement was linked with higher life satisfaction and lower psychological distress, and people who were economically vulnerable, meaning lower income, no private pension, or poor health, did not benefit less.

Davis How much of this looks like retirement helping, and how much looks like leaving a bad job helping?

Jenny That's exactly where their method matters: they use fixed-effects regression, which basically compares each person with themselves before and after retirement, so stable differences between people are less likely to explain the change.

Jenny And their interpretation leans toward the bad-job story, because the biggest gains showed up for workers with fewer resources, which fits the idea that leaving high-strain, low-reward work can be a wellbeing boost; the catch is that this study tracks the transition well, but it doesn't tell us when retirement should happen.

Davis That's a pretty concrete warning for longer-working-life debates: if a policy saves money by keeping people in work longer, the wellbeing ledger has to include distress, especially for the people who started with the least cushion.

This Week In Wellbeing Measurement

When measures become the problem

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 Comparative performance of the EQ-5D-5L, EQ-HWB, and PROMIS-10 in screening for history of anxiety and depression in the general adult population in 15 countries

Paper 2 Quality Assessment of Public Health Datasets and Their Alignment With the United States Core Data for Interoperability (USCDI) Data Elements.

Paper 3 Retirement, Economic Vulnerability, and Subjective Wellbeing: A 13-Year Panel Study from the UK

free_promo

Other Episodes

This Week In Wellbeing Measurement

When measures become the problem

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 Comparative performance of the EQ-5D-5L, EQ-HWB, and PROMIS-10 in screening for history of anxiety and depression in the general adult population in 15 countries

Paper 2 Quality Assessment of Public Health Datasets and Their Alignment With the United States Core Data for Interoperability (USCDI) Data Elements.

Paper 3 Retirement, Economic Vulnerability, and Subjective Wellbeing: A 13-Year Panel Study from the UK

free_promo

Other Episodes

Get This Week In Wellbeing Measurement by email