This Week In Media Measurement

Transcript 27 lines

Cold Open Stats Overview Paper Walkthrough free_promo

Cold Open

Jenny When you like or share something, does that actually mean you are any closer to buying?

Davis I want to say no, because my thumb has endorsed plenty of shoes I never bought, but attention has to become action somewhere.

Jenny Right, and I'm wary of calling every like a lead, because a lead should mean a person moved closer to spending, not just gave a platform one more tiny signal.

Davis But if the bridge is what fans actually do inside the platform, and one study says 55 percent of the link between digital interaction and spending runs through that participation, then the metric is not the sale, it's the road to the sale; welcome to This Week In Media Measurement on paperboy.fm.

Stats Overview

Davis This week we analyzed 2,382 hits and qualified 113 papers, from 382 unique authors across 29 countries. So the feed is still broad, but the usable set is tighter.

Jenny And that qualified pile fell by 14 papers, an 11.0 percent drop from last episode. Query hits only slipped from 2,504 to 2,382, down 122, so I don't read this as fewer papers existing. I read it as fewer papers matching the harder question: did the media metric prove delivery, behavior, or accountability?

Davis The odd part is authors went the other way, from 372 to 382. That's about ten more people in a slightly smaller haystack. The country count narrowed from 35 to 29, with Indonesia at 9 papers, China at 7, and India at 5, so the week looks more concentrated by geography but a little more distributed by byline.

Jenny Author mix matters here too: 107 authors were first-time, meaning first-ever paper in the metadata, not just new to us. Then 135 were emerging, and 140 were experienced. That's basically a three-way split, which makes me ask whether these measurement debates are being pushed by newer scholars testing platform data, or by established teams validating the old metrics.

Davis Topic-wise, social media dominated with 36 papers. After that it drops fast: digital marketing had 6, mental health had 5, and social media marketing had 5. That fits the through-line, because these are exactly the spaces where a like, a view, or an impression can look convenient while still not proving a person changed anything.

Jenny Methods explain part of the feel: surveys led with 39 papers, quantitative studies had 31, and qualitative work had 23. A survey is useful because it asks people directly what they saw or did, but it's still self-report, so the accountability bar is different from a behavioral trace, an audit, or an experiment.

Paper Walkthrough

Paper 1 Open Platform Field Experiments: Expanding the Design Space of Experimental Research on Social Media

Jenny Alright, let's get into the papers with paper one, Open Platform Field Experiments, a 2026 method paper by Jordi Guillem Condom-Tibau, Giovanni Puccetti, Clara Bacciu, Matteo Abrate, and Stefano Cresci about how researchers could test social media effects inside live platforms instead of relying on whatever dashboard the platform chooses to show them.

Jenny The plain idea is that if a platform is modular enough, outsiders can change one working piece and watch what happens in real use; the authors call these OPFEs, or Open Platform Field Experiments, meaning independent experiments that intervene directly on live components like clients, recommendation systems, and moderation services.

Davis So what would actually have to be open before outsiders could test cause and effect, not just describe another feed or scrape another pile of posts?

Jenny Their answer is pretty concrete: they map the design onto Bluesky and the AT Protocol, where the architecture can separate the app client, the recommendation layer, and moderation services, then they compare that setup with older workarounds like surveys, simulations, browser overlays, and platform partnerships.

Jenny The catch is important, though: this is mainly a design framework with practical examples, not a completed field experiment with outcome results, so the support is strong for feasibility and weaker if you're asking, what effect size did this intervention produce.

Davis That lands squarely in the bias-in-the-pipe theme, because the measurement pipe changes when researchers can inspect and alter the plumbing, and the practical takeaway is hopeful but narrow: open, governed platform components could make independent causal measurement real, not just convenient.

Paper 2 Key digital advertising measurement challenges in Kazakhstan

Davis That word plumbing from Bluesky is useful, because Bekenova and Kazybayeva are asking what happens when the plumbing is commercial and local. Their paper is Key digital advertising measurement challenges in Kazakhstan, and it treats Kazakhstan's ad dashboards as only one piece of the measurement environment.

Davis Plain version: more platform data doesn't mean better proof that an ad worked. In their model, technological conditions ranked first at zero point three eight five four, social conditions were close behind at zero point three three nine nine, economic came next at zero point one eight one three, and political-legal looked small at zero point zero nine three four.

Jenny If the dashboards are sophisticated, how do we know the metrics are measuring the right thing, like sales lift, trust, reach among real people, or just clicks from whatever the platform can easily count?

Davis They don't run a campaign test; they map the environment. They use documentary analysis, then PESTEL, which is a checklist for political, economic, social, technological, environmental, and legal forces, plus AHP, a way to weight criteria through paired judgments, and DEMATEL, a way to see which factors push on the others. The twist is that political-legal conditions ranked lowest, but DEMATEL still marks them as net drivers, so law and governance can steer the system even when they don't look like the top day-to-day problem.

Jenny That feels like the cleanest warning label for the beyond vanity metrics thread: before you optimize a buy to impressions, clicks, or completion rate, say what outcome that number is supposed to stand for, and say where the boundary is where you need extra evidence. Otherwise the dashboard looks precise, but the claim is doing all the heavy lifting.

Paper 3 From digital interaction to fan spending through platform-based participation in the Saudi Pro League's fan ecosystem

Jenny That warning label about dashboards doing the heavy lifting is exactly where this next paper lands, but with money attached. It's called From digital interaction to fan spending, and Abdolmaleki, Glebova, and Baker look at four Saudi Pro League clubs from twenty twenty-three to twenty twenty-five to ask whether online engagement turns into actual fan spending.

Jenny Their answer is not, likes magically become revenue. The missing middle step is platform-based participation, meaning fans doing structured things on club platforms, like paid interactions, voting, and membership-tier activities, and that participation mediated fifty-five percent of the relationship between digital interaction and spending.

Davis So are likes valuable on their own, or only when they move people into something deeper, like a membership vote or a paid club feature?

Jenny The authors treat likes and similar behaviors as an input, not the prize. They use longitudinal panel data, which means repeated club-level observations over time, then fixed-effects regression, which compares a club against itself to reduce stable club-to-club differences, plus path analysis, which estimates the indirect route from interaction to participation to spending. The big limitation is that the data are aggregated at the club level, so they can't follow one individual fan from click to purchase.

Davis That's a useful upgrade to the beyond vanity metrics thread. If you're a club, the practical takeaway isn't just post better content; it's build the next rung after attention, because voting, paid interactions, and tiered membership give engagement somewhere measurable to go.

This Week In Media Measurement

Media metrics meet messy reality

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 Open Platform Field Experiments: Expanding the Design Space of Experimental Research on Social Media

Paper 2 Key digital advertising measurement challenges in Kazakhstan

Paper 3 From digital interaction to fan spending through platform-based participation in the Saudi Pro League's fan ecosystem

free_promo

Other Episodes

This Week In Media Measurement

Media metrics meet messy reality

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 Open Platform Field Experiments: Expanding the Design Space of Experimental Research on Social Media

Paper 2 Key digital advertising measurement challenges in Kazakhstan

Paper 3 From digital interaction to fan spending through platform-based participation in the Saudi Pro League's fan ecosystem

free_promo

Other Episodes

Get This Week In Media Measurement by email