This Week In Media Measurement

Transcript 48 lines

Cold Open Stats Overview Paper Walkthrough free_promo

Cold Open

Jenny When you say something online “messed with your head,” do you mean the content, the app, or just the way you were using it?

Davis I mean, I say “the app,” but half the time it’s me doomscrolling at 1 a.m., and that’s not the same thing as a video being persuasive.

Jenny Right, and we talk like it’s one blob called “social media use,” but that’s basically a measurement choice—are we counting minutes, clicks, or the weird feeling after you close it?

Davis And that choice quietly decides the whole debate, because “two hours a day” could be messaging your cousin or getting rage-baited, and those shouldn’t land the same in your brain.

Jenny So if a new tool can split “use” into seven different behaviors and experiences instead of just time spent, that’s not nerd trivia, that’s the difference between blaming the internet and fixing the habit...welcome to This Week In Media Measurement on paperboy.fm.

Stats Overview

Davis Quick map of the week: we analyzed about 1,400 hits, and 95 papers made the cut. That’s about 255 unique authors across 27 countries, so it’s still global, just a tighter slice than usual.

Jenny And “made the cut” matters here, because qualified papers dropped to 95 from 137 last episode, down 42, or about 31%. Do we know if that’s a quality shift, or did the feed skew toward formats we screen out, like thin surveys or duplicate conference write-ups?

Davis The top methods hint at what you’re seeing: surveys lead at 39, then broadly “quantitative” at 20, and “qualitative” at 17, with case studies at 9. If the stream leaned even more survey-heavy, you can get lots of papers that look like media effects but don’t really measure exposure or outcomes cleanly, which fits our through-line about measurement driving the headline.

Jenny Total query hits also fell, to 1,375 from 1,751, down 376, about 21.5%. Is that fewer things being published in our date window, or are we just seeing a topic drift away from our query terms—like “social media” staying dominant at 21 while everything else fragments into digital media and consumer behavior at 4 each?

Davis And the author pool shrank even more: about 255 authors this week versus 391 last time, down 136, roughly 35%, with countries sliding to 27 from 35. That reads like less geographic spread, and maybe more clustering in a few places—Indonesia alone is 10, India is 5, China is 3—which can change what “media effects” even means in practice.

Jenny One more texture point: the author tiers skew early-career—about 37% first-time authors, meaning their first-ever paper, plus about 40% emerging, and only about 22% experienced. That’s exciting, but it also raises the question: are we watching new researchers reinvent measures, or are they inheriting shaky ones—especially with themes like social media, digital media, and consumer behavior driving the week?

Paper Walkthrough

Paper 1 The Comprehensive Assessment of Social Media Use: Development and Validation Study

Jenny Alright, let’s get into the papers, and I wanna start with a measurement one called The Comprehensive Assessment of Social Media Use: Development and Validation Study.

Jenny It’s in JMIR Formative Research in twenty twenty-six, and the whole pitch is that “time spent” is a blunt instrument if you’re trying to link social media to mental health.

Jenny So they build a new survey, the CASM, that tries to capture what people are actually doing and feeling online, not just how many hours they rack up.

Jenny Plain version first: they end up with a twenty-nine item checklist that covers seven different flavors of engagement, including both positive and negative stuff, and they test it in two online samples of college-aged young adults.

Jenny Study one had two hundred sixty people with a mean age of about nineteen point seven, and study two had five hundred eight people with a mean age just under nineteen.

Jenny They use factor analysis—basically a statistical sorting hat that groups survey questions that move together—to land on those seven subscales, and the final model explains about sixty-one percent of the variance in responses.

Jenny And the reliability, meaning how consistently the items hang together inside each subscale, runs from about zero point six nine nine up to zero point eight one seven.

Davis Okay, but if this is self-report, what convinces you it measures real behavior and not just people’s self-image, like “I’m the kind of person who uses social media well”?

Jenny That’s the core risk, and the authors try to earn it by doing this in two phases: first they generate a big pool of items and run exploratory factor analysis in the two hundred sixty person sample, then they lock the structure and run confirmatory factor analysis in the five hundred eight person sample to see if it holds up.

Jenny Then they do validity testing, which in plain terms is checking whether the CASM scores line up with other measures in the directions you’d predict, instead of just being noise.

Jenny But yeah, biggest limitation is right in the design: it’s a convenience sample of college-aged young adults online, so we should be careful about claiming this captures how, say, a fourteen-year-old or a working parent uses TikTok.

Davis I like this as a move in the “measuring social media use” thread, because it stops us from pretending “two hours” means the same thing across people.

Davis If your seven buckets include both positive and negative engagement, then a study that only counts minutes could miss the whole story, even if the sample here is still kind of narrow and the evidence is solid-but-not-final.

Davis And honestly, for anyone designing an intervention, this is the difference between telling people “log off” and telling them “stop doomscrolling at midnight, but keep the group chat that actually makes you feel connected.”

Paper 2 Use of Instagram and its effect on the mental well-being of university students: A perspective from Pakistan

Davis Speaking of how “two hours” can hide the whole story, I brought a paper that zooms in on one app and one pathway: “Use of Instagram and its effect on the mental well-being of university students: A perspective from Pakistan.”

Davis It’s a survey of five hundred fifteen university students, ages eighteen to twenty-five, from two universities in Islamabad, and they’re basically asking: when Instagram use goes up, what happens to self-esteem and depression, and does social comparison change the chain.

Davis Plain version first: heavier Instagram use lines up with lower self-esteem, and lower self-esteem lines up with more depression symptoms.

Davis In their model, Instagram use strongly predicts decreased self-esteem, with a beta of minus zero point six six one, and self-esteem predicts lower depression with a beta of minus zero point four three nine, both with p-values under point zero zero one.

Jenny Okay but how do we know this isn’t just that depressed students use Instagram differently, like they scroll more or interpret posts more harshly, and that’s what’s driving the link?

Davis We don’t know clean causality here, because it’s a one-time online survey with convenience sampling from two Islamabad campuses, so it’s strong association, not “Instagram caused depression.”

Davis What they do show is a mediation story—mediation meaning “the effect travels through a middle step”—where the direct Instagram-to-depression link goes non-significant once self-esteem is included, and the indirect path via self-esteem is significant at about zero point two nine.

Davis Then they add moderated mediation—basically “that middle-step pathway changes depending on who you are”—and upward comparison matters, with a moderated mediation index of minus zero point zero three five, p equals point zero one six.

Jenny That comparison twist is the part I’d actually use, because it says the intervention target isn’t “stop using Instagram,” it’s “stop using it in a way that turns your brain into a ranking machine.”

Jenny And it fits our measurement thread: if you only tracked minutes, you’d miss that the same feed time can mean totally different self-esteem math for different people, even if this sample and design can’t fully settle direction of cause.

Paper 3 Negotiating at a distance: the impact of communication media and negotiator traits

Jenny That whole “minutes don’t equal impact” thing you just said about Instagram—same time, totally different head-math—made me think of this negotiation paper called Negotiating at a distance: the impact of communication media and negotiator traits.

Jenny They took four hundred people, paired them into two hundred negotiating dyads, and made them hash out a mixed-motive relational conflict over four channels: face-to-face, video, audio, or synchronous text messaging—so basically live texting back and forth.

Jenny Plain version: live text made people worse at negotiating than talking, whether that talking was in person, on video, or just voice.

Jenny And the twist is it’s not only the medium; negotiator traits mattered too—things like conflict management style and personality, meaning stable tendencies like extraversion or emotional stability, changed which channel worked best for which pairs.

Davis When they say “poorer outcomes,” what exactly tanked—money, trust, satisfaction—and did those measures disagree depending on the medium?

Jenny They split it into economic outcomes at the dyad level—joint value creation, basically how big the pie got for the pair—and non-economic outcomes at the individual level, like trust, and they analyzed those with one-way and two-way ANOVAs for the money side and linear mixed regression for the individual side, which is just a model that handles paired data without pretending everyone’s independent.

Jenny Text came out worse overall, but face-to-face, video, and audio were mostly similar, with two specific differences: higher trust in face-to-face than video and audio, and higher value creation in audio than face-to-face.

Jenny Then they show moderation—meaning “it depends who you are”—where conflict style, indirect communication style, and traits like extraversion, conscientiousness, agreeableness, and emotional stability shift outcomes; and one concrete example is dyads where both people were high-assertive actually created lower joint value face-to-face than in audio or video.

Jenny Big limitation they admit: you’ve got two hundred dyads, which is decent, but more dyads would give them more statistical power to nail the trait-by-medium effects on the economic outcomes.

Davis This is such a clean “format matters” paper, because it’s not vibes—it’s literally: don’t do the high-stakes part over live text if you care about outcomes.

Davis And I love the practical split: if the goal is trust, face-to-face still wins; if the goal is value creation, audio beating face-to-face is wild, like maybe voice strips out some of the status theater.

Davis I’m also hearing “moderately strong, not definitive”—two hundred dyads is real, but if you’re building a workplace policy, you’d want a replication in actual organizations before you ban Slack negotiations forever.

This Week In Media Measurement

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 The Comprehensive Assessment of Social Media Use: Development and Validation Study

Paper 2 Use of Instagram and its effect on the mental well-being of university students: A perspective from Pakistan

Paper 3 Negotiating at a distance: the impact of communication media and negotiator traits

free_promo

Other Episodes

This Week In Media Measurement

Episode

Cold Open

Stats Overview

Paper Walkthrough

Paper 1 The Comprehensive Assessment of Social Media Use: Development and Validation Study

Paper 2 Use of Instagram and its effect on the mental well-being of university students: A perspective from Pakistan

Paper 3 Negotiating at a distance: the impact of communication media and negotiator traits

free_promo

Other Episodes

Get This Week In Media Measurement by email