S26 · Chapter 13 · MC 451 Research Methods in Mass Media
Dr. Alex Leith
Today’s Agenda
Choosing your test before you run it
Reporting it in one line, then in English
Effect size, every time
The White Paper, assigned today
Today’s Job
Inferencing Data, 100 points, due this week
Tuesday I ran the test on mine, today you run yours
One test, correctly chosen, correctly read, honestly reported
Then the two sentences that go in your paper
The White Paper is assigned today
Work in your .qmd, not the console
This is smaller than it has felt: one function call and two sentences.
Pick Your Test First
Write this down before you touch the keyboard:
My outcome variable is ______, continuous or categorical
My grouping variable is ______, with ______ levels
Therefore the test is ______
Continuous and two groups: t-test. Three or more: ANOVA. Two categorical: chi-square.
Running Your Test
library(v2v)analysis <-readRDS("data/analysis.rds")run_t_test(analysis, value = message_length, group = is_gaming)
Swap message_length for your outcome and is_gaming for your two-level grouping column. Everything else stays exactly as written.
Your Turn
Read your output back: the two means, the difference, p, and d
Is your effect small, medium, or large by Cohen’s benchmarks?
Did your p and your d tell the same story, or pull apart?
Report It in One Line
The compact form journals expect, using the chat study’s numbers:
Gaming channels produced shorter chat messages on average than non-gaming channels (28.49 vs. 33.70 characters), Welch’s t(3768.7) = -4.95, p < .001, Cohen’s d = -0.13.
Every element earns its place: both means, the test with its degrees of freedom, the p-value, the effect size. Never p alone.
Then Say It Again in English
A string of statistics is not an interpretation
Write a second sentence a non-statistician can read
Ours: the two kinds of channel do differ in message length
The difference is so small they are better thought of as alike
That sentence is the call, reporting without inflating
If you cannot write it, you do not yet understand your result.
Effect Size, Every Time
Statistical significance: is the effect distinguishable from zero?
Practical significance: is it large enough to matter?
Different questions, and large samples pull them apart
A finding is honestly reported only with both on the page
“Significant” without a d is a claim you have not supported.
If You Have More Than Two Groups
four_games <- analysis %>%filter(game %in%c("Fortnite", "Hearthstone","Just Chatting", "League of Legends"))summary(aov(message_length ~ game, data = four_games))
ANOVA asks one question of all the groups at once. Ours returned F(3, 13429) = 92.26, with eta-squared of 0.02: real, and small.
If You Want a Model
summary(lm(message_length ~ is_gaming, data = analysis))
The intercept is your reference group’s mean, 33.70. The coefficient, -5.22, is the gap itself, recast as a baseline plus an adjustment.
A t-test is a regression with one two-level predictor. An ANOVA is one with a many-level predictor. Same framework, different shape.
Common Errors and Meanings
must have exactly 2 levels: you have NAs, or three groups
not enough 'x' observations: your filter left almost nothing
A p of exactly 1: identical groups, usually a bad join
NA for the mean: add na.rm = TRUE, then ask why
An impossibly large t: you compared a variable to itself
Run ?v2v::common_errors for the ones specific to this data.
The White Paper, 250 Points
Assigned today, due finals week
The full study as one reproducible document, published
Executive summary, then the four standard sections
Not a term paper, but a document a client would read
Rendered from Quarto, so every number comes from your code
Published at a public URL, not submitted as a file
What Goes In It
Introduction: your question, and why it is worth asking