Reading the Data Against the Codebook

Today’s Agenda

• Can this data carry the study you designed?
• The messages that will fight your rules
• One wrinkle, turned into a decision rule
• The qualitative memo, due with the codebook

What We Are Doing Today

Lab session

• Tuesday we loaded the data, today we hold it next to the codebook
• One question: can the data carry your study?
• Then the messages that will fight your rules
• Then the qualitative memo, due with the codebook

Open codebook.qmd and your R session side by side.

The Blunt Question

• Does the data contain what each variable needs?
  • Open the codebook next to your glimpse() output
  • Answer it now, not a thousand messages in
• Walk the variables one at a time, and do not skim

A missing column found today is a good result. Found in three weeks it costs you weeks.

Variable 1, Message Target

chat %>% select(channel, sender, message) %>% head(5)

select() keeps only the columns you name, and head(5) shows the first five rows, so you read what a coder would actually be looking at.

Message target is coded from the message text, plus sender for at-mentions and replies. Both columns exist. Supported.

Variables 2 and 3

chat %>% select(message) %>% head(3)

The same move, narrowed to the one column both remaining variables read from.

• Message length: the character count, so supported
• Contains emote: the same column, so supported
• Three variables, three confirmations, and the study is buildable

What First Contact Confirmed

• The columns your variables need are present
• The messiness the book warned about is visible in row one
  • Your Chapter 7 edge cases are not hypothetical
  • Twenty-three question marks: no obvious target
  • “TriEasy Clap” repeated: copypasta, already Rule 3

What the Data Makes Harder

• Short emote-only tokens are everywhere, which is Rule 2
• A wall of punctuation has no addressee, hence “unclassifiable”
• Stream titles carry line breaks and long digit strings
• None of this is broken data, it is what people type

Your codebook has to meet the data as it is.

One Genuine Wrinkle

• A message begins “@xQcOW”
  • The channel column on that row reads “xqcow”
• Same name, different casing
  • The display name typed against the stored login name
• A rule keyed on the channel name will miss it
• Two coders resolve this differently unless you decide

Turning a Wrinkle Into a Rule

• Write it down now, while the codebook is still living
  • At-mentions are matched ignoring capitalization
• State it in the decision rules, numbered
  • Next to Rules 1 through 5
• This is what special cases is for, and yours stops being thin

Every rule you add is a disagreement you will not arbitrate later.

Hunting Your Own Edge Cases

chat %>%
  filter(str_detect(message, "@")) %>%
  select(channel, sender, message) %>%
  head(10)

filter() keeps only rows meeting a condition, here messages containing an at-sign, so you can read ten real at-mentions and see how they behave.

Swap the "@" for anything your rules depend on and read what comes back.

The Qualitative Memo

Due with the codebook

• 200 to 300 words on the patterns you observed
  • The written record of what your watching taught you
  • It explains why your codebook has these categories
• Graded with the codebook as one 50-point assignment

Short does not mean easy. Every sentence should point at something you saw.

The Four Kinds of Field Note

Your memo is assembled from notes you already have:

• Observational: what you saw, with times and counts
• Methodological: a measurement problem you noticed
• Theoretical: an observation tied to a framework
• Comparative: how two cases differed

From Notes to Memo

• Name the dimensions that mattered for your question
• Name the edge cases that drove your rules, with examples
• Say where you are still uncertain
  • Uncertainty is a finding, not a weakness
• Connect at least one pattern to your theory

Do not summarize Twitch. Write what changed your codebook.

Common Errors Today

• could not find function "str_detect": no library(tidyverse)
• object 'chat' not found: the data is not loaded here
• object 'message' not found: a column name outside a verb
• filter() returning zero rows: usually the condition, not the data

Run ?v2v::common_errors for the ones specific to this data.

Checkpoint

You should now have:

• Written confirmation that each variable has the data it needs
• At least one new decision rule, traceable to a real message
• A special-cases section that is no longer empty
• Field notes sorted into the four types
• A draft memo between 200 and 300 words

Before Next Time

• Due this week: Codebook and Qual Memo
  • Render the codebook before you submit it
• Read Chapter 10, The Sample
• Next Tuesday: why you cannot code all 35,267 messages
  • Then your codebook goes on trial, two coders, one number

Questions?