Research Questions

The paralysis

Where projects stall

• You have read thirty articles and taken notes in five different documents
• You see connections everywhere and you cannot start
• The project feels too small and too large at the same time
• That is not a shortage of ideas. It is a shortage of focus, and it is the normal condition at this point in a semester.
• You leave today with one question narrow enough to actually finish

Two questions

• A: “How does livestreaming affect people?”
• B: “Does the rate of chat messages per viewer differ between gaming and non-gaming streams?”

A is a career-spanning program disguised as a single question. B names what will be measured, what will be compared, and what evidence would settle it.

Getting from A to B is most of the craft in this course.

Five criteria for a question that works

• Specific: name the platform, the outcome, and the population. “How does social media influence politics?” leaves all three undefined.
• Measurable: it must survive operationalization, the move from an abstract idea to something you can actually record
  • “Do authentic streamers build better communities?” cannot start until authentic and better become observable.
• Answerable: not decades of data or hundreds of interviews
• Not already settled: your literature review is what tells you
• Worth asking: Tuesday versus Thursday stream start times is answerable and is trivia

A template for building one

Assemble the parts on purpose:

• Does there exist …
  • a specific variable or concept,
  • showing a relationship, pattern, or difference,
  • within a bounded population or context?

“Is there a relationship between stream category and chat message rate among channels in the Twitch working corpus?” is built from exactly those four parts.

The Everything Study

• It announces itself: “I want to study the effect of social media on society”
• Scope creep arrives as too many questions, too many theories, and data too large for the time you have
  • It also creeps by addition: one more variable, one more comparison, one more group
• The cure is narrowing until the project fits the time and the skill you actually have
• Narrowing is not weakness. It is the discipline that lets you finish.

The funnel

1. “I am interested in Twitch chat.” Too broad. Which aspect? Which streams?
2. “How viewers participate in chat.” Better, still vast.
3. “Whether chat participation differs across kinds of streams.” Which kinds? Measured how?
4. “Whether chat message volume differs between gaming and non-gaming streams.” Clearer.
5. “I will draw a stratified sample from the 50-channel working corpus, code each message as directed at the streamer or broadcast to the room, and test whether the proportion of directed messages differs by stream type.”

Version 5 will not answer everything about Twitch chat. It will answer something properly.

Three narrowing mistakes

• Impossible comparison: “Are viewers of a platform lonelier than non-viewers?”
  • The two groups differ in a hundred ways besides the one you care about.
• Circular question: “Do popular creators attract large audiences because people want to watch them?”
  • The terms define each other, so nothing could disconfirm it.
• False binary: “Is it the streamer or the game?”
  • Ask instead how much each contributes and whether they interact.

Concepts need operational choices

A parasocial relationship (Horton & Wohl, 1956), the one-sided bond between a viewer and a media figure, is not something you can see directly. Three routes to it:

• Self-report scale: captures subjective experience, invites social desirability bias
• Behavioral indicators: donations, subscription length, hours watched. Reliable, and ambiguous about what they mean.
• Language analysis: code what viewers write for intimacy and direct address. Naturalistic, and labor-intensive.

Sentiment, the same problem

Chat sentiment is the emotional valence of what gets typed. Same shape of choice:

• Automated scoring: fast and replicable, blind to sarcasm and to emote vocabulary
• Human coding: catches context, costs labor, and needs reliability testing
• Dimensional coding: valence and arousal at once, richer, and demands a much more careful codebook

There is no single right answer. You pick, and then you defend the proxy in writing.

Three goals, three kinds of question

• Exploratory: “What is going on here?” Generates hypotheses.
  • Output is patterns and themes.
• Descriptive: “What does the landscape look like?”
  • Output is frequencies and distributions.
• Explanatory: “Why does this happen?” Tests relationships.
  • Output is support or disconfirmation.
• Research on a topic tends to move through these in order. You do not have to do all three, and you do have to know which one you are attempting.

Question or hypothesis?

• A research question is interrogative and open: use it when exploring, when describing, or when the literature contradicts itself
• A hypothesis is declarative and predictive: use it when theory makes a specific prediction you could be wrong about
• Behind every hypothesis sits the null hypothesis: no relationship, no difference
  • The null is the skeptical default, and it is what a statistical test actually evaluates
• Same logic as the sacred flaw from Chapter 1

Topic, theory, data must align

• Topic: bounded by population, time, and medium
  • “Livestreaming” is not a topic. “Chat participation in the 50-channel corpus” is.
• Theory: must predict patterns your data could actually show
• Data: accessible, sufficient, and appropriate to the question

A worked mismatch: a parasocial topic, political economy theory, and chat content analysis. Political economy explains industry structure while chat messages are individual expression, so the three never meet. Swap in parasocial interaction and they start talking.

Before next time

• The Annotated Manuscript and CITI Certification are due this week.
• Topic Selection and Research Questions is due next week. Put your question through the funnel before you submit it.
• Run your draft past the five criteria and name the one it is weakest on
• Read Chapter 6 on the prospectus, especially the model prospectus and the weak-draft example
• Next session we assemble the six components of the prospectus, and talk about pre-registration