1. What your time budget buys
Time yourself on twenty real units before trusting the first box. A guess there is a guess everywhere downstream.
2. How big the reliability sample has to be
3. The smallest effect this sample could detect
How these numbers are produced
- The time estimate charges full price for the first variable and 60 percent for each one after it, because the coder reads the unit once, then holds back a fifth of the total for the second reliability pass and the disagreement discussion. Both assumptions are stated so you can argue with them in your methods section.
- The reliability sample is the larger of 50 units, 10 percent of the corpus capped at 300, and whatever it takes for your rarest category to appear about ten times. The rare category is usually what drives it, and it is the part students miss.
- The detectable effect uses the normal approximation for a two-group comparison and the noncentral chi-square for the crosstab. It is a planning figure, accurate enough to tell you whether a sample is in the right order of magnitude.
- Decide all of this before you look at the data. A sample size chosen after a result is not a sampling plan.
Serves the Sampling Plan and Pilot assignment in MC 451 and MC 501, and the sampling module in MC 427.