Asking one question at a time

One of the more interesting pearls of wisdom from my statistics class is that one of the basic ideas of scientific experimentation, that of setting up a control and changing only one variable at a time and repeating many times, is wasteful of runs. Through proper factorial designs, you can more efficiently examine main and interaction effects from the same runs, and each effect can use all of your data and so not sacrifice standard error.

Is this controlling one variable at a time one of those little lies that even educated people keep propagating?


4 thoughts on “Asking one question at a time

  1. Wouldn’t changing one variable at a time make sense? For example, wouldn’t measuring the effect of mass of an object on the drag by itself be better than measuring the effect of mass, shape, and material of an object on the drag together? It wouldn’t be clear in the second case which one had the greater effect, right?

    • So thought experiment: say you had resources for 24 runs. What you suggest would be along the lines of: use 8 runs to examine the effect of mass, with shape and material held constant; similarly 8 runs for shape, 8 for material. Firstly, you only capture the effect of mass in the context of one shape/material condition. Secondly, you have no way of examining the interaction of any of the 3 factors. Finally, you’re only using 8 runs to calculate the standard error of each main effect.

      In contrast, a 2^3 factorial design would run all 8 combinations of +/- mass (the extremes of the mass parameter), +/- shape, +/- material, and do 3 replications of each. Under a linear model, this can examine the main effects, any interactions, and use all 24 runs to calculate standard error for each. Much more efficient, much more likely to detect significant factors.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s