Bakeman Roger.Robinson Byron F. - Understanding Statistics in the Behavioral Sciences

The 16 scores for the button-pushing study have been used to illustrate a number of different analyses throughout this book. In , we showed how to analyze these same data, first assuming that instruction set was a within-subjects factor (one-between, one-within), then assuming that both gender (of spouse) and instruction set were within-subjects factors (two-within).

In this section, a somewhat different set of assumptions is made but again the same scores are used as an example. Assume that eight people have been randomly selected to serve as subjects and four of them have been randomly assigned to a training group whereas the second four are assigned to a no-training control group. Thus treatment (training vs. no training) for the present example, like gender in the first two of the previous three examples, is a between-subjects factor or variable. Assume further that subjects are exposed to videotapes and are asked to push a button whenever the infants portrayed in the tapes do something communicative. The number of button pushes constitutes the pretest or the pretreatment assessment. Next, the four treatment-group subjects receive training concerning common infant cues. Finally, all subjects are again asked to push a button whenever infants do something communicative. The number of times the button is pushed this time is the posttest or posttreatment assessment.

As with all such studies involving a pretest, a posttest, and different treatments given to different subjects, the research question is: Does the treatment matter? Are the treated subjects scores, on average, different from the untreated subjects scores? There are two ways we might seek to answer this question, given the situation described in the preceding paragraph, but before we do that it is useful to discuss an even simpler situation. Imagine that we did not have a control groupthat all subjects were first given a pretest, then a treatment, and finally a posttest. Such a single-group procedure is not recommended as a way to demonstrate unambiguously that a particular treatment has a desired effect. Subjects might score higher on the posttest, for example, simply because of some effect of elapsed time, and not because of the treatment. Still, it is useful for present purposes to consider how data from such a study would be analyzed.

Pretest and posttest scores constitute repeated measures, two scores from each subject. If this single-group study had been presented in , you might have been tempted to correlate pretest and posttest scores. Certainly you could compute regression and correlation coefficients and these would tell you how, and how well, you could predict the posttreatment from the pretreatment assessment. But this would not tell you whether the pretreatment and posttreatment assessments were different. This is a different question and requires a different analysis.

In order to determine if the mean pretest score is different from the mean posttest score, you would perform an analysis of variance. Time of test (pre versus post) would be the single within-subjects factor. If the P main effect were significant, you would conclude that the mean pretest and posttest scores were significantly different. Do not confuse these two very different analyses. Given two repeated measures, such as a pretest and a posttest:

1. The correlation coefficient tells you how well the second score can be predicted from the first.
2. The analysis of variance, on the other hand, tells you whether the mean score for the first test is different from the mean score for the second test.

These are two separate but important pieces of information. But note that there is no necessary relation between their significance: One, neither, or both might be significant.

The single-factor study just described, like the somewhat more complex two-factor study described a few paragraphs earlier, assesses subjects both before and after treatment, but the more complex study, in addition, assigns some subjects to a treatment group and the remaining subjects to a no-treatment control group. For this reason, your first thought might be to analyze the data from the two-factor study as a one-between, one-within, 2 2 design. The between-subjects factor is treatment (training vs. no training) and the within-subjects factor is time of testing (before vs. after training). A main effect for training (factor A ) would indicate that, considering both pretest and posttest scores, one group scored higher than the other. A main effect for time of testing (factor P ) would indicate that, considering both treatment conditions, scores tended to be higher at one time compared to the other.

The result the investigator most likely desires, however, is a significant interaction. If the treatment affects the posttest score, then there should be a difference between pretest and posttest scores only for treated subjects. In other words, the effect of the treatment factor on score should be conditional onthat is, should interact withthe level of treatment. It is a significant interaction between treatment level and time of testing that allows the investigator to conclude that subjects were affected by the treatment offered in the treatment condition. Thus one way to test the effectiveness of a treatment, given the present example, is to analyze the data as a one-between (factor A = treatment level), one-within (factor P = time of assessment) study, in which case a significant AP interaction might indicate that scores for the treated subjects, but not the untreated subjects, were significantly different at the second assessment.

There is a second way to analyze these same data. It is somewhat simpler and more straightforward than the mixed two-factor strategy just described and, in fact, it is usually preferred by experts. Simply put, the question of interest concerns whether the treatment mattersthat is, whether it has an effect on a particular outcome. Treatment is the research factor of interest and, as noted previously, it is a between-subjects factor. This does not change. However, rather than treat the repeated measures as levels of a within-subjects factor, it makes sense to regard the second measure as the outcome variable and the first measure as the covariate. This reduces a one-between, one-within design to a one-between design with a covariate.