1.1.1 The Art of Experimentation
One of the first questions facing an experimenter is, How many observations do I need to take? or alternatively, Given my limited budget, how can I gain as much information as possible? These are not questions that can be answered in a couple of sentences. They are, however, questions that are central to the material in this book. As a first step towards obtaining an answer, the experimenter must ask further questions, such as, What is the main purpose of running this experiment? and What do I hope to be able to show?
Typically, an experiment may be run for one or more of the following reasons:
- (i)
to determine the principal causes of variation in a measured response,
- (ii)
to find the conditions that give rise to a maximum or minimum response,
- (iii)
to compare the responses achieved at different settings of controllable variables,
- (iv)
to obtain a mathematical model in order to predict future responses.
Observations can be collected from observational studies as well as from experiments , but only an experiment allows conclusions to be drawn about cause and effect. For example, consider the following situation:
The output from each machine on a factory floor is constantly monitored by any successful manufacturing company. Suppose that in a particular factory, the output from a particular machine is consistently of low quality. What should the managers do? They could conclude that the machine needs replacing and pay out a large sum of money for a new one. They could decide that the machine operator is at fault and dismiss him or her. They could conclude that the humidity in that part of the factory is too high and install a new air conditioning system. In other words, the machine output has been observed under the current operating conditions (an observational study), and although it has been very effective in showing the management that a problem exists, it has given them very little idea about the cause of the poor quality.
It would actually be a simple matter to determine or rule out some of the potential causes. For example, the question about the operator could be answered by moving all the operators from machine to machine over several days. If the poor output follows the operator, then it is safe to conclude that the operator is the cause. If the poor output remains with the original machine, then the operator is blameless, and the machine itself or the factory humidity is the most likely cause of the poor quality. This is an experiment. The experimenter has control over a possible cause in the difference in output quality between machines. If this particular cause is ruled out, then the experimenter can begin to vary other factors such as humidity or machine settings.
It is more efficient to examine all possible causes of variation simultaneously rather than one at a time. Fewer observations are usually needed, and one gains more information about the system being investigated. This simultaneous study is known as a factorial experiment. In the early stages of a project, a list of all factors that conceivably could have an important effect on the response of interest is drawn up. This may yield a large number of factors to be studied, in which case special techniques are needed to gain as much information as possible from examining only a subset of possible factor settings.
The art of designing an experiment and the art of analyzing an experiment are closely intertwined and need to be studied side by side. In designing an experiment, one must take into account the analysis that will be performed. In turn, the efficiency of the analysis will depend upon the particular experimental design that is used to collect the data. Without these considerations, it is possible to invest much time, effort, and expense in the collection of data which seem relevant to the purpose at hand but which, in fact, contribute little to the research questions being asked. A guiding principle of experimental design is to keep it simple. Interpretation and presentation of the results of experiments are generally clearer for simpler experiments.
Three basic techniques fundamental to experimental design are replication, blocking, and randomization. The first two help to increase precision in the experiment; the last is used to decrease bias. These techniques are discussed briefly below and in more detail throughout the book.
1.1.2 Replication
Replication is the repetition of experimental conditions so that the effects of interest can be estimated with greater precision and the associated variability can be estimated.
There is a difference between replication and repeated measurements. For example, suppose four subjects are each assigned to a drug and a measurement is taken on each subject. The result is four independent observations on the drug. This is replication. On the other hand, if one subject is assigned to a drug and then measured four times, the measurements are not independent. We call them repeated measurements. The variation recorded in repeated measurements taken at the same time reflects the variation in the measurement process, while the variation recorded in repeated measurements taken over a time interval reflects the variation in the single subjects response to the drug over time. Neither reflects the variation in independent subjects responses to the drug. We need to know about the latter variation in order to generalize any conclusion about the drug so that it is relevant to all similar subjects.
1.1.3 Blocking
A designed experiment involves the application of treatments to experimental units to assess the effects of the treatments on some response. The experimental units, which may be subjects, materials, conditions, points in time, or some combination of these, will be variable and induce variation in the response. Such variation in experimental units may be intentional, as the experimental conditions under which an experiment is run should be representative of those to which the conclusions of the experiment are to be applied. For inferences to be broad in scope, the experimental conditions should be appropriately varied. Blocking is a technique that can often be used to control and adjust for some of the variation in experimental units.
To block an experiment is to divide, or partition, the experimental units into groups called blocks in such a way that the experimental units in each block are intended to be relatively similar, so that treatments assigned to experimental units in the same block can be compared under relatively similar experimental conditions. If blocking is done well, then comparisons of two or more treatments are made more precisely in the experiment than similar comparisons from an unblocked design. For example, in an experiment to compare the effects of two skin ointments for rash, the two treatments can be compared more precisely on two arms of the same person than on the arms of two different people. Either circumstance can be replicated, ideally using subjects randomly sampled from or at least representative of the population of interest.
