4. A good (well designed) Experiment

Should:

1. Have a clear specification of the aims of the experiment.
The hypothesis to be tested needs to be clearly formulated before starting any detailed planning. It should be one which the experiment is capable of answering.
It would be a serious error to look at the results of the experiment and then adjust the hypothesis to fit them!

2. Be unbiased
There should be no systematic differences between the treated and control groups apart from the effects of the treatment.

Bias may result in false positive results when the effects of some other factor are confounded (mixed with) the treatment effect. It is avoided by correct identification of the experimental unit, blinding, and by randomisation

Bias is minimised by 1. correct choice of the experimental unit, 2. randomisation of the units to treatments and in the order in which subjects are housed and outcomes are measured, and 3. blinding where possible, using coded samples.

3. Be powerful
 If the treatment really has an effect, there should be a high chance that it can be detected. Experiments which lack power will have too many false negative results.

Power is increased by 1. Larger sample sizes, 2. Good control of variability and 3. Use of sensitive  subjects. However, large sample sizes cost animals and money so emphasis should be placed on the last two of these.

4. Have a wide range of applicability
An experiment where the results can only be replicated in some animal houses but not in others lacks generality.  The range of applicability is explored using factorial and randomised block designs which can sample different situations. See the concepts of internal and external validity, below.

5. Experiments should be simple
They should not be so complex that mistakes are made, the statistical analysis is excessively complex or they are impossible to interpret.

Clearly written protocols and stand operating procedures should be used. In some cases it may be necessary to work to “Good Laboratory Practice” standards. http://en.wikipedia.org/wiki/Good_Laboratory_Practice

6. It should be possible to statistically analyse the result of an experiment.
The statistical analysis and the experiment should be planned at the same time.

 An investigator should never start an experiment without knowing how it is going to be analysed. The results of each experiment should be analysed before starting the next one so that the findings from the first experiment can be taken into account. The most powerful available statistical methods should be used, such as parametric rather than non parametric tests, where applicable..

Internal and external validity

An experiment can be said to have high internal validity if it has a high probability of getting the correct answer. Basically, this means that it should be unbiassed and powerful so that it is unlikely to produce either a false positive or a false negative result.

An experiment will have high external validity if the results can be generalised to other conditions or situations. Note that it can not have high external validity unless it first has high internal validity. The use of randomised block designs (which can sample a range of environments) and factorial experimental designs can be used to increase external validity. Repeating an experiment in another laboratory by other investigators will also increase external validity, assuming the results are repeatable.

As an example, an experiment which uses only a single strain of mice may have high internal validity, but if the same results are not seen with other strains of mice, then it will have low external validity.

It is acceptable to do an experiment with high internal validity but no exploration of its external validity, provided it is made clear that the external validity is unknown. But note that in many cases randomised block and factorial designs can be done at little or no extra cost.