How Many Times Should an Experiment Be Repeated?

There is no universal number of times to repeat an experiment. A useful sample size depends on how variable the measurements are and what the experiment needs to establish. Repeating a measurement until the answer “looks stable” is not a reliable planning rule: random variation can persist, and repeatedly checking results can encourage selective stopping.

Instead, decide whether you are estimating a value to a chosen precision or testing for a particular effect. Those goals lead to different sample-size calculations.

First define what counts as a repetition

A repetition should usually be an independent experimental unit: for example, a separately prepared sample, a different participant, or a new run performed under the planned conditions. Repeated readings of the same object are not necessarily independent experiments. They may help characterize instrument noise, but they do not replace independent samples when the goal is to generalize beyond that object.

In a physics experiment, you might take several readings during one run and repeat the run on different occasions. Record these levels separately. If observations share a sample, instrument setup, batch, or operator, they may be correlated. Treating correlated readings as independent can make uncertainty appear smaller than it is.

Choose a planning goal

Estimate a mean to a target precision

If the main goal is to estimate a population mean, specify a margin of error $E$ for the confidence interval. For independent, approximately normally distributed observations with estimated standard deviation $\sigma$, a large-sample planning approximation is

where $1-\alpha$ is the confidence level. For a 95% confidence interval, $z_{1-\alpha/2}$ is about 1.96. This formula assumes the observations are independent and that $\sigma$ is a reasonable planning estimate. For small samples, use a $t$ critical value and refine the sample size iteratively.

Detect a difference or effect

If the aim is to detect a difference between conditions, specify the smallest effect that would matter scientifically, along with a significance level (often 0.05) and desired power (often 80% or 90%). Power is the probability of detecting an effect of the specified size if that effect is real. The required sample size also depends on variability, the statistical test, and the design—for example, paired measurements generally need a different calculation from two independent groups.

Do not choose an effect size just because it produces a convenient sample size. Base it on scientific relevance, prior evidence, or a justified pilot estimate. A test with low power can fail to detect a meaningful effect; a very large sample can make a negligible effect statistically significant.

Worked example: estimating a mean

Suppose you want to estimate a physical quantity with a 95% confidence interval whose half-width is at most 0.10 units. Earlier measurements suggest a standard deviation of 0.20 units.

The normal-approximation calculation gives

Round up to 16 as an initial estimate. Because the standard deviation is estimated and the sample is small, check the $t$ interval instead. For $n=16$, the 95% $t$ critical value with 15 degrees of freedom is about 2.131, giving a half-width of

That exceeds the 0.10 target. At $n=18$, the critical value with 17 degrees of freedom is about 2.110, and the half-width is

So 18 independent measurements meet the planned precision under these assumptions. This is a planning estimate, not a guarantee: the actual variability may differ, and the interval’s coverage depends on the model assumptions.

Use a pilot study carefully

A small pilot can reveal practical issues, suggest a plausible standard deviation, and help estimate how long the procedure takes. It is usually not precise enough to determine the final sample size by itself, especially when the pilot is tiny. Use prior experiments or instrument knowledge where available, and consider planning for a range of plausible variability values.

If you expect unusable observations, recruit or schedule extra experimental units. For example, if the calculation calls for 18 complete measurements and you expect 10% to be lost, plan for about $18/(1-0.10)=20$ measurements. This adjustment does not fix biased losses; investigate why data might be missing.

Common mistakes

• Using an arbitrary round number: “Three repeats” is not automatically sufficient. It may be a useful minimum for checking repeatability, but it does not establish adequate precision or power.
• Counting technical readings as independent samples: Multiple readings from one sample often share the same underlying conditions. Identify the true experimental unit before calculating $n$.
• Stopping when the result becomes significant: Unless you use a valid sequential design with prespecified rules, repeated significance checks can inflate the false-positive rate. Set the sample size or stopping rule in advance.
• Confusing precision with power: A narrow confidence interval and a high chance of detecting a chosen effect are related but distinct goals. State which one drives the design.
• Ignoring the design: Pairing, clustering, repeated measures, and multiple outcomes can change the required number. Use a calculation or statistical model that matches the experiment.

Before collecting data, write down the outcome, experimental unit, target precision or effect size, assumed variability, confidence level or power, and planned analysis. Reporting these assumptions makes the number of repetitions interpretable and lets others judge whether the experiment could answer its intended question.