How to Calculate the Detection Limit

“Detection limit” most often means the lowest analyte concentration that a measurement method can distinguish from its background signal with a stated level of confidence. There is no single formula that applies to every instrument or field. For a linear analytical calibration, a widely used estimate is the response standard deviation divided by the calibration slope, multiplied by a method-specific factor.

The common calculation

Use

$$\mathrm{LOD} = \frac{3.3\sigma}{S}$$

where:

• $\sigma$ is the standard deviation of the measured response near the blank or detection limit;
• $S$ is the positive slope magnitude of the calibration curve, in response units per concentration unit.

The units work out because response units cancel: $\sigma/S$ has concentration units. The factor 3.3 is used in some analytical-validation guidance, including ICH guidance. Other conventions use a factor of 3, so follow the standard specified for your method or field. This is an estimate, not a universal physical constant.

If you also need the limit of quantitation (LOQ), a common estimate is

$$\mathrm{LOQ} = \frac{10\sigma}{S}.$$

The LOQ is higher because it is intended to indicate a level that can be measured with acceptable precision and accuracy, not merely distinguished from background.

Step-by-step method

1. Choose a suitable blank or low-level data set. Use blanks that represent the samples as closely as possible, including the same matrix and preparation steps. If the method’s validation procedure specifies another way to estimate variability, use that procedure.
2. Estimate $\sigma$. Calculate the standard deviation of replicate blank responses, or use an accepted estimate such as the residual standard deviation from an appropriate calibration model. The estimate should reflect the variability relevant to the intended measurements.
3. Determine $S$. Fit the calibration curve over a range where a linear model is justified. The slope magnitude must use the same response and concentration units as the variability estimate.
4. Calculate the LOD and check it experimentally. Apply the chosen factor, then measure samples close to the estimated limit. A calculated value is useful only if real samples at that level behave as expected.

Worked example

Suppose a method measures absorbance against concentration. Replicate blank measurements give a response standard deviation of $0.012$ absorbance units, and the calibration slope is $0.080$ absorbance units per mg/L.

Using the 3.3 convention,

$$\begin{aligned} \mathrm{LOD} &= \frac{3.3(0.012\ \text{absorbance units})}{0.080\ \text{absorbance units per mg/L}} \\ &= 0.495\ \text{mg/L}. \end{aligned}$$

So the estimated detection limit is about 0.50 mg/L. Using the same $\sigma$ and slope, the estimated LOQ is

$$\mathrm{LOQ} = \frac{10(0.012)}{0.080} = 1.5\ \text{mg/L}.$$

These results are estimates. You would check them by measuring blank samples and samples prepared near 0.50 mg/L and 1.5 mg/L, following the method’s validation plan. If the response is not linear near those concentrations, this calculation may not be suitable.

Why the factor is about 3.3

The factor can be understood through a simple statistical model. Suppose blank responses are approximately normal, the response standard deviation is known, and increasing concentration raises the response linearly with slope $S$. A threshold set about 1.645 standard deviations above the blank mean gives a 5% one-sided false-positive rate. To also detect a low-level sample with 95% probability, its mean response must be about another 1.645 standard deviations above that threshold. The total is approximately $3.29\sigma$, which rounds to $3.3\sigma$.

This explanation relies on assumptions, including similar response variability for blank and low-level samples. When those assumptions do not hold, a method-specific statistical procedure may be more appropriate. For example, some methods define a decision threshold and detection capability separately to control false-positive and false-negative rates.

Common mistakes

• Mixing units. If $\sigma$ is in absorbance units, the slope must be in absorbance units per concentration. Do not divide by a slope expressed in different response units.
• Using the slope alone as the detection limit. A steeper slope improves sensitivity, but the noise also matters. The LOD depends on both $\sigma$ and $S$.
• Treating LOD and LOQ as interchangeable. Detection means distinguishing a signal from background; quantitation requires adequate measurement performance.
• Using an unrepresentative blank. A clean solvent blank may underestimate variability in real samples if the sample matrix or preparation adds noise.
• Reporting a formula without its assumptions. State the convention and explain how you estimated $\sigma$, what calibration slope you used, and the units of the result.

A detection limit is therefore tied to a particular procedure, instrument, matrix, and decision rule. If a regulation, laboratory standard, or validated protocol prescribes a calculation, use that definition rather than substituting a generic formula.