Chapter 6: Analyze science laboratory research findings

6.1 Standard data analysis methods

Once the results are in, the analysis answers the research questions with methods that other laboratories recognise and can repeat. The families of methods and the choice of a statistical test are introduced in the topic on Data analysis methods in the data analysis outcome. This topic takes the standard routines that a laboratory researcher applies to findings: checking that the results can be used, comparing them with a limit or a reference, comparing methods, monitoring results over time and judging performance against an assigned value. Each routine is shown with a worked example.

The standard sequence of analysis

1. Confirm that every sample planned has a result or a recorded reason for none, and that quality control results (blanks, duplicates, check standards) were within their limits.
2. Apply the rules for suspect values and values below the detection limit that were written in the analysis plan, and record every decision.
3. Compare the summary results with any limit, specification or reference value fixed in the plan, using the routines below.
4. Check the assumptions of the chosen test and carry out the planned comparison or relationship analysis, as set out in the topic on Data analysis methods, and record the test statistic, the sample size, the p-value and a confidence interval.
5. Interpret the result against the research question, stating both the size of the effect and its uncertainty.
6. Save the data file, the analysis steps and the output together so that the analysis can be repeated.

The plan for analysis is written before data are collected. A method chosen after the results are seen can be steered towards the answer the researcher hopes for.

Comparing a result with a limit or reference value

A common question is whether the true mean of a lot is above or below a stated value. The mean of the replicates is compared with the value using a one-sample t-test or, equivalently, a confidence interval.

Worked example (food-quality laboratory): a producer's own working limit for the moisture of a lot of dried fruit is 14.0%. Eight replicate moisture results are 13.6, 13.9, 13.4, 13.8, 13.5, 13.7, 13.3 and 13.6%. The question is whether the lot mean is below 14.0%.

• Mean = 108.8 ÷ 8 = 13.6%.
• Standard deviation s = 0.20%, so the standard error = 0.20 ÷ √8 = 0.0707%.
• Test statistic t = (13.6 − 14.0) ÷ 0.0707 = −5.66, with 7 degrees of freedom.
• The one-sided critical value at the 5% level and 7 degrees of freedom is 1.895.

Because 5.66 is greater than 1.895, the mean is significantly below the limit. The upper end of a one-sided 95% confidence interval is 13.6 + 1.895 × 0.0707 = 13.73%, which is below 14.0%, giving the same conclusion in terms the customer can read.

Comparing two methods or two analysts

When the same samples are measured by two methods, the question is agreement, not just a difference in means. The method of differences and limits of agreement shows the average bias between the methods and the spread of the disagreement.

Worked example (a juice laboratory): the sugar content of eight samples in °Brix is measured by a lab reference method and by a handheld refractometer.

• Mean difference (bias) = 1.7 ÷ 8 = +0.2125 °Brix.
• Standard deviation of the differences = 0.1553 °Brix.
• Limits of agreement = 0.2125 ± 1.96 × 0.1553 = −0.092 to +0.517 °Brix.

The handheld device reads slightly high on average, and for 95% of samples it is expected to differ from the reference by between −0.09 and +0.52 °Brix. Whether this is acceptable is decided against the purpose of the measurement and is stated before the comparison is run. A paired t-test on the same differences gives t = 3.87 with 7 degrees of freedom, showing that the bias is real, but it does not say whether the bias matters in practice.

Monitoring results over time with control charts

A control chart plots results from a check standard or a control sample in time order, with a centre line at the mean and limits from the standard deviation of earlier results. It shows whether a measurement process is staying stable.

Worked example (industrial quality-control laboratory): ten runs of a check standard gave 50.2, 49.5, 50.8, 49.9, 50.4, 49.7, 50.1, 50.6, 49.3 and 50.5 mg/L. The mean is 50.1 and the standard deviation 0.494.

• Warning limits = mean ± 2 s = 49.11 to 51.09 mg/L.
• Action limits = mean ± 3 s = 48.62 to 51.58 mg/L.

A new result of 51.2 lies beyond the warning limit but inside the action limit, so the next run is watched closely. A result of 51.9 lies beyond the action limit: analysis stops, the cause is found and the samples run since the last good check are repeated. A run of seven or eight successive points on one side of the centre line, or a steady rise or fall, also signals a drift that needs action even though no point is beyond a limit.

Judging performance against an assigned value

In proficiency testing a laboratory analyses a sample whose value is assigned by a reference or by consensus and compares its result using a z-score.

z = (laboratory result − assigned value) ÷ standard deviation for proficiency assessment

Worked example: a laboratory reports 24.6 mg/kg for a sample with an assigned value of 25.0 mg/kg and a standard deviation for proficiency assessment of 1.0 mg/kg. z = (24.6 − 25.0) ÷ 1.0 = −0.4.

• A z-score with an absolute value of 2 or less is generally taken as satisfactory.
• An absolute value between 2 and 3 is questionable and calls for a check of the method.
• An absolute value of 3 or more is unsatisfactory and calls for corrective action.

Describing and reporting uncertainty

A result is incomplete without its uncertainty, and replicate results are reported as set out in the topic on Data analysis methods. Significant figures follow the precision of the method, so a mean of 13.6% from a method good to 0.1% is not written as 13.6000%. Intermediate figures are carried with extra digits and only the final answer is rounded.

Choosing among standard methods

• Result against a limit: One-sample t-test or confidence interval.
• Two methods on the same samples: Differences, limits of agreement and a paired comparison.
• Stability of a measurement process: Control chart with warning and action limits.
• Performance against an assigned value: z-score.
• Two or more groups, relationships and associations: The techniques taught in the topics on Data analysis techniques and ANOVA.