Chapter 3: Analyse scientific research findings

3.1 Research findings

Research findings are the answers a study gives to its research questions once the collected data have been organised, analysed and interpreted. A field trial that compares three fertiliser treatments on maize produces figures for every plot, but those figures become findings only when they are summarised, compared and explained: for example, that one treatment gave a higher mean grain yield than the others and that the difference is larger than chance alone would explain. Findings must follow from the data. They are never opinions, hopes or what the researcher expected to see.

An agricultural extension officer or farm manager who understands how findings are produced can judge whether a trial result is reliable before advising farmers to change a feed, a variety or a spacing. The kinds of findings a study can report include the following.

• Descriptive findings: state what the data look like, such as the average milk yield of a herd and how much the cows differ from one another.
• Comparative findings: state whether groups differ, such as whether three maize fertiliser treatments gave different mean yields.
• Relationship findings: state whether two measurements move together, such as feed intake and weight gain in growing pigs.
• Trend findings: state how a measurement changes over time, such as monthly egg production in a layer unit over one year.
• Qualitative findings: state recurring themes from interviews and observation, such as the reasons farmers give for not adopting a new technology.

This topic defines the terms used in analysis and then explains three methods for quantitative data: analysis of variance (ANOVA), measures of central tendency and measures of dispersal. Writing the results up is covered in Prepare research report using the methods in 3.1.2.

3.1.1 Definition of terms

Analysis has its own vocabulary, and the same words appear in trial reports, extension bulletins and statistical software output. Each term below is defined as it is used in agricultural research.

• Data: facts or measurements collected in a study, such as the kilograms of grain harvested from each plot. One measurement is a datum and many are data.
• Raw data: the figures exactly as they were recorded in the field or laboratory, before any cleaning or calculation.
• Variable: any characteristic that can take different values from one plot, animal or farm to another, such as grain yield, live weight or milk fat percentage.
• Independent variable: the factor the researcher changes or compares, such as the type of fertiliser or the feed ration. It is also called the treatment factor.
• Dependent variable: the measurement expected to respond to the independent variable, such as yield or weight gain.
• Treatment: one level of the factor being tested. In a feeding trial with three rations, each ration is a treatment, and the group that gets the farm's usual ration is the control.
• Experimental unit: the smallest item that receives a treatment independently, such as one plot, one fish pond or one broiler pen. Measurements taken on animals sharing a pen are not independent units.
• Replication: the number of experimental units given each treatment. Several replicates show how much results vary by chance.
• Population: the whole group the study wants to learn about, such as all dairy cows on a large farm.
• Sample: the part of the population actually measured. How a sample is chosen is covered under sampling techniques in the chapter on research methods.
• Parameter and statistic: a parameter describes a whole population, and a statistic describes a sample. The mean of 10 measured cows is a statistic that estimates the herd's mean, which is a parameter.
• Hypothesis: a statement that can be tested with data. The null hypothesis states that there is no difference or no effect, and the alternative hypothesis states that there is one.
• Significance level: the chance of wrongly rejecting a true null hypothesis that the researcher accepts before analysis, usually 5% (0.05) in agricultural trials.
• P-value: the probability of obtaining results at least as extreme as those observed if the null hypothesis were true. A p-value below the chosen significance level means the null hypothesis is rejected.
• Degrees of freedom: the number of values in a calculation that are free to vary. For example, for n observations around one mean, the degrees of freedom are n − 1.
• Outlier: a value far from the rest of the data. It may be a recording error or a genuine unusual result, and it must be checked before it is kept or removed.

3.1.2 Identify data analysis methods

Data analysis methods are the procedures used to turn measurements into findings. The choice depends on the research question and on the type of data. A study that asks whether three treatments differ needs a different method from one that asks what the typical value is. The three methods in this topic answer different questions and are usually used together.

Data types and the method they allow

Measurements are of different kinds, and the kind decides which summary is meaningful. A mean of breed names is meaningless, while a mean of milk yields is useful.

Preparing data before analysis

Mistakes in the data spoil every method that follows. Check the data before any calculation.

• Check entries against field sheets: compare the typed figures with the original record, because a misplaced decimal point can change a mean enormously.
• Check units: confirm that all weights are in the same unit, such as kilograms, and all areas in square metres, before pooling plots.
• Look for missing values: note any plot or animal with no measurement, such as a pen lost to disease, and record the reason rather than entering zero.
• Look for outliers: list values far from the rest, return to the source to see whether they are errors, and report any value removed and why.
• Label every column: give each variable a clear name and unit so that another researcher can understand the sheet without asking.

3.1.2.1 ANOVA

Analysis of variance (ANOVA) tests whether the means of three or more groups differ by more than chance variation would explain. It works by comparing the variation between the group means with the variation among the plots or animals inside each group. If the groups differ much more than the plots inside a group do, the treatment is likely to have had an effect. Comparing two means at a time would need many separate tests, and each extra test increases the risk of a false significant result, so ANOVA tests all groups in one step.

The simplest form is one-way ANOVA, used when plots or animals are allocated to treatments completely at random and only one factor is tested, such as fertiliser type. The hypotheses are stated before analysis.

• Null hypothesis: all treatment means are equal.
• Alternative hypothesis: at least one treatment mean differs from the others.

ANOVA gives dependable results when these conditions hold.

• Independence: the measurement on one plot or pen does not depend on another.
• Random allocation: treatments were assigned to experimental units by chance, for example by drawing numbers.
• Normality: the variation around each group mean is roughly bell-shaped.
• Equal variation: the groups have similar spread, which is checked with the standard deviation of each group.
• Measurement scale: the response, such as kilograms of grain, is a measured quantity.

The calculation uses these quantities, where k is the number of groups, n is the number of observations in each group and N is the total number of observations.

Grand mean = total of all observations ÷ N
Between-group sum of squares (SSB) = Σ n × (group mean − grand mean)²
Within-group sum of squares (SSW) = Σ (observation − its group mean)²
Total sum of squares (SST) = SSB + SSW
Degrees of freedom between groups = k − 1
Degrees of freedom within groups = N − k
Mean square = sum of squares ÷ its degrees of freedom
F = mean square between (MSB) ÷ mean square within (MSW)

The steps of a one-way ANOVA are as follows.

1. State the null and alternative hypotheses and choose the significance level, normally 0.05.
2. Arrange the data in columns, one per treatment, and find each column total and mean.
3. Find the grand total and the grand mean of all observations.
4. Calculate SSB from the distance of each group mean from the grand mean, weighted by the number of observations in the group.
5. Calculate SSW from the distance of each observation from its own group mean, and check that SSB + SSW equals SST.
6. Divide each sum of squares by its degrees of freedom to get MSB and MSW, then divide MSB by MSW to get F.
7. Compare F with the critical value from an F table for the two degrees of freedom at the chosen significance level. If F is larger, reject the null hypothesis.
8. If the null hypothesis is rejected, use a follow-up test to find which means differ, and write the conclusion in plain words.

Example: A demonstration farm tests three treatments on maize, each on four plots of 50 m² laid out at random: no fertiliser, DAP fertiliser and farmyard manure. Grain yield in kg per plot is recorded.

Grand total = 78 + 102 + 88 = 268 kg
N = 12, k = 3
Grand mean = 268 ÷ 12 = 22.3333 kg

SSB = 4 × (19.5 − 22.3333)² + 4 × (25.5 − 22.3333)² + 4 × (22.0 − 22.3333)²
SSB = 4 × 8.0278 + 4 × 10.0278 + 4 × 0.1111
SSB = 32.1111 + 40.1111 + 0.4444 = 72.6667

SSW, no fertiliser = (−1.5)² + (0.5)² + (−0.5)² + (1.5)² = 2.25 + 0.25 + 0.25 + 2.25 = 5.0
SSW, DAP = (−1.5)² + (0.5)² + (−0.5)² + (1.5)² = 5.0
SSW, manure = 0² + 1² + (−1)² + 0² = 2.0
SSW = 5.0 + 5.0 + 2.0 = 12.0

SST = 72.6667 + 12.0 = 84.6667
Check: Σx² − (grand total)² ÷ N = 6070 − 268² ÷ 12 = 6070 − 5985.3333 = 84.6667

Degrees of freedom between = 3 − 1 = 2
Degrees of freedom within = 12 − 3 = 9
MSB = 72.6667 ÷ 2 = 36.3333
MSW = 12.0 ÷ 9 = 1.3333
F = 36.3333 ÷ 1.3333 = 27.25

The critical value of F for 2 and 9 degrees of freedom at the 5% level is 4.26. Because 27.25 is greater than 4.26, the null hypothesis is rejected: at least one treatment mean differs from the others.

ANOVA does not say which means differ. A follow-up test such as the least significant difference (LSD) does. The LSD is the smallest difference between two treatment means that is significant, using the critical t value for the error degrees of freedom (2.262 for 9 degrees of freedom at the 5% level, two-sided).

LSD = t × √(2 × MSW ÷ n)
LSD = 2.262 × √(2 × 1.3333 ÷ 4)
LSD = 2.262 × √0.6667 = 2.262 × 0.8165 = 1.85 kg per plot

The differences between the treatment means are 25.5 − 19.5 = 6.0 kg for DAP against no fertiliser, 22.0 − 19.5 = 2.5 kg for manure against no fertiliser, and 25.5 − 22.0 = 3.5 kg for DAP against manure. Each is larger than 1.85 kg, so all three treatments differ significantly. In plain words: DAP gave the highest mean yield, farmyard manure the next, and the plots with no fertiliser gave the lowest. Converted to kilograms per hectare, the means are 3 900 kg (no fertiliser), 4 400 kg (manure) and 5 100 kg (DAP), for example 19.5 ÷ 50 × 10 000 = 3 900 kg per hectare.

A significant result applies only to the conditions of the trial, such as the soil, rainfall and variety used. Spreadsheet programs and free statistical software can run a one-way ANOVA directly, but a trainee who has done the working by hand can check that the software output makes sense.

3.1.2.2 Measure of central tendency

A measure of central tendency gives one value that represents the centre of a set of data. It lets a researcher describe a herd, a field or a flock with a single figure and compare it with another. The three common measures are the mean, the median and the mode. Each is calculated differently and suits different data.

• Mean: the sum of all values divided by the number of values. It uses every value, so it is the usual choice for measured data such as yield and weight.
• Median: the middle value when the data are arranged in order. If there are two middle values, the median is their average. It is not affected by extreme values.
• Mode: the value that occurs most often. A set of data can have one mode, more than one mode or none. It is the only measure that suits categories such as breed or crop variety.

Mean = Σx ÷ n
Median = the value in position (n + 1) ÷ 2 when the values are in order
Mode = the value with the highest frequency

Example: A dairy farm records the milk yield, in litres per day, of 10 cows on one morning's herd record: 12, 15, 14, 18, 15, 13, 16, 15, 20, 14.

Mean:
Σx = 12 + 15 + 14 + 18 + 15 + 13 + 16 + 15 + 20 + 14 = 152
Mean = 152 ÷ 10 = 15.2 litres per cow per day

Median:
Ordered values: 12, 13, 14, 14, 15, 15, 15, 16, 18, 20
Position = (10 + 1) ÷ 2 = 5.5, so the median is the average of the 5th and 6th values
Median = (15 + 15) ÷ 2 = 15 litres

Mode:
15 occurs three times, more often than any other value
Mode = 15 litres

When the same data are given as a frequency table, the mean is found by multiplying each value by its frequency.

Mean = Σ(f × x) ÷ Σf = 152 ÷ 10 = 15.2 litres

The mean is pulled towards extreme values and the median is not. Suppose the 20 litres had been typed as 200 litres by mistake. The mean would become (152 − 20 + 200) ÷ 10 = 332 ÷ 10 = 33.2 litres, which is more than double the true figure, while the median would stay at 15 litres. A large gap between mean and median is therefore a warning to check the data for errors or for a skewed distribution.

The mode is the right summary for categories. If a herd of 10 cows has 5 Friesian, 3 Ayrshire and 2 Jersey cows, the modal breed is Friesian, and no mean or median can be calculated for breed names.

3.1.2.3 Measure of dispersal

A measure of dispersal, also called a measure of spread or variability, shows how widely the values in a set of data are scattered around the centre. Two sets of data can have the same mean and still be very different. A fish farmer who harvests a pond where every fish weighs about 250 g can sell to one market, while a pond with the same mean but fish from 180 g to 320 g needs grading. The mean alone hides this, so a measure of dispersal is always reported with it.

The common measures are the range, the interquartile range, the variance, the standard deviation and the coefficient of variation.

• Range: the highest value minus the lowest value. It is quick to find but uses only two values.
• Interquartile range (IQR): the third quartile minus the first quartile, which is the spread of the middle half of the data. Each quartile is the median of the lower half or upper half of the ordered data.
• Variance: the average squared distance of the values from the mean, calculated for a sample by dividing by n − 1.
• Standard deviation (SD): the square root of the variance. It is in the same unit as the data, so it is the most widely reported measure.
• Coefficient of variation (CV): the standard deviation as a percentage of the mean. It allows spread to be compared between variables with different units or sizes.

Range = highest value − lowest value
Variance (s²) = Σ(x − mean)² ÷ (n − 1)
Standard deviation (s) = √variance
Coefficient of variation = (s ÷ mean) × 100%

For a roughly bell-shaped distribution, about two thirds of the values lie within one standard deviation of the mean, and about 95 out of 100 lie within two standard deviations.

Example: A fish farm samples 8 tilapia from each of two ponds at harvest and weighs them in grams.

Pond A: 244, 246, 250, 250, 250, 250, 254, 256

Pond B: 180, 210, 210, 230, 250, 290, 310, 320

Pond A
Σx = 2000, n = 8, mean = 2000 ÷ 8 = 250 g
Deviations from the mean: −6, −4, 0, 0, 0, 0, 4, 6
Σ(x − mean)² = 36 + 16 + 0 + 0 + 0 + 0 + 16 + 36 = 104
Variance = 104 ÷ 7 = 14.857
SD = √14.857 = 3.85 g
CV = 3.8545 ÷ 250 × 100 = 1.54%
Range = 256 − 244 = 12 g
IQR: Q1 = (246 + 250) ÷ 2 = 248, Q3 = (250 + 254) ÷ 2 = 252, so IQR = 4 g

Pond B
Σx = 2000, n = 8, mean = 2000 ÷ 8 = 250 g
Deviations from the mean: −70, −40, −40, −20, 0, 40, 60, 70
Σ(x − mean)² = 4900 + 1600 + 1600 + 400 + 0 + 1600 + 3600 + 4900 = 18 600
Variance = 18 600 ÷ 7 = 2657.14
SD = √2657.14 = 51.55 g
CV = 51.547 ÷ 250 × 100 = 20.62%
Range = 320 − 180 = 140 g
IQR: Q1 = (210 + 210) ÷ 2 = 210, Q3 = (290 + 310) ÷ 2 = 300, so IQR = 90 g

Both ponds have a mean weight of 250 g, but pond B has an SD of 51.55 g against 3.85 g for pond A, and a CV of 20.62% against 1.54%. The fish in pond A are very uniform, and the fish in pond B vary widely, which suggests uneven feeding, mixed ages or crowding that the farm manager should investigate. Dispersal is also needed in ANOVA, because the within-group spread is the yardstick against which differences between treatments are judged.