Data analysis is a critical phase in applied statistics research, transforming raw collected data into meaningful information that can inform decision-making and policy formulation. For professionals in Kenya, particularly those working with institutions such as county governments, banks, or healthcare facilities, mastering data processing techniques ensures that research findings are accurate, reliable, and actionable. This chapter focuses on the essential steps involved in handling collected research data, beginning with data processing, the foundation for all subsequent statistical analysis.

7.1 Data Processing

Data processing in applied statistics involves organizing, cleaning, and preparing collected data for analysis. This phase is crucial because raw data often contains errors, inconsistencies, or irrelevant information that can distort research results if not addressed properly. Kenyan statisticians working in sectors such as education or agriculture must apply rigorous data processing methods to maintain data integrity and enhance the validity of their findings.

7.1.1 Data Cleaning and Validation

Data cleaning and validation are the first critical steps in data processing. These procedures ensure that the dataset is free from errors, missing values, and inconsistencies that could compromise the analysis.

Data Cleaning

Data cleaning involves identifying and correcting or removing inaccurate, incomplete, or irrelevant data entries. This process is vital because data collected through surveys or interviews in Kenya, for example from a SACCO or a county health department, often contain typographical errors, duplicate records, or outliers.

• Error Detection: This step involves scanning the dataset for anomalies such as impossible values (e.g., negative ages), duplicate entries, or inconsistent coding. Automated software tools like R or Excel can flag these issues quickly.
• Handling Missing Data: Missing data can arise from non-responses or data entry errors. Techniques like imputation, where missing values are estimated based on other available data, or deletion of incomplete records are employed depending on the context and extent of missingness.
• Outlier Identification: Outliers are data points that deviate significantly from other observations. In financial data from banks, for example, unusually high loan amounts might need verification to avoid skewing results.
• Standardization: Data collected from different sources, such as multiple county offices, may use varying formats or units. Cleaning involves converting these into a consistent format to enable comparison.
• Consistency Checks: Cross-verifying related data fields ensures logical consistency, such as matching age and date of birth or verifying that income levels align with occupational categories.

Data Validation

Validation confirms that the dataset accurately represents the intended information and meets quality standards before analysis.

• Verification Against Source: Comparing entered data with original survey forms or digital logs to detect transcription errors.
• Range Checks: Ensuring that all data values fall within acceptable limits defined during the research design phase.
• Logical Checks: Applying rules such as ensuring that respondents’ answers to related questions do not contradict each other.
• Completeness Checks: Confirming that all required fields have been filled appropriately.
• Audit Trails: Maintaining records of changes made during cleaning to provide transparency and reproducibility.

Data Validation

Data validation is the process of ensuring that the cleaned dataset accurately reflects the intended information and adheres to quality standards before proceeding to analysis. In Kenyan applied statistics, validation is especially important when data is collected from diverse sources such as county government departments, banks, or healthcare facilities, where errors can easily occur during manual entry or digital transfer.

• Verification Against Source Documents: This involves comparing data entries with original survey forms, hospital records, or transaction logs to identify transcription errors. For example, a county health department may cross-check patient data entered into a database against physical registration forms.
• Range Checks: Data values are checked to ensure they fall within acceptable limits defined during research design. For instance, in a survey of school children, age values should be within the expected range (e.g., 5–18 years) as per Ministry of Education guidelines.
• Logical Consistency Checks: Applying rules to ensure related data fields do not contradict each other, such as confirming that income levels correspond with occupational categories in a Kenya Revenue Authority (KRA) dataset.
• Completeness Checks: Ensuring all required fields are filled and no critical information is missing. For example, in a National Hospital Insurance Fund (NHIF) dataset, each record should have a valid membership number and claim details.
• Audit Trails: Maintaining records of changes made during cleaning and validation, which provides transparency and allows for reproducibility. This is crucial in collaborative research projects involving multiple universities or agencies.

7.1.2 Data Coding and Categorization

Once data is cleaned and validated, it needs to be organized systematically for analysis. Coding and categorization convert raw responses into numerical or categorical variables suitable for statistical software.

Data Coding

Data coding assigns numeric or symbolic codes to qualitative responses or open-ended questions. For example, in a survey conducted at a university about student satisfaction, responses like "Very Satisfied" or "Dissatisfied" can be coded as 4 and 2 respectively.

• Developing a Codebook: A detailed guide that defines each code and its corresponding response ensures consistency across data entry clerks or analysts.
• Assigning Codes to Responses: Both manual and automated coding methods can be used. In Kenya’s agricultural cooperatives, enumerators may code responses on the field using mobile data collection apps.
• Handling Multiple-Response Questions: Such questions require assigning multiple codes per respondent, which must be carefully managed to avoid data loss.
• Dealing with Missing or Ambiguous Responses: Special codes (e.g., 99 for "No Response") are assigned to distinguish these cases during analysis.
• Ensuring Mutually Exclusive Codes: Codes should be distinct and non-overlapping to avoid confusion during analysis.

Data Categorization

Categorization involves grouping continuous or detailed data into meaningful classes or categories, facilitating summary and comparison. For instance, income data collected from county government employees can be categorized into income brackets.

• Defining Categories Based on Research Objectives: Categories should reflect the analysis goals, such as grouping ages into youth, middle-aged, and elderly.
• Using Standard Classification Systems: Employing national or international standards, such as Kenya’s Standard Classification of Occupations, ensures comparability.
• Determining Class Widths: For numerical data, deciding on appropriate intervals (e.g., 0–10,000 Ksh, 10,001–20,000 Ksh) to balance detail and clarity.
• Ensuring Exhaustive and Mutually Exclusive Categories: Each data point should fit into one category without overlap.
• Labeling Categories Clearly: Category labels must be descriptive to aid interpretation and reporting.

Data Categorization

Data categorization involves grouping continuous or detailed data into meaningful classes or categories, which facilitates summary, comparison, and further statistical analysis. Kenyan applied statisticians often use categorization to make large datasets manageable and interpretable, especially when working with demographic or financial data.

• Defining Categories Based on Research Objectives: Categories should be tailored to the goals of the research, such as grouping ages into youth, middle-aged, and elderly in a county government workforce survey.
• Using Standard Classification Systems: Employing recognized systems like Kenya’s Standard Classification of Occupations or the International Standard Industrial Classification (ISIC) ensures comparability across studies and institutions.
• Determining Class Widths: For numerical data, appropriate intervals (e.g., income brackets of 0–10,000 Ksh, 10,001–20,000 Ksh) are chosen to balance detail and clarity, as seen in Central Bank of Kenya financial surveys.
• Ensuring Exhaustive and Mutually Exclusive Categories: Each data point should fit into one category without overlap, such as assigning each patient to a single diagnostic group in a Kenyatta National Hospital study.
• Labeling Categories Clearly: Descriptive labels help stakeholders interpret findings, for example, labeling employment status as 'Permanent', 'Contract', or 'Casual' in a Kenya National Bureau of Statistics (KNBS) labor force survey.

7.1.3 Data Organization and Summarization

Organizing data efficiently is essential for effective analysis. Summarizing data reduces complexity, making patterns and trends easier to identify.

Organizing Data

Data organization involves structuring the dataset into formats that facilitate analysis and visualization.

• Frequency Distribution Tables: Presenting data as classes with corresponding frequencies helps summarize large datasets. For example, a frequency table showing the number of patients per age group at a county hospital.
• Cross-Tabulations: These tables display relationships between two or more variables, such as gender and employment status among university graduates.
• Data Matrices: Structuring data in rows and columns where rows represent cases and columns represent variables, commonly used in Excel or statistical packages.
• Use of Software Tools: Programs like SPSS, Stata, or R are employed to organize and manage data efficiently.
• Data Storage and Backup: Proper storage in secure databases or cloud services protects data integrity and allows easy retrieval.

Data Summarization

Summarization techniques simplify data presentation and support initial insights before detailed analysis.

• Measures of Central Tendency: Calculating mean, median, and mode to identify typical values in datasets such as household incomes in a retail business survey.
• Measures of Dispersion: Including range, variance, and standard deviation to understand data variability.
• Graphical Summaries: Employing bar charts, histograms, and pie charts to visualize distributions and proportions.
• Use of Stem-and-Leaf Plots: This technique preserves original data values while showing distribution, useful in exploratory data analysis.
• Tabular Summaries: Presenting key statistics in tables for quick reference by decision-makers in institutions like the National Hospital Insurance Fund (NHIF).

7.1.4 Data Presentation Formats and Their Effectiveness

Choosing the appropriate format for data presentation enhances comprehension and communication of research findings. Different formats serve distinct purposes depending on the nature of the data and the audience.

Types of Data Presentation Formats

• Tables: Ideal for detailed numerical data and reference purposes. For example, a frequency distribution table showing the number of students per course in a university.
• Bar Graphs: Effective for comparing quantities across categories, such as monthly sales figures in a retail business.
• Line Graphs: Useful for illustrating trends over time, like daily patient visits at a county hospital.
• Pie Charts: Best for showing simple compositions or percentages, for instance, market share distribution among SACCOs.
• Histograms: Display frequency distributions of continuous data and reveal the shape of data, such as age distribution of employees in a bank.
• Stem-and-Leaf Plots: Combine graphical and numerical data presentation, maintaining original data while showing distribution.

Effectiveness of Different Formats

• Clarity: Tables are clear for detailed data but can be overwhelming with large datasets; graphs simplify complex data visually.
• Comparisons: Bar graphs excel in side-by-side comparisons of categories, which is beneficial in project monitoring reports for county governments.
• Trend Analysis: Line graphs effectively show changes over time, aiding financial analysts at insurance firms in forecasting.
• Composition Representation: Pie charts succinctly represent parts of a whole but are limited to simple datasets.
• Data Exploration: Stem-and-leaf plots allow statisticians to detect patterns and outliers during exploratory phases.
• Audience Consideration: Technical audiences may prefer tables and detailed statistics, while policymakers benefit from graphical summaries.

Chapter Summary

This chapter explored the essential stages involved in handling research data after collection, beginning with data processing which involves organizing and preparing raw data for analysis by cleaning and coding it to ensure accuracy and usability. It then examined data management practices that focus on the systematic storage, protection, and retrieval of data to maintain its integrity throughout the research lifecycle. Finally, the chapter addressed data analysis, emphasizing the interpretation of processed data to extract meaningful insights and conclusions, particularly in the context of written tests. Together, these stages form a critical continuum that transforms raw information into reliable evidence to support research findings. Effective data processing, management, and analysis are indispensable for producing valid and trustworthy results. The chapter highlighted the interconnectedness of these processes in enabling researchers to draw accurate conclusions from their investigations.