Evaluating social services is a vital function for child protection professionals in Kenya, as it determines the quality, effectiveness, and accountability of interventions provided to vulnerable children and their families. Rigorous evaluation ensures that limited resources are allocated efficiently, and that services truly meet the needs of children in diverse settings, from county hospitals to local community-based organizations. Kenyan child welfare practitioners must be familiar with a variety of evaluation tools, understand the different types of evaluation, and develop robust instruments that suit both urban and rural contexts. Mastery of these processes is essential for improving practice, securing donor funding, and upholding the rights of children in line with national and international standards.

5.6 Evaluation Tools

Evaluation tools are the instruments and methodologies used to systematically assess the performance, outcomes, and impacts of child welfare services. In the Kenyan child protection context, selecting and utilizing the right tools ensures that assessments are reliable, culturally appropriate, and capable of capturing the realities faced by children and service providers. This topic explores the different types of evaluation tools, their specific applications, and how child protection practitioners can adopt or adapt these tools for varied service environments.

5.6.1 Types of Evaluation Tools

A comprehensive understanding of evaluation tools enables child protection professionals to collect, analyze, and interpret data effectively. The selection of tools depends on the objectives of the evaluation, the characteristics of the target population, and the available resources. In child welfare services, tools must be sensitive to children’s rights, confidentiality, and cultural diversity across Kenya.

Common Evaluation Tools in Child Welfare Services

• Surveys and Questionnaires: Structured sets of questions used to gather quantitative or qualitative information from children, caregivers, or service providers. For example, a county children’s office may use client satisfaction questionnaires to gauge perceptions of case management services.
• Interviews: Face-to-face or virtual conversations designed to elicit detailed responses about experiences, attitudes, or outcomes. Social workers in Nairobi may conduct in-depth interviews with children who have been reunified with their families to assess reintegration success.
• Focus Group Discussions: Guided group conversations that explore collective views, challenges, and suggestions related to child welfare programs. NGOs supporting street-connected children often use focus groups to understand barriers to accessing shelter services.
• Observation Checklists: Systematic lists used to record observable behaviors, conditions, or interactions during site visits. In children’s homes, staff may use checklists to monitor hygiene standards or staff-child interactions.
• Case File Reviews: Systematic examination of case documentation to evaluate service delivery, decision-making, and compliance with protocols. Supervisors in county departments may audit case files to ensure adherence to the Children Act and best practices.

Key Attributes of Effective Evaluation Tools

• Validity: The tool accurately measures what it is intended to assess, such as child wellbeing or service satisfaction.
• Reliability: The tool yields consistent results when used by different evaluators or at different times.
• Cultural Sensitivity: The tool respects local customs, languages, and beliefs, making it suitable for diverse Kenyan communities.
• User-Friendliness: The tool is easy for both evaluators and respondents to understand and use.
• Confidentiality: The tool safeguards the privacy of children and families, complying with legal and ethical standards.

How to Match Tools to Evaluation Contexts

• Routine Monitoring: Quick checklists and client feedback forms are suitable for ongoing service monitoring in busy children’s clinics.
• Outcome Evaluation: In-depth interviews and case file reviews are more appropriate for assessing long-term impact, such as the effectiveness of foster care programs.
• Participatory Evaluation: Focus groups work well when involving children and caregivers in the evaluation process, especially in participatory action research.

Limitations and Challenges

• Language Barriers: Some tools may not be available in local languages, limiting inclusivity in rural counties.
• Resource Constraints: Inadequate staffing or funding can restrict the use of comprehensive tools such as longitudinal surveys.
• Bias and Subjectivity: Interviews and focus groups may be influenced by the interviewer’s or participants’ biases.
• Data Security Risks: Poorly managed tools can lead to breaches of confidential child protection data.
• Training Needs: Effective use of certain tools, like structured observation, requires specialized training.

5.6.2 Types of Evaluation

Child welfare services require different types of evaluation to ensure that interventions are not only delivered as planned but are also achieving desired outcomes and adapting to changing needs. Each type of evaluation serves a unique purpose and is conducted at different stages of program implementation. Understanding these types is essential for managers, practitioners, and funders in the Kenyan child protection sector.

Formative Evaluation

• Purpose: Conducted during program development or early implementation to improve design and processes.
• Application: Used by NGOs piloting new child protection outreach programs in informal settlements to refine service delivery before scaling up.
• Methods: May include needs assessments, pilot testing of interventions, or feedback from stakeholders.
• Benefits: Identifies potential barriers and enhances program relevance.
• Limitations: May require additional time and resources before full rollout.

Process Evaluation

• Purpose: Assesses how services are delivered, focusing on fidelity to protocols and quality of implementation.
• Application: County children’s departments may evaluate whether caseworkers are following standard procedures for child rescue operations.
• Methods: Involves reviewing activity logs, observing staff, or analyzing service delivery records.
• Benefits: Detects operational challenges and facilitates corrective action.
• Limitations: May not capture ultimate outcomes for children.

Outcome Evaluation

• Purpose: Measures the immediate and intermediate effects of services on beneficiaries.
• Application: A hospital social work team may assess the reduction in child neglect cases following a parenting education program.
• Methods: Utilizes pre- and post-intervention surveys, child wellbeing assessments, or service utilization data.
• Benefits: Provides evidence of positive change attributable to the intervention.
• Limitations: Can be influenced by external factors outside the service’s control.

Impact Evaluation

• Purpose: Examines the long-term, sustained effects of a program or policy on target populations.
• Application: National-level studies by the Ministry of Labour and Social Protection may evaluate the impact of cash transfer programs on child poverty rates.
• Methods: Involves longitudinal studies, control groups, or comparison with baseline data.
• Benefits: Offers robust evidence for policy advocacy and funding decisions.
• Limitations: Time-consuming and often costly.

Summative Evaluation

• Purpose: Provides a comprehensive assessment of a program’s overall effectiveness after completion.
• Application: Used by donor agencies to decide whether to continue funding a community-based child protection project.
• Methods: Combines findings from outcome, impact, and process evaluations.
• Benefits: Supports accountability and strategic decision-making.
• Limitations: Does not inform ongoing improvements during program implementation.

5.6.3 Steps in Developing Effective Evaluation Tools

Developing effective evaluation tools is a structured process that ensures the tools are tailored to the needs of the child welfare program, are practical, and generate actionable data. Child protection professionals should involve stakeholders, pilot instruments, and continually refine tools to reflect lessons learned and emerging challenges in Kenya’s dynamic social service landscape.

Key Steps in Developing Evaluation Tools

1. Define the Purpose and Scope: Clearly articulate what the evaluation seeks to measure, such as service quality, client satisfaction, or program outcomes. For example, a children’s home may focus on evaluating the effectiveness of its reintegration services.
2. Engage Stakeholders: Involve children, caregivers, frontline staff, and funders in tool design to ensure relevance and buy-in. Stakeholder input from county governments and local CBOs can highlight context-specific concerns.
3. Select the Appropriate Tool Type: Choose between surveys, interviews, checklists, or mixed methods based on objectives, available resources, and respondent profiles. Urban-based organizations may opt for digital surveys, while rural programs may require oral interviews.
4. Draft and Adapt Questions or Indicators: Develop questions or indicators that are specific, measurable, and culturally appropriate. Questions should be piloted in both Kiswahili and local languages where necessary.
5. Pilot Test the Tool: Test the draft tool with a small group similar to the target population to identify confusing items, cultural issues, or logistical barriers. Adjust the tool based on feedback before full-scale use.
6. Train Data Collectors: Ensure all evaluators are trained on tool administration, ethical considerations, and data protection. Hospitals and schools often require refresher training for staff involved in sensitive child welfare evaluations.
7. Implement the Tool and Monitor Use: Roll out the tool systematically, monitoring for challenges such as non-response or data entry errors. Supervisors should check for consistent application across teams.
8. Analyze Data and Refine the Tool: Review collected data for completeness and accuracy, and revise the tool to address any gaps or weaknesses identified during analysis.

Field Practice: Piloting for Cultural Relevance

When developing a satisfaction survey for caregivers in a Kisii-based children’s program, staff piloted questions in both English and Ekegusii. Caregivers found some terms unclear, leading to revisions that improved response rates and the quality of feedback collected.

Chapter Summary

This chapter explored the critical aspects of evaluating social services within child welfare programmes, beginning with an overview of evaluation tools and their various types and purposes. Different forms of evaluation were discussed, highlighting how each approach contributes to assessing service effectiveness. The chapter detailed the steps required to develop robust evaluation tools, ensuring that they are tailored to specific objectives. Procedures for administering these tools were outlined, with emphasis on strategies to achieve accurate and reliable data collection. The importance of data analysis was examined, alongside common techniques used to interpret information gathered during evaluation. Guidance was provided on preparing comprehensive reports, covering the essential components and effective methods for sharing findings with stakeholders. The chapter also addressed the significance of proper document management, describing the essentials of a well-organized filing system and reviewing common filing methods to support efficient recordkeeping in child welfare services.