UKZN SOCY306: Programme Evaluation for Social Interventions Study Notes

Programme evaluation is the systematic process of judging whether social interventions achieve intended outcomes, why they succeed or fail, and what should be changed to improve impact. In SOCY306 (Programme Evaluation for Social Interventions) at UKZN, the emphasis is typically on applying evaluation logic to real-world social issues—using theory, methods, data, ethics, and implementation realities to produce credible, decision-useful findings. These notes build a coherent toolkit for planning and conducting evaluations in South Africa’s social development and public-sector contexts, where constraints (capacity, data systems, political oversight, inequality, and diverse communities) shape what “good evaluation” looks like.

Section 1: Foundations of Programme Evaluation for Social Interventions (UKZN SOCY306)

Programme evaluation differs from general “assessment” or “monitoring” because it explicitly asks evaluative questions about effectiveness, relevance, efficiency, and often equity—not only whether activities happened. For social interventions—such as youth employment programmes, gender-based violence (GBV) prevention initiatives, community mental health supports, or school-based learner retention projects—evaluation must also grapple with complex, multi-causal pathways. SOCY306 therefore treats evaluation not just as a technical exercise but as an applied social research practice embedded in policy and organisational settings.

1.1 Key concepts: intervention, programme, theory of change, and logic models

A social intervention is an organised attempt to change social conditions or behaviour (for example, improving parenting practices, reducing substance abuse relapse, or increasing service uptake among vulnerable groups). A programme is a specific set of activities and resources delivered to a target population under defined conditions.

To evaluate programmes, evaluators often use a logic model (sometimes called a results chain) that links:

Inputs (funding, staff, facilities, partnerships),
Activities (training, counselling sessions, outreach),
Outputs (number of sessions delivered, people reached, materials produced),
Outcomes (changes in knowledge, attitudes, service access, well-being),
Impacts (longer-term changes such as reduced harm, improved livelihoods, better health indicators).

Because social outcomes are rarely caused by one programme alone, SOCY306 evaluations commonly rely on a theory of change (ToC). A ToC makes assumptions explicit: If participants receive the intervention then they will do X because of Y mechanisms, and external conditions will not block the pathway, therefore outcomes Z will occur. This is crucial in South Africa, where programmes operate in environments shaped by unemployment, household poverty, service delivery gaps, and spatial inequalities.

Example scenario (GBV intervention):
A programme trains community volunteers to facilitate monthly group sessions on consent and safe relationship skills. The logic model might predict that participants gain knowledge (output and immediate outcome), then adopt safer practices (intermediate outcome), then experience reduced risk of violence (impact). The ToC would articulate mechanisms (social norms shifting through peer influence, improved self-efficacy, pathways to support services), and assumptions (participants can access safety resources; police respond effectively; sessions are culturally appropriate).

1.2 Monitoring vs evaluation: what distinguishes them?

While monitoring and evaluation are linked, they serve different purposes:

Monitoring tracks performance and implementation: whether planned activities occurred, and with what quality and frequency.
Evaluation assesses value and change: whether the programme achieved intended outcomes/impacts, and why.

A programme may be well-monitored but poorly evaluated. For SOCY306, learners should be able to distinguish evaluative questions such as:

“Did the programme increase employment among youth?” (evaluation)
“How many recruitment workshops were delivered, and did attendance targets meet?” (monitoring)

In social interventions, evaluation often uses mixed methods—quantitative measures (e.g., employment status at 6 months) and qualitative insights (e.g., participant experience of barriers to job retention).

1.3 Why evaluate? Accountability, learning, and social justice

Evaluation serves multiple purposes:

Accountability: funders and government departments require evidence that resources produce results.
Learning and improvement: programmes need feedback to adjust design, targeting, delivery, and staffing.
Equity and ethical responsibility: social interventions can inadvertently harm or exclude groups. Evaluation should ask who benefits, who does not, and whether effects differ by subgroup.

In South Africa, the ethical stakes are heightened due to historically entrenched inequality and the possibility that programmes may reinforce stigma, exclude informal settlements, or over-target certain communities while under-serving others. In an evaluation plan, “success” should not only mean average improvement; it must also consider distributional effects.

1.4 Evaluative criteria in practice: relevance, effectiveness, efficiency, sustainability, and equity

SOCY306 commonly draws on evaluative dimensions such as:

Relevance: Does the programme address real needs of the target population and align with policy priorities?
Effectiveness: Did it achieve outcomes as intended?
Efficiency: Were outcomes achieved with reasonable resources relative to alternatives?
Sustainability: Are benefits likely to continue after funding ends? Is capacity built?
Equity: Are impacts fair across gender, age, disability status, language group, geographic area, and socioeconomic status?

Concrete illustration:
Suppose a substance abuse support programme provides weekly counselling sessions. It is effective for participants in urban centres where transport is available. It may be less relevant or less effective for participants in rural areas if there is no transport support. Equity analysis might therefore show that impact differs by geography, prompting a redesign (e.g., mobile outreach or transport stipends).

1.5 Evaluation questions and hypotheses: moving from description to judgement

Evaluative questions translate stakeholder concerns into answerable forms. A good question is specific about:

Population (who),
Programme components (what),
Outcomes (what change),
Time horizon (when),
Comparison (relative to what).

Examples of evaluation questions:

“To what extent did the youth skills programme improve employment outcomes after 12 months compared to similar youth who did not participate?”
“How do participants describe changes in household decision-making after the parenting intervention?”
“Which implementation factors (staff training, outreach intensity, referral partnerships) predict better outcomes?”

In theory-driven evaluation, questions may also test hypotheses grounded in ToC mechanisms:

“If participants gain reliable information and support, then they will increase service uptake—especially among first-time clients.”

1.6 Practical constraints in South African evaluation settings

SOCY306 also prepares you for real-world limitations:

Data quality and availability: baseline data may be missing; administrative data can be incomplete.
Attrition and tracking: especially for mobile populations (young people moving for work).
Ethical constraints and consent: evaluation research must protect participants, particularly in vulnerable settings (GBV survivors, adolescents).
Implementation capacity: monitoring tools may be introduced but not maintained.
Political and organisational pressures: results may be expected to support continued funding.

A credible evaluation plan anticipates these constraints. For example, if baseline data are missing, evaluators may use retrospective baseline reconstruction or comparison strategies that rely on available proxy indicators. If attrition is high, sampling and follow-up protocols must be adjusted and tracked carefully.

Section 2: Research Designs, Sampling, and Measurement in Programme Evaluation (UKZN SOCY306)

To evaluate social interventions responsibly, evaluators must connect evaluative criteria to rigorous research designs and measurement strategies. SOCY306 typically expects understanding of design types (experimental, quasi-experimental, non-experimental), sampling logic, and the practical process of creating outcome measures that reflect social realities.

2.1 Choosing an evaluation design: experimental, quasi-experimental, and non-experimental approaches

Experimental designs (e.g., randomized controlled trials) create strong causal inference by randomly assigning participants to treatment and comparison groups. In social policy contexts, randomization may be difficult due to ethical concerns (withholding services) or administrative feasibility. However, some programmes can use phased rollouts.

Quasi-experimental designs attempt causal inference without random assignment, using strategies like:

comparison groups matched on observable characteristics,
difference-in-differences (DiD),
regression discontinuity (if eligibility thresholds exist),
interrupted time series,
propensity score matching (PSM) or weighting.

Non-experimental designs use:

pre-post comparisons (with caution),
single group designs with triangulation,
qualitative evaluations focused on understanding mechanisms and context.

In SOCY306, the central skill is not memorising design labels but selecting the design that best fits the programme context, ethics, resources, and decision needs. Often, social intervention evaluations use mixed-methods designs: quantitative designs for outcomes and qualitative methods for mechanisms and contextual explanations.

2.2 A mixed-methods evaluation logic: triangulation and complementarity

Mixed methods are valuable when:

outcome measures are uncertain,
mechanisms are complex,
implementation is context-dependent,
numbers alone do not reveal why outcomes changed.

A typical mixed-methods evaluation process:

Use quantitative data to estimate changes in outcomes (e.g., service uptake rates, employment status).
Use qualitative data (interviews, focus groups, case studies) to understand the “how” and “why.”
Integrate findings to interpret causal plausibility and guide programme improvement.

Example (school-based learner support):
Quantitative data may show improved learner attendance. Qualitative data may reveal that improved communication with caregivers, not the tutoring sessions alone, changed attendance behaviour. Without qualitative insight, evaluators might incorrectly attribute effects to the wrong mechanism.

2.3 Causal inference and confounding: threats to validity

A major concern in programme evaluation is attribution: whether changes in outcomes are due to the programme or due to other factors. Key threats to validity include:

Selection bias: participants differ systematically from non-participants (motivation, prior access, community networks).
History effects: external events influence outcomes (e.g., labour market changes, policy shifts).
Maturation: natural changes over time unrelated to intervention.
Regression to the mean: participants chosen for high risk may later “improve” naturally.
Instrumentation changes: measurement tools change between baseline and endline.
Contamination: comparison group receives part of the programme.
Attrition: those lost to follow-up differ from those retained.

SOCY306 requires evaluators to name threats and respond with design choices and statistical controls. For example, if selection bias is likely, matching or DiD might reduce confounding; if contamination is possible, documentation of exposure and careful group definitions are essential.

2.4 Sampling strategies for evaluation studies

Sampling depends on evaluation design and practical constraints. Common approaches:

Probability sampling for surveys to estimate outcome levels with known sampling error.
Purposive sampling for qualitative interviews to capture diverse experiences.
Stratified sampling when subgroup differences matter (e.g., by gender, district, language, disability status).
Cluster sampling if programme delivery is geographically clustered (schools, communities).

For evaluation, a crucial distinction is between:

sampling evaluation participants (e.g., those receiving programme services),
sampling respondents for outcome measurement (which may include non-participants for comparison).

Case example (community employment programme):
If the programme is delivered in selected wards, then clustering is likely. A sample may need to include participants within programme wards and comparison participants in similar non-programme wards. Stratification might ensure adequate representation of women and young men, given gendered impacts of job-seeking and training.

2.5 Sample size, power, and feasibility

While SOCY306 may not require advanced power calculations in every assessment, you should understand the logic:

Detecting small effects requires larger samples.
Attrition reduces effective sample size.
Multi-outcome evaluations require balancing depth with statistical detectability.

In many social intervention contexts, feasible sample sizes are constrained. Evaluators therefore use:

conservative effect expectations,
robust standard errors,
careful measurement,
mixed methods that can detect nuanced mechanism changes even if average quantitative effects are modest.

Practical approach:
If an endline follow-up of youth training participants is expected at 80% due to tracking challenges, and the initial sample is N=500, then the expected retained sample is 0.80 × 500 = 400. The evaluation analysis must base power assumptions on the retained number, not the baseline recruitment number.

2.6 Operationalisation and measurement: indicators that mean what you think they mean

A major SOCY306 theme is operationalisation: translating abstract outcomes into measurable indicators. For example:

“Improved empowerment” can be measured via decision-making scales, confidence ratings, or behavioural proxies (depending on validity).
“Reduced violence risk” could involve reported incidents, help-seeking behaviour, perceptions of safety, or changes in attitudes.

Measurement quality requires:

validity (does the indicator measure the intended construct?),
reliability (are measures consistent?),
sensitivity to change (can it detect improvement over the evaluation period?),
cultural and linguistic appropriateness (translation and meaning).

In South Africa, measurement must be sensitive to language diversity and different cultural understandings of constructs. For example, “respect” in interpersonal relationships may be interpreted differently across communities; survey items should be tested through piloting and cognitive interviewing where possible.

2.7 Building an outcomes matrix: from objectives to indicators

An outcomes matrix (sometimes called an evaluation framework) aligns:

objectives,
indicators,
data sources,
frequency,
methods,
responsible persons.

Example outcomes matrix snippet (parenting programme):

Objective	Indicator	Data Source	Frequency
Improve positive parenting practices	% of caregivers demonstrating consistent non-violent discipline strategies	caregiver survey + observation checklist (subset)	baseline, midline, endline
Increase child well-being	child behaviour problems score (validated tool)	caregiver report	endline
Improve linkage to services	% caregivers using parenting support or child health services	service utilisation records	endline

Such matrices prevent “indicator drift,” where evaluation teams collect data that are easy rather than meaningful. SOCY306 emphasises alignment: outcomes must correspond to programme activities and ToC mechanisms.

2.8 Ensuring measurement ethics: avoiding harm and protecting privacy

Measurement in social interventions can be sensitive. Evaluation must:

use informed consent,
minimise distress (e.g., careful wording for GBV-related questions),
ensure confidentiality,
provide referrals if respondents disclose urgent needs.

In GBV contexts, evaluations must avoid procedures that increase risk to participants (e.g., collecting data in places where perpetrators could overhear). Evaluators must also consider data security and secure storage of identifiable information.

Section 3: Implementing Evaluations: Data Collection, Analysis, and Theory-Driven Interpretation (UKZN SOCY306)

After designing an evaluation, SOCY306 focuses on implementation: collecting credible data, analysing results, and interpreting findings through theory, context, and stakeholder perspectives. This section develops practical steps and detailed reasoning for credible programme evaluation.

3.1 Evaluation planning process: milestones and responsibilities

A rigorous evaluation plan typically includes:

Stakeholder mapping and engagement
Identify decision-makers (funder, implementing agency, government unit), programme managers, frontline staff, and beneficiaries. Clarify who will use which results, and how findings will be communicated.
Define evaluation purpose and scope
Is the purpose formative (improve implementation), summative (judge effectiveness), or both? Determine time horizon and which programme components are evaluated.
Finalise evaluation questions
Example categories:
- Process evaluation questions (what happened, how was it delivered?),
- Outcome evaluation questions (what changed?),
- Mechanism/context questions (why did change happen or not happen?).
Develop ToC and evaluation framework
Map indicators to ToC pathways and assumptions.
Develop instruments
Survey questionnaires, interview guides, observation protocols, data extraction tools.
Pilot testing
Check understanding, cultural appropriateness, and logistical feasibility.
Data management plan
Data storage, anonymisation, naming conventions, handling missing values.
Ethical review and consent procedures
Ensure compliance with ethics requirements (including confidentiality and referral mechanisms).
Analysis and reporting plan
Decide how results will be integrated (quant + qual), and how uncertainty will be presented.

This planning structure matters in South Africa because implementation partners may have limited evaluation capacity; a clear plan clarifies roles and reduces risk of incomplete data collection.

3.2 Process evaluation: understanding implementation fidelity, reach, and dose

Programme outcomes often depend on whether programmes are delivered as intended. Process evaluation examines:

Fidelity: Were activities delivered according to protocol?
Reach: Did the programme reach the intended target group?
Dose delivered: How much of the intervention was provided?
Dose received: Did participants actually engage and benefit from the delivered content?
Implementation quality: Were facilitators trained, supportive, and consistent?
Context: What external factors affected delivery?

Illustrative case (youth entrepreneurship training):
A programme might deliver 10 training sessions. If participants attend only 4 sessions on average (dose received), outcomes such as business plan completion may be low even if the training content is high quality. Process evaluation data—attendance records, facilitator logs, participant engagement measures—can reveal the “why” behind outcome results.

3.3 Data collection methods: surveys, administrative data, interviews, and observational tools

Common data sources in SOCY306 include:

Surveys (structured questionnaires): useful for outcomes and baseline/endline comparisons.
Administrative records: attendance registers, service referrals, employment placement records.
In-depth interviews: explore experiences, perceptions, barriers.
Focus group discussions: capture group-level norms and collective perspectives.
Participant observation: assess implementation quality (especially for delivery processes).
Document analysis: programme reports, policy documents, training manuals.

Best practice: use triangulation. If survey results show improved outcomes, interviews can explain mechanisms and detect unintended effects (e.g., increased conflict due to family expectations). If outcomes show no change, qualitative data can reveal implementation gaps or mismatch between programme design and participant needs.

3.4 Handling missing data and measurement errors

Missing data is a common evaluation challenge. SOCY306 expects awareness of:

missing completely at random (MCAR),
missing at random (MAR),
missing not at random (MNAR) — where missingness correlates with unobserved outcomes.

Practical steps:

Document reasons for missingness (tracking difficulties, refusal, relocation).
Use appropriate strategies (e.g., multiple imputation where justified; or sensitivity analysis).
Report missingness rates transparently by group.

Measurement errors occur when respondents misunderstand items or when scales are not culturally aligned. Piloting and training of fieldworkers mitigate errors.

3.5 Quantitative analysis: estimating change, differences, and uncertainty

Quantitative analysis depends on design. Common tasks:

Compute baseline and endline summary statistics by group.
Estimate changes over time (e.g., difference in means).
For quasi-experimental designs, incorporate controls and matching variables.
Use regression models when feasible to adjust confounding.

Analytical outputs should interpret:

magnitude (effect size) rather than only statistical significance,
confidence intervals (uncertainty),
subgroup patterns (e.g., differences by gender or district).

Subgroup analysis caution: if sample sizes are small, subgroup estimates can be unstable. SOCY306 evaluations should therefore interpret subgroup differences carefully and avoid overclaiming.

3.6 Qualitative analysis: coding, themes, and mechanism building

Qualitative data analysis often involves:

Transcription and organisation.
Coding (thematic or framework-based).
Developing themes and linking themes to ToC mechanisms and assumptions.
Comparative analysis across participant types and delivery contexts.

A strong evaluation report connects qualitative findings to:

whether mechanisms appear to have operated (e.g., changes in norms, improved self-efficacy, reduced barriers),
contextual facilitators and barriers,
differences between expected and observed pathways.

Example mechanism mismatch:
If the ToC assumes improved knowledge leads to improved service uptake, but interviews show participants have knowledge yet do not seek services due to fear of stigma, then the ToC pathway needs revision: stigma reduction and confidential access may be essential.

3.7 Integrating findings: triangulation and “so what” interpretation

Integration is the interpretive heart of theory-driven evaluation. Approaches include:

Triangulation: check whether quantitative and qualitative results align.
Convergence coding matrix: summarise where evidence converges or diverges across methods.
Explanatory integration: use qualitative findings to explain quantitative patterns (or lack of them).
Programme adjustment logic: translate evidence into actionable recommendations.

Unintended effects are important. A programme may reduce one problem while increasing another (e.g., a financial support scheme may trigger household conflicts). Evaluation should therefore ask:

Did any harm occur?
Were there trade-offs?
Did impacts differ across groups?

3.8 Equity-focused interpretation: analysing differential impact

Equity analysis requires asking whether impacts differ by:

gender,
age group,
disability status,
language,
rural/urban residence,
socioeconomic status.

Even when average results look positive, unequal access or different experiences can undermine equity goals. For instance, a training programme delivered in a central town may be less accessible to rural participants due to transport. Evaluation should therefore examine both who reaches and who benefits.

Section 4: Ethics, Stakeholders, and Evaluation Use in Public and Community Settings (UKZN SOCY306)

Programme evaluation is not value-neutral; it involves power, relationships, and decisions about what counts as evidence. SOCY306 therefore emphasises ethics, stakeholder engagement, and evaluation use: how findings influence policy, funding, and programme redesign.

4.1 Ethical principles in evaluation research

Key ethical principles include:

Respect for persons: informed consent, voluntary participation, right to withdraw.
Beneficence: minimise harm and maximise potential benefits.
Justice: fair selection of participants and fair distribution of burdens and benefits.
Confidentiality and privacy: protect identities and sensitive information.
Minimising risks in high-stakes contexts: especially for GBV, child protection, and mental health.

In South African contexts, ethics also includes cultural sensitivity and avoiding procedures that create suspicion or distrust. Fieldworkers may need to liaise with community structures while maintaining participant confidentiality.

4.2 Stakeholder engagement: empowering learning and shared ownership

Stakeholders include:

funders and government departments,
implementing NGOs or community organisations,
frontline staff and programme coordinators,
community leaders and structures,
beneficiaries (participants and non-participants).

Stakeholder engagement should:

clarify decision-use needs early (what decisions will evaluation inform?),
ensure cultural appropriateness,
encourage transparency about limitations.

Common failure mode: evaluation teams conduct surveys and interviews but do not feed results back to the community, producing distrust and limited learning. SOCY306 prepares learners to design communication plans and feedback sessions.

4.3 Managing power dynamics and ensuring meaningful participation

Beneficiaries may have limited power relative to implementers. This affects:

consent quality (are people pressured?),
response honesty (fear of repercussions),
interpretation of results (do participants have input into meaning?).

Ethical evaluation design includes:

training fieldworkers to avoid coercion,
ensuring confidentiality,
using participant-friendly materials (language, explanations),
creating avenues for participant feedback on findings.

4.4 Data governance and confidentiality

Evaluation often produces sensitive datasets. Data governance includes:

secure storage (password-protected files, encrypted drives),
controlled access (only evaluation team members),
anonymisation (removing identifiers where possible),
retention schedules aligned with ethics approval.

In addition, evaluators must ensure that reporting does not allow re-identification. In small communities, combinations of demographics may identify participants.

4.5 Evaluation credibility: transparency, limitations, and reporting integrity

Credibility depends on:

clear description of design and sampling,
transparent reporting of implementation fidelity and attrition,
clear explanation of analysis methods,
discussion of limitations (without undermining the usefulness of findings).

A credible report should not hide uncertainty. For example, if effects are small and confidence intervals include zero, the report should state that evidence suggests limited or no measurable impact under the current conditions, and propose improvements or further research.

4.6 Evaluation use: from evidence to decisions

Evidence must be translated into actions. Evaluation use depends on:

timing (interim feedback during implementation),
communication strategy (briefing notes for decision-makers; accessible summaries for communities),
relevance (answers to decision questions),
implementation readiness (whether recommendations are feasible).

SOCY306 typically highlights the difference between reporting and using:

Reporting: producing a technical document.
Using: changing programme design, budgets, or delivery protocols.

Strategy for use:

Produce interim learning briefs (e.g., at midline).
Conduct stakeholder debriefing sessions.
Provide actionable recommendations mapped to programme components and resource needs.
Create a recommendation tracking matrix (who will do what by when).

4.7 Conducting feedback responsibly: avoiding blame and promoting learning

Feedback can become politicised. If evaluation results are negative, implementers may feel blamed. Ethical evaluation use promotes a learning approach:

focus on systems and design rather than personal blame,
separate implementation shortcomings from participant culpability,
encourage co-creation of solutions.

For example, if outcomes are low due to poor referral partnerships, recommendations should address partnership strengthening and service integration rather than implying participants failed to “engage enough.”

4.8 Common evaluation pitfalls in social interventions

SOCY306 also prepares for typical pitfalls:

Overclaiming causality without appropriate design.
Ignoring context: concluding “programme doesn’t work” despite delivery changes or political instability.
Indicator misuse: treating outputs as outcomes (e.g., “we trained people” instead of “people changed behaviour”).
Neglecting equity: reporting average impact while ignoring subgroup harms.
Weak measurement: using unvalidated tools that do not capture constructs.
Ethical shortcuts: collecting sensitive data without adequate safeguards.

Recognising these pitfalls improves both evaluation quality and credibility.

Section 5: Writing, Delivering, and Reviewing Evaluation Outputs (and Applying Them to UKZN-Situated Social Work/Applied Sociology Contexts)

Evaluation is not complete when data are analysed; it becomes complete when results are communicated clearly, ethically, and in forms that decision-makers can use. This final section focuses on evaluation reporting, recommendation design, and practical application of SOCY306 ideas to programme contexts relevant to applied sociology, including training systems and social development programming in South Africa.

5.1 Structure of an evaluation report: what exam and workplace assessors expect

A high-quality evaluation report typically includes:

Executive summary
- purpose, methods, key findings,
- headline conclusions,
- recommendations.
Background and context
- problem statement,
- programme description,
- stakeholders and rationale.
Evaluation approach
- evaluation questions,
- ToC/logic model,
- design type,
- sampling and data sources,
- limitations.
Findings
Organised by evaluation criteria:
- relevance,
- effectiveness,
- efficiency,
- process/implementation,
- equity,
- unintended effects.
Conclusions and interpretation
- link findings back to ToC,
- explain mechanisms and context,
- assess plausibility of causal claims.
Recommendations
- actionable,
- prioritised,
- cost-aware or resource-aware,
- mapped to programme components.
Annexes
instruments, coding frameworks, additional tables.

SOCY306 learners should practice writing findings in a way that distinguishes:

what the data show,
what the data suggest,
what is hypothesised,
what is uncertain.

5.2 From findings to recommendations: ensuring recommendations are specific and feasible

Recommendations must be grounded in evidence and feasibility. A useful recommendation includes:

Problem identified (e.g., low attendance leading to low outcomes),
Evidence basis (attendance records, interview themes),
Proposed change (e.g., transport support or session timing adjustments),
Implementation steps (who does what),
Resources required (staff time, budget lines if appropriate),
Indicators for monitoring success (what to measure after change).

Example recommendation (attendance challenge):

Problem: participants attend fewer than half of scheduled sessions due to transport barriers.
Evidence: process evaluation shows average attendance below expected dose received; interviews confirm transport costs and timing conflicts.
Change: introduce transport stipends and relocate sessions to community venues closer to participants.
Follow-up indicators: average attendance increases; outcomes related to participation and knowledge improvement show measurable growth at endline.

5.3 Using theory of change in the reporting narrative

In SOCY306, reports should not merely list outcomes; they should explain pathways. A theory-driven reporting narrative might follow:

Expected mechanism: training increases self-efficacy.
Observed outcome: improved self-efficacy scores on survey.
Contextual factor: participants report fear of confidentiality breaches at local service sites.
Result: despite improved self-efficacy, service uptake does not increase.
Conclusion: revise programme component to strengthen confidentiality protocols and build safer referral pathways.

This approach ensures evaluation outputs inform programme redesign rather than ending with generic “recommendations.”

5.4 Case-style application: programme evaluation mini-scenarios

To demonstrate the integration of SOCY306 concepts, consider three evaluation mini-scenarios. While these are simplified, they reflect the typical logic required in exams and assignments.

Scenario A: Youth skills and employability programme

Programme description:
A youth employability programme offers short skills workshops plus career guidance delivered over 12 months. The target group is unemployed youth aged 18–24 in two urban districts. The evaluation aims to assess changes in employment status and training completion.

Key evaluation questions:

Did participants experience higher employment rates at 12 months compared to non-participants?
Which delivery factors (workshop completion, mentoring exposure) predict stronger outcomes?
Are effects different by gender?

Design choice:
A quasi-experimental matched comparison approach using baseline characteristics (education level, prior work experience, household income proxy) plus endline outcomes at 12 months.

Measurement indicators:

employment status (employed/self-employed/unemployed),
hours worked in last month (for those employed),
training completion rate,
self-reported job search intensity.

Process evaluation lens:
attendance logs and mentoring session records to determine dose received.

Likely integration:
Quantitative results might show modest increases in employment among participants; qualitative interviews might show improved interview confidence and networking access, but persistent barriers due to limited local vacancies. Equity analysis may show women benefit less if care responsibilities limit training attendance, prompting scheduling adjustments.

Scenario B: Parenting support for reducing child behaviour problems

Programme description:
A parenting support programme provides group sessions and home-based check-ins for caregivers of children aged 6–10. The evaluation examines whether parenting practices change and whether child behaviour problems reduce.

Key evaluation questions:

Do caregivers demonstrate improved non-violent discipline practices?
Does child behaviour problems score reduce by endline?
What contextual factors influence programme effectiveness?

Design choice:
A mixed-methods design with pre-post survey measures and a comparison group from a similar community receiving a lighter information-only intervention.

Qualitative focus:
focus groups exploring caregivers’ perceptions of parenting authority and stigma about accessing services.

Interpretation:
If outcomes improve but vary by caregiver education, the evaluation identifies whether programme materials need easier language or more targeted coaching. If some caregivers report fear of judgement by facilitators, recommendations include facilitator training for non-judgemental communication.

Scenario C: Community GBV prevention through volunteer facilitation

Programme description:
Community volunteers facilitate monthly sessions focusing on consent, healthy relationships, and referral pathways. The evaluation assesses whether participants’ attitudes shift and whether help-seeking behaviour increases.

Key evaluation challenges:
Ethical risks in GBV contexts require careful handling. Data collection must avoid re-traumatisation.

Design choice:
A theory-driven evaluation with qualitative case studies and a careful quantitative attitude survey. Where randomisation is not feasible, a comparison design based on similar communities may be used with careful exposure measurement.

Equity analysis:
Assess whether young women, older women, and men have different experiences and whether referral pathways are accessible across language groups.

Expected mechanism reasoning:
If participants show improved attitudes but no change in help-seeking, qualitative data may reveal that fear of retaliation or stigma blocks uptake. Programme redesign could incorporate trusted confidential referral points and strengthened safety planning resources.

These scenarios highlight how SOCY306 expects evaluation to connect design, measurement, process understanding, and interpretation to decisions.

5.5 Critical appraisal and peer review: how to score an evaluation in practice

In exams, you may be asked to critique evaluation proposals. A practical rubric for appraisal includes:

Clarity of evaluation questions: Are they specific, feasible, and decision-relevant?
Appropriateness of design: Does the design match causal inference needs?
Sampling logic: Are target groups and comparison groups justified?
Measurement validity: Are indicators linked to constructs and ToC?
Ethics: Are consent, privacy, and risk mitigation addressed?
Credibility: Are limitations transparent and handled?
Integration: Are quantitative and qualitative findings connected logically?
Use: Are recommendations actionable and mapped to programme components?

Applying such a rubric helps learners both to write stronger proposals and to interpret evaluation results critically.

5.6 Exam-focused skills: how to answer typical SOCY306 questions

While question formats vary, common SOCY306 assessment tasks include:

propose an evaluation design,
distinguish monitoring and evaluation,
explain validity and threats to attribution,
design indicators and an evaluation matrix,
write an evaluation report outline,
critique ethical considerations,
connect results to theory of change and recommendations.

A high-scoring exam answer typically:

Defines key terms precisely (e.g., effectiveness vs efficiency).
Justifies method choice (design must match causal question and context).
Links measurement to ToC (indicators must reflect mechanisms).
Shows awareness of limitations and how to mitigate them.
Ends with application (recommendations that follow from evidence).

5.7 South Africa relevance: how context shapes evaluation priorities

South Africa’s social intervention landscape influences evaluation emphasis. Common evaluation realities include:

reliance on administrative systems that may be incomplete,
resource constraints affecting data collection frequency and follow-up,
diverse languages and cultural meanings affecting measurement validity,
political oversight requiring careful reporting and stakeholder management,
spatial inequality affecting access and participation.

Therefore, SOCY306 evaluation proposals should not assume ideal conditions. They should include:

data quality assessments and mitigation,
culturally appropriate tools and piloting,
realistic tracking strategies,
ethical safeguards for vulnerable contexts.

5.8 Delivering evaluation outputs: briefing notes, dashboards, and stakeholder-ready formats

In practice, evaluation findings are communicated through multiple formats:

Executive summary and slide decks for decision-makers.
Technical annexes for researchers and analysts.
Briefing notes with key findings and recommended actions.
Infographics for community stakeholders (in accessible language).
Data dashboards for implementation teams (where systems exist).

While SOCY306 may emphasise report writing, exam answers can also benefit from demonstrating awareness of audience needs. A final recommendation is to present findings clearly:

use plain language,
present uncertainty,
avoid jargon without explanation,
specify who should act and what change should occur.

5.9 Final synthesis: the SOCY306 “evaluation logic” chain

Across all topics, SOCY306 can be summarised as a coherent chain:

Start with programme theory (logic model/ToC).
Translate theory into evaluative questions (relevance, effectiveness, efficiency, equity).
Choose design and sampling aligned to causal and decision needs.
Operationalise indicators and ensure measurement quality.
Collect data ethically using appropriate methods and safeguards.
Analyse results with awareness of validity threats and uncertainty.
Interpret findings through mechanisms and context (theory-driven integration).
Report honestly and transparently with limitations and equity considerations.
Convert evidence into feasible recommendations and ensure use through stakeholder engagement.

This chain helps learners avoid superficial evaluation (activities counted, outputs reported) and instead produce evidence that explains change—why it happens, for whom it happens, under what conditions, and what should be improved to enhance social impact.

Appendix-style quick reference (exam practice)

A. Monitoring vs evaluation (fast distinction)

Monitoring: “Are we doing what we planned?”
Evaluation: “Did it work, and why?”

B. A theory-driven ToC checklist

What outcome do we expect?
What mechanism is supposed to produce it?
What assumptions must hold?
What context could block or enable the pathway?

C. Equity questions to always ask

Who benefits and who does not?
Are there harms or unequal burdens?
Do outcomes differ by gender, age, disability, language, and location?

D. Reporting “must-haves”

Purpose, design, limitations
Clear findings linked to questions and indicators
Recommendations grounded in evidence and feasible actions

These quick references are meant to support revision and exam recall while the main text develops the conceptual and practical reasoning expected in SOCY306.