Nicole Maestas - An Outcome Evaluation of the Success for Kids Program

CHAPTER ONE
Introduction

CHAPTER TWO
Program Description

CHAPTER THREE
What Do We Know About After-School Programs?

CHAPTER FOUR
Evaluation Design and Implementation

CHAPTER FIVE
Data Quality

CHAPTER SIX
Outcome Analysis

CHAPTER SEVEN
Conclusions and Recommendations

APPENDIX
Training of Teachers and Facilitators

4.1. Location, Study Enrollment, and Neighborhood Characteristics of Participating
Sites

4.4. Logistic Regression of Treatment Assignment on Baseline Characteristics: Child
Reports

4.9. Logistic Regression of Treatment Assignment on Baseline Characteristics: Child
Reports, Nonattritors Only

This report presents results from a multisite, quantitative evaluation of the international Success for Kids (SFK) after-school program. The program seeks to build resilience in children by teaching them to access inner resources and build positive connections with others. The level I SFK program, titled The Game of Life, is a 10-part course offered weekly in 90-minute lessons free of charge to children between the ages of 6 and 14 who are enrolled in existing after-school programs. The course uses structured games and activities to teach such concepts as cause and effect, how to control reactive behaviors, the value of sharing with others, and the importance of making an effort. Through its curriculum, the program aims to increase prosocial behaviors and reduce antisocial and problem behaviors. The SFK program is unlike most after-school programs both in its focus on spiritual development and in its emphasis on outcomes related to resilience rather than academics.

Although after-school programs in the United States receive significant financial backing from both public and private sources, the literature assessing their effectiveness has many methodological weaknesses. Among the better studies of programs addressing social behaviors (such as SFK), program effects have been shown to be small. This study attempted to avoid the pitfalls evident in the research literature by using a very rigorous methodology: randomization of program sites to treatment and control groups and repeated measurements over time. This research design was possible owing to a unique window of opportunity: SFK was rapidly expanding its operations in southeast Florida, which enabled us to randomly assign 19 new program sites to either immediate implementation of the program or delayed implementation after a 12-week waiting period. During the waiting period, the delayed-implementation sites formed a control group for the sites receiving immediate implementation (the treatment group). The 19 program sites were diverse both in their demographic composition and in their spatial location throughout Miami-Dade, Broward, and Palm Beach counties.

Behavioral outcomes were measured using the well-known Behavior Assessment System for Children, Second Edition (BASC-2). Children and their regular after-schoolprogram teachers (not the SFK teachers) were assessed at pretest, posttest, and follow-up. An extensive analysis of reliability and validity revealed that the self-report data collected from children were plagued with inconsistencies, and thus conclusions are based on the data collected from teachers.

We found that the program had beneficial effects on virtually every domain covered by the BASC-2. In particular, the program had medium to large effects on adaptive skills (effect sizes [ESs] of 0.55 to 0.73), which include adaptability, social skills, leadership, study skills, and communication skills. The program had small to medium effects on behavioral problems (0.19 to 0.37), especially attention problems and withdrawal, and small effects on overexternalization of problems (0.16 to 0.29). The program had small to medium effects on the reported incidence of school problems (0.32 to 0.48). There is suggestive evidence that many effects persisted at 12-week follow-up. Notably, the ESs exceeded the average ESs found for after-school programs targeting similar outcomes (0.19 for positive social behaviors and 0.18 for problem behaviors), as calculated by Durlak and Weissberg (2007) in a recent meta-analysis.