CG GLOBAL REPORT 5:
CONTINUOUS QUALITY IMPROVEMENT AND COLLABORATIVE LEARNING TO IMPROVE ADAPTABILITY AND SCALABILITY IN A PROFESSIONAL DEVELOPMENT PROGRAMME FOR PRE-SCHOOL TEACHERS IN CHILE
MARYCATHERINE ARBOUR, HIROKAZU YOSHIKAWA, FRANCIS ROMINA DURAN MELLADO, KAREN ASKOV ZERIBI, MARCELA MARIA MARZOLO MALAGARRIGA AND CATHERINE E. SNOW
If early childhood education (ECE) is to fulfil its promise of promoting equity and ensuring that all children develop to their fullest potential, efforts to improve the quality of ECE will need to be executed on a large scale. Any intervention intended for large-scale implementation must be adaptable to a diversity of contexts; scale-up of efficacious interventions across contexts is a central challenge in global ECE. This case study from Chile reports on a professional development intervention to improve ECE quality that incorporated continuous quality improvement (CQI) methods and collaborative learning. This innovative approach is designed to enable teachers to 1) adapt the intervention as they apply it while continuously monitoring its efficacy in their specific context, and 2) share their insights in networked learning communities to promote the spread, scale-up and sustainability of practice improvements. The case study describes the programme’s key innovations, lessons learned and implications for national and international policy.
Background and Context
THE CHILEAN ECD CONTEXT
Chile is one of South America’s most stable and prosperous nations. The average annual growth rate in gross domestic product (GDP) was more than 4% between 2011 and 2014, and GDP per capita is one of the highest in the region.1 It is also one of the most inequitable nations in the region, with the largest Gini coefficient of economic inequality among OECD nations.2 Inequality is observed in Chile beginning at the earliest stages of children’s development. Chilean children under age 5 from low socio-economic backgrounds present significantly higher rates of social-emotional problems and language delays than children from families at the top of the country’s income distribution.3 In an effort to close these gaps and address persistent economic inequality, in 2007 the Government of Chile established early childhood development (ECD) policy as a key priority. It created a national integrated system for early childhood protection, called Chile Grows with You, and expanded free ECE opportunities for the poorest 40% of the population by increasing funding for public ECE centres and for vouchers to private subsidized centres.4 The policy was effective in increasing access to ECE: by 2012, 73% of 4-year-olds and 93% of 5-year-olds were enrolled in pre-school, and most of this growth occurred in the poorest quintiles of the population, who enrolled their children in pre-kindergarten and kindergarten classrooms within public and subsidized voucher primary schools.5 However, the impact of ECE depends on its quality.6 While access had increased, concerns remained about pre-school quality in Chile.7 Specifically, non-instructional activities . snacks, behaviour management and recess time . were found to occupy more than half of the overall time. Instructional activities typically focused on unstructured conversations and arts and crafts, with limited time spent reading books, teaching letters and developing vocabulary or concepts.8
UN BUEN COMIENZO (UBC)
To address such concerns, a group of international and national researchers and Chilean policy-makers, leaders and pre-school teachers designed an intervention to improve ECE classroom quality and outcomes. Called Un Buen Comienzo (UBC . A Good Start), the programme targeted children ages 4 to 5 enrolled in public pre-schools in low-income municipalities of Santiago, Chile. The intervention focused primarily on instructional strategies to promote oral language and early literacy development, with secondary support in the areas of social.emotional development, family involvement and coordination with health services.
A professional development approach was chosen for UBC because although the trained ECE workforce in Chile is large (nationally, there are 19,895 trained pre-school teachers and 21,446 trained day care providers in public, voucher and private institutions), it is characterized by multiple issues that affect quality, including:9
- Poor performance on the national standardized post-secondary school exam;
- Pre-service education of variable quality (24% of institutions that provide pre-school teacher education are not accredited);
- High retention and long duration of service with limited in-service training opportunities; and
- Poor remuneration.
Through UBC, pre-kindergarten and kindergarten teachers and aides received 12 monthly workshops, 24 bi-weekly in-classroom coaching sessions and 6 group reflection sessions over the course of 2 years. From 2008 to 2011, a cluster-randomized experiment enrolled 64 schools with 107 classrooms, 140 teachers and 110 aides serving 1,876 children. The trial showed that the UBC intervention had positive impacts on classroom quality, but minimal impacts on child outcomes.10
These mixed findings presented a dilemma: moderate-to-large impacts on classroom quality suggested that the theory of change underpinning UBC was partially correct, but the lack of impact on child outcomes revealed real deficiencies. Ultimately, with input from all collaborators, the Board of Directors of the principal funder, Fundacion Educacional Oportunidad, decided to continue to work on UBC because it had noteworthy strengths: incorporation of the most up-to-date evidence on professional development; thoughtful participatory design that included researchers, policy-makers, stakeholders and practitioners; and successful implementation. Starting over with a new evidence-informed intervention, one that likely would have been designed and tested in a very different context, seemed less promising than improving UBC. Therefore, in 2011, UBC adopted an innovative approach: it integrated continuous quality improvement and networked learning collaboratives.11 Both methods are designed to improve the original intervention and simultaneously promote sustainability in scale-up.
Continuous Quality improvement (CQI) and Learning Collaboratives in UBC
Continuous quality improvement is a method that uses a deliberate and defined process to adapt proven, evidence-based interventions by engaging the entire organization and its front-line providers in a series of ongoing observations, adjustments and interventions, in order to induce measurable improvements in outcomes.12 CQI is a practical application tied to strong, formal science. The CQI approach first emerged as a way to overcome manufacturing deficiencies,13 and has subsequently been applied in health care, public health and, recently, in education.14
In 2011/12, UBC integrated CQI methods in 3 municipalities with 14 schools and 28 teachers serving 128 children. To begin the process, municipalities located in the O’Higgins Region (VI Region del Libertador General Bernardo O’Higgins) of Chile with a high proportion of at-risk children and a mix of rural and urban schools were invited to apply. Interviews with municipal representatives were conducted to make sure the goals of the programme were clear, explain the evaluation design and answer questions. Three municipalities were selected, and all of their schools were offered the choice of receiving UBC professional development alone or UBC professional development with CQI. A subset of 14 ‘pioneer’ schools volunteered to receive CQI training using the Institute for Healthcare Improvement (IHI) Breakthrough Series Collaborative Model, which combines the IHI Model for Improvement with collaborative learning (see Figures 1 and 2 for illustrations of the two models).15
FIGURE 1: THE IHI’S MODEL FOR
Finally, CQI teams collected common measures, used data in iterative feedback loops to make decisions about their own practice and shared data transparently across the Learning Collaborative. Expert faculty selected measures to reflect the Collaborative’s overall aim and the processes essential to reaching that aim: measures of children’s language and literacy skills (assessed three times per year), and monthly measures of instructional time, instructional quality, children’s behaviour and attendance.
FIGURE 3: SAMPLE PDSA CYCLE FROM A UBC PIONEER SCHOOL
Impact and Lessons Learned
Early results from quasi-experimental studies show encouraging patterns of continued improvement in classroom quality and some positive impacts on children’s language outcomes as a result of integrating CQI and collaborative learning into the UBC programme.16 In addition, the experience revealed a number of valuable lessons for potential future implementations in Chile as well as other countries and contexts. These lessons are discussed in the sections that follow.
Using the CQI method in ECE in the Chilean context was feasible, despite the fact that the country’s ECE workforce is often inadequately trained and poorly compensated. All pioneer schools that participated in UBC with CQI formed CQI teams that included principals, teachers, aides and parents, and all teams completed multiple PDSA cycles and developed the capacity to report data by the end of the first semester.
Using CQI in ECE in Chile led to cultural change in schools. Typically, Chilean schools are characterized by hierarchical leadership and circumscribed roles. Including teachers and parents on CQI teams with principals expanded their usual roles to include school improvement and created a venue through which they worked together with administrators towards a common aim. The CQI method’s reliance on frequent data collection, reflection and transparent sharing was also countercultural. Most participants had some experience reporting data but little experience reflecting, analyzing or using data to inform practice. Over time, teams grew more comfortable and eager to use data, as the data revealed the fruits of their efforts and made it possible to recognize and celebrate good work. Sharing data transparently with peers across the Collaborative created an element of peer-to-peer motivation and drew attention, on the one hand, to sites that were showing improvement in order to learn from them, and on the other, to teams that were struggling so that UBC coaches could investigate during coaching sessions. CQI team members reported that a cultural shift was occurring in their schools, from a culture where data were used for judgment to one where data are viewed with an eye towards learning and identifying opportunities for improvement.
TEACHERS AS CO-DESIGNERS AND CO-INVESTIGATORS
CQI transformed teachers from passive recipients to co-designers and co-investigators in the UBC programme. Training front-line teams in CQI methods invited those who knew the most about the local contexts to design solutions to make the intervention work, and it provided them with the necessary skills to evaluate whether their ideas were leading to improvements. The ideas teachers tested were highly specific to their context and included adaptations that could not be introduced by policy-makers or researchers. For example, to help the children in their classrooms achieve the Collaborative’s language goals, teachers set an aim to dedicate 60 minutes per day to teaching language skills. This seemingly basic decision is revolutionary: in Chile, there is a strong belief in the principle of curricular freedom.17 Neither the Ministry of Education nor the UBC design team could have required or even suggested such a specific target. In fact, one of the hypotheses proposed to explain the original UBC experiment’s lack of impact on child language outcomes was that, although the quality of language instruction improved, the quantity was insufficient: on average, during the UBC experiment, intervention teachers spent only 30 minutes per day on high-quality language activities.18 In 2015, 33 schools participating in UBC’s Learning Collaborative reported spending an average of 57 minutes per day on high-quality language activities.19
POSITIVE RIPPLE EFFECT
CQI fostered the spread of good ideas within and beyond participating schools, leading to a positive ripple effect throughout school districts and municipalities. With the creation of CQI teams, principals in participating schools witnessed for the first time the changes occurring in classrooms through UBC and CQI, and they all developed strategies to reproduce these changes in other classrooms. Many principals hired UBC pre-school teachers to teach CQI methods and language instructional strategies to first grade teachers. In addition, two years after UBC integrated CQI, some teachers asked to continue working with UBC to improve their own practice and that of their peers. Thus, a cadre of teacher mentors emerged. UBC supported these mentors to train their colleagues in UBC language instructional strategies and CQI methods at monthly ministry-sanctioned microcentros (micro-centres), the established mechanism for peer-to-peer professional development in rural areas, and through professional organizations (such as el Colegio de Profesores de Chile [CPC . Teachers’ College of Chile], the Chilean teachers union). Now when UBC partners with new municipalities, the programme only enrols some schools, with an explicit strategy to create mentors among the teachers who receive UBC training, so that they spread the language strategies and CQI skills to all teachers. In this way, CQI has extended the reach of the UBC programme beyond the schools it serves directly.
IMPLICATIONS FOR CHILE AND BEYOND
Since 2012, UBC has used CQI methods in nearly 100 public pre-schools in Chile, training almost 400 ECE professionals to use data in real time to test and drive improvements in classroom quality and children’s outcomes. The Chilean Ministry of Education incorporated CQI in its technical orientation manual for classroom teams, school leadership, stakeholders and technical assistance providers.20 The government’s Agencia de Calidad de la Educacion (Agency for Quality in Education) is exploring strategies for building CQI expertise which would institutionalize the approach and promote the sustainability and reach of efforts to continuously improve the quality of ECE in public primary schools.
The success of this intervention also has implications beyond the Chilean context. Prior to CQI integration, UBC’s original mixed outcomes. showing improvements in some areas but not in others. was not unique; around the world, it is common to find partial positive impacts in studies of ECE interventions.21 The UBC experience suggests that CQI may provide a way forward for improving ECE interventions that have a sound evidence base, careful design, successful implementation and only partial positive impacts. In the post-2015 global agenda for sustainable development, CQI could play an important role in promoting equity by potentiating workforce development in a new way, one that engages front-line providers as protagonists and builds their capacity not only as educators but also as co-investigators in pursuit of a common aim: improved ECE quality to ensure that all children develop to their fullest potential.
- World Bank, 2016; IMF, 2015.
- OECD, 2011.
- Behrman et al. 2010; Schady et al., 2015.
- Peralta, 2011; Molina and Silva, 2010.
- Ministry of Education of Chile, 2014.
- Camilli et al., 2010; Yoshikawa et al., 2013.
- Eyzaguirre and Le Foulon, 2001; Manzi et al., 2008.
- Strasser and Lissi, 2009; PUC Faculty of Education, 2011.
- Ministry of Education of Chile, 2014.
- Yoshikawa et al., 2015.
- Berwick, 2003; Bryk et al., 2011.
- Kritchevsky and Simmons, 1991; Riley et al., 2010.
- Deming, 1986; Juran, 1951.
- Dilley et al., 2012; Nicolay et al., 2012; Park et al., 2013.
- IHI, 2003, 2016; Langley et al., 2009.
- Trevino et al., 2014; Arbour et al., 2015.
- Peralta, 2011.
- Mendive et al., 2016.
- Fundacion Educacional Oportunidad, 2015.
- Ministry of Education of Chile, 2012.
- Burchinal et al, 2010; PCERC, 2008.
- Arbour, M. C., Yoshikawa, H., Atwood, S., Duran, F. R., Godoy, F., Trevino, E. and Snow, C. E. 2015.
- Quasi-experimental study of a learning collaborative to improve public preschool quality and children’s language outcomes in Chile. BMJ Quality and Safety,
Vol. 24, No. 11, p. 727. doi:10.1136/bmjqs-2015-IHIabstracts.11
- Behrman, J., Bravo, D. and Sergio Urzua. 2010.
- Encuesta Longitudinal de la Primera Infancia: Aspectos Metodologicos y Primeros Resultados [Longitudinal Survey of Early Childhood: Methodology and Preliminary Results]. Santiago, University of Chile, Department of Economics. (In Spanish.)
- Berwick, D. M. 2003.
- Disseminating innovations in health care. JAMA, Vol. 289, No. 15, pp.1969.75. doi:10.1001/jama.289.15.1969
- Bryk, A. S., Gomez, L. M. and Grunow, A. 2011.
- Getting ideas into action: building networked improvement communities in education. M. T. Hallinan (ed.), Frontiers in Sociology of Education. Dordrecht, Springer Netherlands, pp. 127.62.
- Burchinal, M., Vandergrift, N., Pianta, R. and Mashburn, A. 2010.
- Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Childhood Research Quarterly, Vol. 25, No. 2, pp. 166.76. doi:10.1016/j.ecresq.2009.10.004
- Camilli, G., Vargas, S., Ryan, S. and Barnett, W. S. 2010.
- Meta-analysis of effects of early education interventions on cognitive and social development. Teachers College Record, Vol. 112, No. 3, pp. 579.620.
- Deming, W.E. 1986.
- Out of the Crisis. Cambridge, Mass., MIT Press.
- Dilley, J., Bekemeier, B. and Harris, J. 2012.
- Quality improvement in public health systems: a systematic review. American Journal of Preventative Medicine, Vol. 42, No. 5, Supplement 1, pp. S58.71. doi:10.1016/j.amepre.2012.01.022
- Eyzaguirre, B. and Le Foulon, C. 2001.
- La calidad de la educacion Chilena en cifras [The quality of Chilean education in numbers]. Estudios Publicos, Vol. 4, pp 85.204. (In Spanish.)
- Fundacion Educacional Oportunidad. 2015.
- Reporte Seguimiento Conductores Primarios del Proyecto Un Buen Comienzo [Monitoring Report on the Primary Drivers of the Good Start Project]. Santiago, Author. (In Spanish.)
- International Monetary Fund (IMF). 2015.
- World Economic Outlook Database 2015. Washington, DC, Author. https://www.imf.org/external/pubs/ft/weo/2015/02/weodata/index.aspx (Accessed 12 March 2016.)
- Institute for Healthcare Improvement (IHI). 2003.
- The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series White Paper. Boston,
- –2016. How to Improve [web page]. Boston,
- Author. http://www.ihi.org/resources/Pages/HowtoImprove/default.aspx
- Juran, J. M. 1951.
- Quality Control Handbook. New York, McGraw-Hill.
- Kritchevsky, S. B. and Simmons, B. P. 1991.
- Continuous quality improvement: concepts and applications for physician care. JAMA, Vol. 266, No. 13, pp. 1817.23. doi:10.1001/jama.1991.03470130097036
- Langley, G. L., Moen, R., Nolan, K. M., Nolan, T. W, Norman, C. L. and Provost, L. P. 2009.
- The Improvement Guide: A Practical Approach to Enhancing Organizational Performance, 2nd edn. San Francisco, Jossey-Bass.
- Manzi, J., Strasser, K., San Martin, E. and Contreras, D. 2008
- Quality of Education in Chile. Santiago, Centro de Medicion MIDE UC.
- Mendive, S., Weiland, C., Yoshikawa, H. and Snow, C. 2016.
- Opening the black box: intervention fidelity in a randomized trial of a preschool teacher professional development program. Journal of Educational Psychology, Vol. 108, No. 1, pp. 130.45. doi:10.1037/edu0000047
- Ministry of Education of Chile. 2012.
- Metodologia de Mejora Continua [Continuous Improvement Methodology]. Orientaciones Tecnicas: Lineas Estrategicas para los Niveles de Transicion. Santiago, Author. (In Spanish.)
- Estado del Arte de la Educacion Parvularia en Chile [State-of-the-Art Early Childhood Education in Chile]. Santiago,
Author. (In Spanish.) http://portales.mineduc.cl/usuarios/mineduc/parvularia2014/index.html
- Molina, H. and Silva, V. 2010
- Four Years Growing Together: History of the Installation of the System of Integrated Childhood Protection, 2006.2010. Santiago, Ministry of Health of Chile.
- Nicolay, C. R., Purkayastha, S., Greenhalgh, A., Benn, J., Chaturvedi, S., Phillips, N. and Darzi, A. 2012.
- Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. British Journal of Surgery, Vol. 99, No. 3, pp. 324.35. doi:10.1002/bjs.7803
- OECD. 2011.
- An overview of growing income inequalities in OECD countries: main findings. Divided We Stand: Why Inequality Keeps Rising. Paris, OECD Publishing, pp. 21-45. http://www.oecd.org/els/soc/49499779.pdf (PDF)
- Park, S., Hironaka, S., Carver, P. and Nordstrum, L. 2013.
- Continuous Improvement in Education. Stanford, Calif., Carnegie Foundation for the Advancement of Teaching.
- Peralta, M. V. 2011.
- Early childhood education and public care policies in Chile: a historical perspective to analyze the present. International Journal of Child Care and Education Policy, Vol. 5, No. 1, pp. 17-27. doi:10.1007/2288-6729-5-1-17
- Preschool Curriculum Evaluation Research Consortium (PCERC). 2008.
- Effects of Preschool Curriculum Programs on School Readiness (NCER 2008.2009). Washington, DC, National Center for Education Research, Institute of Education Sciences, US Department of Education.
- Pontificia Universidad Catolica de Chile (PUC) Faculty of Education. 2011.
- Alfabetizacionen Establecimientos Chilenos Subvencionados: Informe Final [Literacy in Subsidized Chilean Institutions: Final Report]. Santiago, Author. (In Spanish.)
- Riley, W. J., Moran, J. W., Corso, L. C., Beitsch, L. M., Bialek, R. and Cofsky, A. 2010.
- Defining quality improvement in public health. Journal of Public Health Management and Practice, Vol. 16, No. 1, pp. 5.7. doi:10.1097/PHH.0b013e3181bedb49
- Schady, N., Behrman, J., Araujo, M. C., Azuero, R., Bernal, R., Bravo, D., … Vakis, R. 2015.
- Wealth gradients in early childhood cognitive development in five Latin American countries. Journal of Human Resources, Vol. 50, No. 2, pp. 4188.8.131.5268/jhr.50.2.446
- Strasser, K. and Lissi, M. R. 2009.
- Home and instruction effects on emergent literacy in a sample of Chilean kindergarten children. Scientific Studies of Reading, Vol. 13, No. 2, pp. 175.204. doi:10.1080/10888430902769525
- Trevino, E., Godoy Ossa, F., Yoshikawa, H., Snow, C. E. and Marzolo, M. 2014.
- Quasi-Experimental Impacts of Continuous Quality Improvement in a Preschool Teacher Professional Development Program in Chile: A Propensity Score Matching Approach. Manuscript under review.
- World Bank. 2016.
- World Development Indicators Database. Washington, DC,
Author.http://data.worldbank.org/ (Accessed 12 March 2016.)
- Yoshikawa, H., Weiland, C., Brooks-Gunn, J., Burchinal, M. R., Espinosa, L. M., Gormley, W. T., … Zaslow, M. J. 2013.
- Investing in Our Future: The Evidence Base on Preschool Education. Ann Arbor, Mich./New York, Society for Research in Child Development/Foundation for Child Development. http://fcd-us.org/resources/evidence-base-preschool
- Yoshikawa, H., Leyva, D., Snow, C. E., Trevino, E., Barata, M., Weiland, C. … Arbour, M. C. 2015.
- Experimental impacts of a teacher professional development program in Chile on preschool classroom quality and child outcomes. Developmental Psychology, Vol. 51, No. 3, pp. 309.22. doi:10.1037/a0038785.