2007. External Assessment of the training activities.  In 2007 the Internal Oversight Service of UNESCO HQ took the activities of the SLN as one of the cases to accomplish a Mandate of the Executive Council, in order to evaluate the activities of the Organization. By means of an international bid, a Mexican firm was selected.

The complete report is at:

http://unesdoc.unesco.org/images/0016/001610/161087S.pdf#page=4

 

The assessment:

- Approach & Methodology: “evaluation addressed the relevance, efficiency, effectiveness, impact and sustainability of the program. Carried out by a team of three independent consultants, the evaluation makes recommendations to improve the program. The evaluation applied a mixed methodology approach: a survey to principals who had received training; in-depth focus group interviews to OREALC-UNESCO functionaries and program consultants; a review of relevant documents; in depth case-studies of participating schools; and statistical analysis of repetition and dropout rates”.

 Findings:

 “The information gathered throughout the evaluation shows that the program has been highly relevant for UNESCO’s institutional goals related to improvements in the quality of education. In fact, the program’s placing of schools at the centre of efforts to change the management of school systems was ahead of its time and pre-empted UNESCO recommendations in 2007. The evaluation team concluded that the conceptual approach was innovative and highly relevant to Latin American school contexts with the program meeting the demands of beneficiaries. For instance, the majority of school principals who were interviewed stated that the course improved their leadership competencies and gave them new skills for improving their school management capabilities”. 

 “Three different kinds of sources of information were used to assess the impact of the training course for principals. The first source was responses by trained principals to questions related to (1) individual principal capacity and practice; and (2) school organisation and environment. Next, repetition and dropout rates were examined to assess changes following the training courses. Thirdly, in depth case-studies were conducted at the school level in which the evaluators visited schools led by trained principals who had received training”.

 “The data provides sufficient empirical support to conclude that the training course has brought about changes in daily leadership practices and in school organizational environments. For example, the principals’ responses to the 28 survey questions related to school organisation and environment showed that changes occurred across all of the dimensions asked. The greatest changes were in: student motivation; teachers’ ability to accept critical thinking; class planning; teachers’ willingness to accept directors in their classrooms; acknowledging common project goals and strategies; and the joy of coexisting”.

 “The responses to the 25 survey questions on individual principal capacity also showed that the program had significant impact in this area. The practices that showed the most change after the course were: the ability to make clear and precise requests; confidence building in the teaching team; establishing standard procedures in administrative and educational routines; and equity in expressing judgment”.

 “Most of those school principals who were interviewed for the survey questions were able to narrate specific applications of the skills developed by the leadership course. Most felt that teachers and their supervisors/superiors were able to recognize changes and considered them to be long-lasting”.

 “The program made large-scale interventions on two school modalities in Mexico (secondary schools in the state of Chihuahua and national job training centers CECATI)) through training almost all of the principals. Only the secondary school level intervention had relevant indicators to assess impact.  Although the data is only illustrative, it showed that dropout and repetition rates fell after the leadership course had been introduced on a large scale. Although the changes in the indicators cannot be solely attributed to the intervention, they do show that progress has been made towards the program’s objectives”

 “The case studies were of two primary rural schools (one in each of Nicaragua and Ecuador) and a CECATI in Mexico City. These case studies also showed the impact the leadership course can bring to schools, in particular by improving conditions under which the program and additional initiatives operate at the school level. The studies also showed that the leadership course cannot be considered as an isolated, albeit important, factor in the process of inducing change in schools”.

 Although this positive evaluation, it was clear that the training processes needs to move fast towards training processes in pedagogical leadership, because:

 “The following showed the least change: defense and dignified representation of the school; knowledge of the curriculum; teaching and assessment strategies; provision of materials and opportunities of continuing learning for teachers; and their involvement in design and implementation of teacher education or institution evaluation change: defense and dignified representation of the school; knowledge of the curriculum; teaching and assessment strategies; provision of materials and opportunities of continuing learning for teachers; and their involvement in design and implementation of teacher education or institution evaluation”.