World University Service Canada (WUSC): Improving Teacher Effectiveness through a Competency-Based Classroom Observation Tool
by Timothy Kinoti, Evaluation & Learning Manager, WUSC Kenya
Teachers are the single most powerful resource in a classroom, especially in humanitarian crisis settings, where resources are limited. In fact, in crisis settings the teacher is often the only resource in the room, meaning learning is entirely dependent upon the quality and efficacy of their instruction. Ensuring teachers deliver high quality lessons is therefore central to improving student-learning outcomes.
WUSC has developed a competency-based classroom observation tool to monitor, evaluate and inform the effectiveness of teachers delivering their remedial classes. Finding out what constitutes effective teaching is the key to driving successful learning and classroom observations provide an opportunity to identify best practice, share innovations and inform teachers’ ongoing professional development. Feedback sessions following each observation are also critical, to support and guide teachers in areas that need improvement.
The tool is essentially a guide to support the observation of various aspects of a remedial class, including:
- Lesson structure;
- Pace;
- Planning;
- Teaching methodologies;
- Use of resources;
- Classroom routines;
- Classroom management;
- Assessment for learning; and
- Student engagement.
The development of the tool was an iterative process, which required close coordination between the monitoring and evaluation team, the project implementation team, and most importantly, the remedial teachers and classroom observers.
Step 1: Talk to the Teachers First!
Beneficiary led inputs are crucial in the process of developing an instrument that can be useful to them. For the classroom observation tool, it was key to hear from those being observed in order to understand what makes sense to them in the observation process. In this regard, WUSC held meetings with remedial teachers and Community Mobilizers in Kakuma and Dadaab on ‘What makes a great remedial teacher?’, including a brainstorming activity to explore all the things that a great remedial teacher does to drive student learning. When asked to identify the qualities of a great remedial teacher, the teachers came up with the following:
Step 2: Building the Classroom Observation Tool
The WUSC Monitoring and Evaluation (M&E) team, with support from HEA’s External Evaluator — American Institutes for Research (AIR) — reviewed an existing classroom observation tool that was used in previous projects by WUSC. The primary advantage of using an existing observation tool is that it saves time and resources that would otherwise be required for its development. Using the information gained in the workshop with the teachers and community mobilisers, the tool was adapted to fit the needs of the current project. First, WUSC categorised the responses of what makes a great remedial teacher, to come up with the observation framework. The team then reviewed existing tools and decided that a four point scale and differentiated criteria against each competency would help to ensure that observations were as objective as possible.
Step 3: Moving from Paper to Digital Data Collection
WUSC also collaborated with UNHCR to use their KoBo Toolbox, an online platform that builds digital questionnaires and includes basic analysis in the form of dashboards. This digitised system has been a unique approach to classroom observations, enabling the collation of real time data on classroom interactions, as well as identifying teaching practices that are innovative or require additional training. The classroom observation process has also contributed immensely towards the generation of data on how remedial learning takes place. The Kobo tool also allows the observer to recall past observation sessions that they have had with the teacher and identify areas that have been improved, in addition to documenting what needs to be emphasised in the subsequent trainings.
This system additionally allows the observer to undertake rapid response coaching with the teacher directly after an observation, working with them to create a better lesson plan for the following week. This data is important for the evaluation of the programme, as well as evidence building.
Step 4: Test the Tool
An essential component of classroom observation is ensuring that the observer understands how to record their observations, to facilitate comparisons across observers and classrooms. This is done by having clear instructions for use, removing the issue of different observers using different methods.
There are three main components of standardisation that we considered when evaluating the classroom observation tool:
1. Training protocol: training on how to go about using the tool;
2. Observation protocol: guidelines on what we want to observe and the sequence to be followed in the observation instrument;
3. Scoring options: what weighting is given in each of the sections on the observation — this is crucial to informing the performance evaluation of the teacher and what areas need to be the focus for mentorship and coaching.
This is an important step, to pilot an instrument in order to find out what works and what needs to be changed. WUSC pre-tested the instrument in a live remedial class where the team regrouped afterwards to consolidate notes and share lessons learned. This process confirmed that having guiding descriptors against each observation point (competency) and using a 4-point scale would be beneficial, as well as helping to standardise observations. Having guiding descriptors also helps to build the capacity of observers and project based education officers to effectively conduct lesson observations. The guidance descriptors will also help to inform the teacher’s professional development, by providing a roadmap for progression against each competency.
Step 5: Rolling Out the Observations
For the purpose of the initial classroom observations, the project team decided to hire one expert classroom observer in Dadaab and Kakuma who has experience in curriculum development and large classroom management. The coach collected data during classroom observations as well as engaging teachers in a rapid capacity building process, focusing on the areas they struggled in. The consultant carried out at least one classroom observation for each remedial class in the programme. With 10 remedial classes running (for class 7 and 8 high achieving learners) across five centres the process took approximately 3 months. This culminated in a 3-day training for all remedial teachers, to address challenges as well as highlight promising practices and different teaching styles seen during the observations.
Step 6: Next steps; feedback loops for continuous improvement
As the same classroom observation instrument is being used in Kakuma and Dadaab, it provides data on the different teaching styles across two very different contexts. The two expert classroom observers also have a joint debrief with the WUSC team to understand similarities in the programme and challenges faced in each location. It is important to note that the goals of conducting observations is not solely for gathering information on the quality of classroom processes but to use that information to help teachers improve their practices (and, eventually, student outcomes). The WUSC team believes that grounding both teacher training and classroom observations in a competency-based framework supports the notion that changing teachers’ values and attitudes will lead to changes in their behaviour and practice, which in turn influences students’ knowledge, attitudes and behaviours and ultimately impacts student outcomes.
The information collected has been translated into professional development planning, allowing the WUSC team to pick up on trends in teaching practices. This data has allowed the team to ensure best practices are embedded into their teacher training manuals. Though the classroom observation tool was designed to improve the remedial classes on an ongoing basis, it has also allowed us to investigate the following characteristics of the remedial classes:
- Do remedial teachers apply the pedagogical techniques learned in the training?
- Do remedial teachers effectively engage with girls in the classroom?
- Do remedial teachers cover the expected content topics in the classroom?
- Do remedial girls participate and engage in the remedial classroom?
These are crucial questions to inform the process of implementation and scalability of the programme.
Through this inclusive process, with participation from all stakeholders (including the girls in the remedial classes themselves) the WUSC team was able to develop a classroom observation tool that was tailored to the needs of the remedial programme. In a similar vein, WUSC further developed the Kobo ToolBox to improve the way Community Mobilizers are able to track and follow up on absenteeism, assisting them in their community outreach work — through digitising remedial attendance and documenting follow ups of absentee girls. This process has been scaled to also be undertaken within other remedial programmes managed by WUSC.
