When you use a Learning Activity, a tool or a resource for the first time in your classroom or with a new group of students, it might not run smoothly, so it is important to get feedback and reflect on what worked well and what still needs improvement. Reflection and feedback are a valuable source of information for pilot evaluation and are used to:

• Compare the expected benefits of new approaches and technologies with the actual benefits experienced during the implementation
• Appraise the effectiveness of teaching and measure learning impact
• Identify good ideas, approaches, tools and resources that can be shared with other teachers either at school or in the community
• Enhance teaching methods, student learning and boost teacher and student motivation

Evaluation guide

At the beginning of the evaluation process, you should identify the objective of the classroom pilot, specify the information you need to collect and select appropriate evaluation tools. Use this template (5.1.1) to help you prepare for the evaluation process.

Objective of the classroom pilot

Before you start, define what you intend to achieve through the pilot, e.g. measure the level of collaboration among your students, enhance student engagement, allow for more student autonomy, improve assessment practices, determine the benefits of new technology, etc. Discuss the objective you want to achieve with your colleagues.

What to evaluate

After you have defined the objective of the classroom pilot, specify the information you need to collect and where you can find it. You can do this by identifying the target of evaluation – e.g. learners can provide you with the required information – or the technology that you need to investigate to collect data and evidence.

• The learners: you might need information such as age range, number of students in class, digital literacy level, attitudes towards innovation (e.g. motivation), classroom behaviour and engagement, impact on their learning, advantages of new learning methods for their learning, challenges and obstacles, evidence of learning, reflection on the learning process.
• The teacher(s): e.g. previous experience and confidence in using advanced approaches supported by digital technology, effects of new or different teaching methods, impact on educational outcomes, the possibility of integrating innovative activities in the teaching practice, ways to encourage innovation, reflection on what has been learned about teaching and students' learning.
• Tools and resources: e.g. support of the learning outcomes and development of skills, enhancement of the learning process, appropriateness regarding student age, ability level or learning goals, levels of difficulty of using the tool.

How to evaluate

Information needed for the pilot evaluation can be obtained by using different methods and instruments, e.g. rating scales, checklists, rubrics or surveys with detailed feedback, etc., depending on how the Learning Activity is being implemented. For example, a single teacher may use self-reflection, rating scales or consultation with students and colleagues. If the adoption of Learning Activities is being implemented as part of a whole-school approach then more structured methods are advisable, including staff and student questionnaires, observations, reflective discussions, peer review, etc. You can use a combination of different tools to ensure efficient evaluation.

Self-reflection: By reflecting on what worked and what didn't while piloting the Learning Activity in your classroom, you become aware of its strengths and weaknesses and gain a realistic insight into what needs to be improved. A self-reflection journal is useful for summarising your experience and putting your thoughts into words.

Peer observation of the teaching process along with peer review of the materials, lesson plans, tools and resources used in the activity provides you with a different perspective and appraisal of your classroom pilot and offers timely feedback that can be used for improvements. Observation is usually carried out in a real-life setting by an observer. However, remote observation may also be possible by using ICT.

Reflective discussion is more than just giving feedback. Its focus is on building upon the feedback you received and using the critiques to consider how your Learning Activity can be improved, what needs to be changed and what remedies you can use to make it better.

Feedback capture grid can be used to organise the feedback you received in four different areas: strengths, weaknesses, questions raised and ideas sparked during the classroom pilot.

Rubric is a tool in the form of a table which includes indicators of what we want to evaluate (the criteria) and a grading scale for each criterion. Rubrics help us define what we mean by quality and how it can be graded.

Checklists are written in a yes/no format. They are efficient and easy to use. The ideal checklist has between five and ten items, with simple and accurate wording and an easy-to-read layout, ideally occupying no more than one page. The simpler they are, the higher their value.

Rating scales show degrees of performance and are used to gain a more specific insight into an item of evaluation than can be obtained through a yes/no format. The scale points can vary depending on what you want to measure, e.g. for measuring

• quality you can use: poor - fair - good - excellent;
• for frequency: never - sometimes - often - always;
• for experience: beginner - somewhat experienced - expert;
• for opinion: strongly disagree - disagree - neither disagree nor agree - agree - strongly agree.

Surveys and questionnaires are used interchangeably in this guide, even though there is a slight difference between the two: a questionnaire is a set of questions used to gather information; a survey is the process of analysing gathered data. However, with the development of online tools, both questionnaires and surveys allow for data analysis. They are efficient for gathering clear information and precise feedback which can easily be analysed and interpreted. They can be used for gathering factual information, attitudes, preferences, beliefs and experiences. Different types of questions can be used to collect information: multiple choice, multiple answers, rating scales, ranking questions as well as open-ended questions to allow the respondents to express themselves in their own words.

Suggested workshop activities

In the following workshop activities, you can evaluate your own learning activities or use those that we have provided. They are teacher-created learning activities and they serve as practical activity and preparation for your evaluation of your learning activities so that you gain insight into how the evaluation process works, what needs to be taken into consideration during the process, and how to make good use of the evaluation results to determine the value of evaluation for future planning and implementation of learning activities.

1 - Peer-review

Be a peer reviewer and review these teacher-created learning activities: Spice It Up or Young Entrepreneurs. Reflect on how the Future Classroom Scenario was used to develop these learning activities. How is innovation encouraged throughout the activity? What benefits are assumed by the teacher? What obstacles might the teacher encounter along the way? Compare your review with that of a colleague and compare your views.

If you wish, you can select other learning activities in the FCL Resource Section or in the Learning Designer for peer review. You can choose learning activities in your own language for peer review.

2 - Reflecting on your own teaching practice

A. Reflect on what happened in your classroom while teaching. You can write your thoughts in a self-reflection journal. Give a summary of what happened. Write about how you felt during the lesson, how you dealt with the problems that came up and how you responded to questions raised by your students. How did your students react to innovation, how engaged and interested were they? Do you think the lesson objectives were met? Use this template (5.1.2) to reflect on your experience.

B. You can also video record your lesson and watch it later to analyse the teaching and the learning process, a specific area or objective that needs to be focused on or the overall implementation of the Learning Activity. This might provide you with the opportunity to notice something that you were not previously aware of.

C. You can share the video recording with your colleague and ask them to reflect on it to give you a peer review. You can also ask them to review your self-reflection journal and then you can compare your views. Discuss the differences and analyse why your colleague's reflection differs from yours.

D. Share with your colleague your Learning Activity, your lesson plan, the tools and materials you used in the pilot as well as student feedback, so that they can examine it before discussing it with you.

3 - Lesson observation

A. Invite your colleague to your classroom to observe your lesson. Give your observer a sheet of paper divided into four quadrants to make notes about your lesson. The upper left quadrant should be for all the things that went well. The upper right quadrant is reserved for constructive criticism. All the questions that the experience raised should be written in the lower left quadrant. Ideas that they come up with go in the lower left quadrant.

After they have observed the lesson, engage in a reflective discussion with your observer. Use the feedback they provided to come up with new ideas, raise questions, exchange ideas and teaching practices. Encourage your observer to share their experiences to help you overcome the problems and obstacles you encountered.

B. Organise teaching triangles. A triangle consists of three teachers. Each teacher conducts one lesson and has the two colleagues visit his/her lesson. After the lesson they meet twice for reflection – first they meet one-on-one, then all three come together to reflect on their colleague's teaching practice.

4 - Evaluation rubric

A. Together with your colleagues, build a rubric for evaluating your Learning Activity. The first step is to define the most appropriate and important criteria that you want to evaluate. You can do this by asking: What features of the Learning Activity provide us with evidence that the learning objective has been met? How will students demonstrate their learning? What knowledge, skills, behaviours and attitudes do they need? For example, if your objective is to enhance student collaboration, the criteria might be: organisation of work, participation, responsibility, respect. Ensure that the criteria are relevant, observable, measurable and distinct from one another. It is recommended to use between three and seven criteria.

The next step is to add a rating scale and the descriptions for varying levels of quality for each criterion. It is recommended to have between three and five rating scale points, e.g. below - meets - above expectations or poor - fair - good - excellent. The descriptions should be written clearly and understandably with the same wording for each rating point. For example, if one criterion measures a task list, then all the criteria should measure it, but on different levels, e.g. doesn't create a task list - creates a simple task list - creates a detailed task list.

To build a rubric you can use one of the tools in the Tools list.

B. Analyse this rubric, Young Entrepreneurs, made by a teacher with the intention of evaluating collaborative student work in a project-based learning assignment.

C. Do peer review and peer observation using the rubric. One of your colleagues implements a Learning Activity in the classroom. Measure the results by applying the descriptions. After the lesson discuss the results with peers.

D. Involve your students in creating a rubric to be used during the classroom pilot. Show them a model rubric, then encourage them to come up with the criteria that will be evaluated. Divide them into three or four groups (depending on how many criteria they selected) and assign each group one criterion. Have each group write descriptions for each of the levels of their criterion.

5 - Checklists

A. Meet with your colleagues to create a checklist. The first step is to define its purpose and then the criteria needed for the successful implementation of the Learning Activity. Use the checklist with your students to measure the results.

B. Look at these two checklists for self-assessment and peer-assessment on collaborative task design. Can the purpose of evaluation easily be recognised? Are the questions easy to understand? Do they measure the desired outcomes successfully?

6 - Rating scale

A. Meet with your colleagues to design or adapt a rating scale. The first step is to define its purpose and then the criteria needed for the successful implementation of the Learning Activity. Add three to five responses to each criterion. Distribute the rating scale to your students or colleagues to measure the results.

B. Use an online tool from the list of tools to create an interactive rating scale so that you can instantly analyse the results.

C. Rating scales, checklists and rubrics are used to systematically collect data. Discuss with your colleagues in which situations each of them would be the most appropriate for use.

7 - Create a questionnaire

A. Use one of the online tools to create a questionnaire for your students and colleagues about the pilot. Choose different types of questions depending on the information you want to collect. Test your survey and analyse the data you collected. Consider how your survey can be improved.

B. Analyse this survey created by a teacher to evaluate teamwork on the topic of Safer Internet as part of her Learning Activity.

8 - Peer-Instruction

Involve your students in peer instruction. Ask them to teach their peers the concepts they have mastered.

You can also involve students in student-to-teacher instruction so that they present their learning to other teachers, the headteacher or the parents and other stakeholders. Afterwards, give feedback to peer instructors and engage all the students in a reflective discussion.