Tool 5.1 – Classroom pilot and evaluation guide

This guide simply covers some of the things to consider when using a Learning Activity for the first time or with a new group of students or using tools and resources which haven't been used before. We call this a pilot of the Learning Activity – it may not go perfectly first time so it's important to consider what works and what needs to be improved. This is particularly useful if the Learning Activities are being used by a number of teachers in a school or across a number of schools or education region.

During the pilot you will need to consider:

  • Getting feedback from students and other people involved, including other teachers, parents etc. This feedback is used to inform further development of the Learning Activity, what resources to use in future, and the impact of any changes made on teaching and learning.
  • Assessing and measuring the impact on the learning – including learner motivation and achieving outcomes.
  • Identifying good ideas, approaches, tools and resources that can be shared with other teachers either in school or as part of a wider community.
  • Comparing the expected benefits of new approaches and technologies with the actual "real" benefits experienced.
  • Evaluating the whole process of implementing changes, and what conditions and support are needed for success (including school policies and processes).
  • Considering your own training and professional development needs.

1 – Preparing for the pilot: Lesson Planning

The Learning Activities and Learning Stories which have been designed, or selected, should be used to guide lesson planning. Teachers do not have to stick rigidly to the Learning Activities, but can use or adapt them taking in to consideration the following constraints:

  • Time available.
  • Access to tools and resources, including digital technologies, people and places.
  • Curriculum learning objectives.
  • Support available (e.g. technical support)
  • The abilities and experiences of your students.

It's important to ensure that the innovative elements of the Learning Activities feature clearly in your lesson plans.

Think also about what you intend to achieve through the pilot. What is the primary objective, e.g. increased levels of collaboration, greater student engagement, more student autonomy, improved assessment, establishing the value of a new process or technology, etc.

2 – Evaluating the pilot

The purpose of the evaluation is:

  1. To identify the benefits and shortcomings of the Learning Activities in relation to learning and teaching, and opportunities for further development.
  2. To identify the specific benefits and shortcomings of the digital tools used, along with other resources.
  3. To capture perceptions of change and innovation from the perspective or others, including learners and teachers.

There are several approaches which you can use for evaluating the pilot. A single teacher may use an informal approach of self-reflection and consultation with students and colleagues. If the adoption of Learning Activities is being implemented as part of a whole school approach to increasing the level of innovation in the school, or across a number of schools, then more structured approaches are advisable, including staff and student surveys, case study reports, observations and focus groups.

Teacher and student questionnaires and/or case studies

If a single teacher is using the Learning Activities to improve their own practice, then the following can be used as a guide to self-review and reflection. If there is more wide-scale piloting across a school or number of schools it may be useful to create a questionnaire for the teachers involved, or bring them together (face-to-face or online), in a focus group to discuss the topics.

  • The learners – Subject, ages range number of students etc.
  • The teacher – Teachers' experience and confidence in using advanced approaches supported by digital technology.
  • Learning Activities and Learning Stories – Details of those used to inform lesson planning.
  • What happened? Teacher and student experiences – Activities and timings, teacher and student experiences, etc.
  • Teacher's comments (+/-) What is new/different? What are the benefits and/or the challenges/obstacles? Is this the way you would like to teach all the time? On reflection, what have you learned about your teaching and your students' learning?
  • Students' comments (+/-) What is new/different? What are the benefits and/or the challenges/obstacles? Is this the way you would like to learn all the time? On reflection, what have you learned about your learning and your teachers' teaching?
  • Key Benefits or challenges – Specify the benefits or challenges, if any, for you as a teacher and for the students. 

3 – Further information

Wide scale impact of the iTEC project

When considering how the Learning Activities you use have impacted on your teaching, and your students learning, consider the results below from the iTEC project. These results may help you develop a better understanding of how innovation in the classroom can bring multiple benefits.

Over the four years of the iTEC project, the evaluators (led by Manchester Metropolitan University) gathered the views of teachers (1,399) and students (1,488), national coordinators and policy-makers through surveys, interviews, focus groups, case studies and observations. The results were collated under three headings in the final evaluation report.

Impact on learners and learning

  • Students developed their 21st century skills, notably independent learning, and felt that they would perform better in examinations.
  • Students took on new roles in classes, becoming peer-assessors, peer-tutors, and co-designers of their learning, even teacher trainers.
  • Participation in Learning Activities underpinned by the iTEC approach impacted positively on student motivation.

Impact on teachers and teaching

  • The Scenario Development and Learning Activity Design process was clearly seen as innovative.
  • Teachers' digital competencies and pedagogy were enhanced.
  • Teachers became more enthusiastic about their pedagogical practices.
  • There was increased use of technology, integrated throughout the learning process rather than only for research or presentation.
  • More collaboration took place between teachers within and beyond schools, facilitated through online communities.

Read more about these and other findings in the iTEC final evaluation report.

iTEC set out with the challenging objective of finding a way to mainstream effective use of learning technology in European classrooms, with a particular focus on supporting the development of advanced competencies in teachers, and 21st century skills in learners. The evaluation evidence overwhelmingly indicates that the project has achieved this, and at scale. The Future Classroom Toolkit has been created to enable this process to be adopted by thousands more of Europe's schools.

4 – The Future Classroom Lab Validation Manual

The term "validation" refers to the total process associated with the practical organisation and running of school or classroom pilots involving the use of ICT by teachers and students, including the collection of feedback and data for evaluation purposes.

European Schoolnet (EUN) has coordinated numerous pan-European school pilots (including the iTEC project, on which this toolkit is based). In many EUN projects, ministries work alongside industry partners and EUN has also designed and run school pilots on a bi-lateral basis for individual ICT suppliers. EUN has become aware that many technology-enhanced learning (TEL) projects and ICT suppliers frequently underestimate the complexity and cost of developing and running effective pan-European validations in schools. Many particularly have a poor appreciation of the degree of support that busy classroom teachers may need in different countries (with different curricula and levels of ICT deployment) in order to test prototype tools and services.

Building on this knowledge and experience, EUN has created a Future Classroom Lab Validation Manual to provide detailed guidance on what is required in order to develop and run pan-European validations in schools and particularly what outputs can be expected.

This manual is primarily designed to be used by ministries, ICT suppliers and projects that wish to run school pilots. The aim is to help different stakeholders to appreciate the challenges faced by busy teachers who are engaged in educational research activities when their first priority must remain delivering a high quality learning experience for their students.