The background

The P2V framework was developed as part of the P2P-Inspectorates project, which was a cooperation between Ofsted (Office for Standards in Education, England), Skolverket (Swedish National Agency for Education, Sweden), HMIE (Her Majesty's Inspectorate of Education, Scotland), Department of Education and Science (Ireland), IGEN (Inspection Générale de l’Éducation Nationale, France) and Inspectie van het Onderwijs (The Netherlands). Partners in P2V were: Scotland, Sweden, France, Catalonia, Lithuania and The Netherlands.

The P2P aim was to bring together standards and indicators as used by several education inspectorates throughout Europe, revise and reorder these criteria and present them as one shared model. Although the framework is developed by the inspectorates involved, its application is not limited to those inspectorates, nor to inspecting authorities only. It could also be used by individual schools to self-assess their ICT use and by groups of schools to carry out peer evaluations.

In the P2V project, the framework is developed into a full evaluation toolbox, containing user guide, observation forms, interview guidelines and self-evaluation forms. The framework is also applied and tested by inspectorates in several countries throughout Europe to evaluate ICT use in primary and secondary schools.

P2P and P2V are initiatives of European Schoolnet and were co-financed by the European Commission. The inspectorates work packages of P2P and P2V were developed under the auspices of SICI (www.sici-inspectorates.org) and coordinated by The Dutch Inspectorate of Education (Inspectie van het Onderwijs).

Looking back on visited schools: a general overview

On the first part of the workshop, held at European Schoolnet quarters in Brussels and chaired by Bert Jaap Van Oel and Hester Visser from the Netherlands, all the participants (Iain Lowson and Elizabeth Cole from Scotland, Asta Buineviciute and Tomas Jankunas from Lithuania, Mireille Golaszewski and Michel Perez from France, Joan Escue and Angel Domingo from Catalonia, and Peter Ekborg and Sven Borg from Sweden), were asked to look back on the visits to the schools of their own country. Every inspectorate visited four schools in his/her country, generally two primary and two secondary, except from France where also a fifth school was assessed (two primary, two lower secondary and one upper secondary school).

Proceeding country by country, all the inspectors highlighted the key-points of their visits by analysing schools' general characteristics as well as their ICT state-of-the-art (equipment, curriculum, pedagogical view, planning and overall ICT school strategy). Each country overview was preceded by an introduction about the reasons of the choice of the schools visited, and followed by a brief summary pointing out differences and similarities between those schools. Many questions were asked to the speakers and a very high level and lively debate arose among the participants.

Scotland

The schools visited had very different characteristics, both in terms of size and context as well as in the use of ICT. The Scottish experience showed a multifaceted but positive landscape where each school have its own singularity: primary schools are keener on using ICT for innovative pedagogical purposes, while secundary schools are more interested in ICT as a means of targeting behavioural problems, attendance, results, etc. The Scottish inspectors missed a chapter for ‘good practices’ in the report of the schoolvisit.

Lithuania

As in Lithuania doesn't exist a national inspection system, schools were asked to propose themselves as volunteer for the P2V inspectors' visit. Therefore it is very difficult to state the average situation of the use of ICT in Lithuania. Furthermore, ICT is not compulsory in primary schools, while in secondary only one hour a week but no matter what subject should be involved. All the schools visited showed interesting ways of using ICT (extra curricular lessons, benchmarking system, participation in international programmes, etc.), but they were local and isolated examples of good practices depending on teachers' and headmasters good will.

France

The visited schools were all located in an "average" region in France (Nantes, Le Mans), supposed to be representative of the national context. The French inspectors were very strict in the evaluation process, due to their role in the French educational system where teachers have a lot of pedagogical freedom. They precisely outlined strengths and weaknesses for each school, observing in general good ICT practices (including the widespread use of VLEs), fostered by the compulsory "B2I" license students must obtain by the end of lower secondary school and the ICT plan ("volet numérique du projet d'établissement") schools must set up every year. At the same time, they also observed the lack of an overall and coherent pedagogical strategy in the implementation of the use of ICT, as teachers have a lot of autonomy.

Spain (Catalonia)

The Catalan schools, although three out of four were also involved in the P2V WorkPackage for schools, didn't score well a part from only one advanced position. The visits were prepared strictly following the toolkit and the framework. The biggest problem to tackle in Spain in order to implement ICT is the lack of infrastructures: sometimes disastrous buildings and poor maintenance and equipment as well are real limits. Another critical point raised is the role of Catalan headmasters: they have strong authority and a clear vision as regards administration and management but not pedagogy, which is left to teachers who have great autonomy. Schools encountered some difficulties in reacting to the timely evaluation given by inspectors, as in Spain there is not an evaluating tradition with marks. The Spanish inspectors also discussed the agenda for the discussion with the pupils and the lack of evidence for the impact of ICT on education.

Sweden

The Swedish schools selected for inspection were very different, going from one located in a poor area and attended by students whose at least one parent was born abroad to another one extremely advanced and well equipped. The main issue emerged from observations on the ground was the lack of compulsory national standards for both ICT in the curriculum and equipment. Therefore the ICT situation in Swedish schools is very heterogeneous, having institutions perfectly equipped (but with a very few whiteboards, as there is no interest ion the country) and others completely deprived of any kind of resources. The inspectors also felt the need of a large scale teacher training programme; otherwise it would be left in charge of local institutions' and individuals' capabilities.

Debating school visits

From the visits' reports emerged two major aspects influencing the implementation of ICT in schools, as highlighted by Roger Blamire from EUN, who attended the meeting as observer. First of all, leadership: ICT implementation depends on the degree of leadership and autonomy headmasters, schools and teachers have within their national education system.

Second, another barrier is the lack of proved link between ICT and improving results in schools: if there were impact studies on learning, it would be a tremendous input to spread the use of ICT widely and rapidly. At the same time, as other participants suggested, the term "impact" should be better defined.

Analysis of the collated scores

A part of the afternoon was also dedicated to a collective analysis of the collated scores attributed to the visited schools. The analysis provided a statistical picture of the presentations made before by every inspector: the collacted scores results underlined those key-aspects emerged in the previous discussion.

Thus, the French schools are those highlighting the most criticalities because of the severity of inspectors, while the Lithuanian have got good marks but they are above average schools, showing very individual and local good practices (i.e. benchmarking system). On the other hand, Scotland shows quite a positive average picture, where especially the creativity of primary schools has been appreciated.

At the opposite there is Spain, whose schools, a part from one which is the most advanced in ICT of the whole Catalonia, scored very differently from one to another. Sweden made more reservations in terms of observations, preferring not to assign points for those issues not directly measurable, such as the integration of ICT in the curriculum as in Sweden it is not compulsory. However, a very impressive administration system emerged through the collated scores of one of the Swedish schools.

The scoring system: the P2V framework

The scores were attributed to schools via the P2V framework, whose basis was laid during the P2P project and then enhanced by adding a number of supports such as: an evaluator guidance; a self-evaluation questionnaire; an interview guidance; a lesson observation form and a template report.

Besides these materials, the framework has also been improved by the possibility to add notes and a column in which scores can be assigned to each indicator. The framework now contains as well clues about the sources that can be used to find evidence about a specific indicator.

- The evaluator guidance contains all kinds of practical information and helps the local evaluator in setting up the visit by making appropriate choices before, during and after the visit. It also gives tips about making observations, documenting evidence, writing and referencing the school report.

- Prior to each visit a self-evaluation questionnaire should be filled in by the school that is going to be visited and be made available to the evaluators before the visit. It contains questions about hardware, overall vision and policies. The schools are also requested to send policy briefs and similar documentation to the evaluators.

- The interview guidance helps the evaluators during the school visit. There are three kinds of guidance: one for teachers, one for management and one for pupils. As part of the methodology, interviews should be scheduled with each of these groups. The questions are directly derived from the quality indicators and evidence pointers in the framework.

- An essential part of each school visit are the lesson observations. Each evaluator makes notes on a lesson observation form, where appropriate boxes can be checked about the subject, type of ICT used, type of classroom, etc.

- After the visit the local inspector writes a report, where he/she elaborates on the scores, providing strengths and weaknesses for each quality area. A template report is provided, with a clear format linked to the framework.

Methodology

The last part of the first day was dedicated to the feedback on the methodology used for assessing ICT schools’ performances. A very intense debate arose, focusing on several aspects observed or encountered during the visits on the ground. A common consensus emerged about the lack of time between the visits and the evaluation process, as well as on the necessity to reframe each school in its context. Later on the group slit up in two to discuss the lessonobservationform and the agenda for the discussion with pupils.

Some people also wished to exploit more deeply the possibility to talk with teachers to better understand what kind of difficulties or facilities they are faced in the school. The differences between schools’ reactions to inspectors’ reports was also an important point, whose explanation should be found in the different cultural and educational system backgrounds.

Which criteria should be then retained to scale up this inspection framework methodology for assessing the use of ICT in European schools? The Spanish participants pointed out issues such as the necessity of finding a minimum number of observations, which should also be representative samples of standard lessons. For instance, in the Lithuanian schools, as Asta Buineviciute recalled, they assisted in special lessons prepared on purpose for the inspectors’ visit. On the other hand, the Scottish inspectors suggested a thematic focused approach, where the observation of one random class would suit best the purpose. The Swedish inspectors discussed whether your judgment can depend on lessonobservations. Bert Jaap van Oel explained that an inspectors uses several instruments to come to any judgement.

The agenda for the discussion with pupils is useful. The inspector can reformulate the suggested questions. Some issues the inspector observed already, so he/she doesnot have to ask them anymore.

Debate goes on at the European School N. 2 in Brussels

On day two the SICI workshop moved to the European School n. 2 in Brussels, where debates and a visit to the school were organised. Some new guests joined the inspectors' team, gathering experts from Germany, UK, Belgium and Romania, as well as from the EC Education Audiovisual & Culture Executive Agency in charge of the P2V project.

The inspectors reported to the experts the main findings of their visits discussed the day before, as well the methodology used and the general framework of the Inspection strand of the P2V project. Roger Blamire from EUN recalled as well the whole P2V project' structure and the latest steps for each of its strands (School, Policy and Inspection), whose documentation is fully available on the P2V website (http://p2v.eun.org/ww/en/pub/p2v/homepage.htm) and related wiki (http://p2v.wikispaces.com/).

First feedback from the Netherlands

A very interesting feedback from the P2V inspectors' visits came from the presentation by Pieter Hogenbirk, principal of the Helen Parkhurst secondary school in the Netherlands. The Helen Parkhurst secondary school was one of the first two schools visited in the framework of P2V, together with De Triangel school, as pilot experience to test the ICT assessment templates and methodology as well as to let inspectors get used to it and to know each other. The visits took place on September 2007.

Helen Parkhurst school

Helen Parkhurst is a school for general secondary education for students aged 12-18, where it is in use the Dalton pedagogical method: objectives are fixed by teachers, but order, pace and methodology of the lessons can be determined by students, in close agreement with their teachers. Many activities are based on collaboration with other students, and reflection on them and on the results of the learning process are very important. The school has some 2050 students and is located in Almere.

The ICT curriculum is divided over different subjects. Mediaplus Office is used to train students in the lower forms for the use of a word processor, presentation tools and spreadsheets. In the higher forms the subject Computer Science can be chosen, and also modules from the Cisco Academy. At Helen Parkhurst ICT was especially used by the pupils for independent and group work, which reflects the value the school puts on self-directed learning.

At the time of the visit (on 18 September 2007), inspectors could observe some very confident and reflective students eager to see ICT used more by teachers. The computer/pupil ratio was 1/5.4, with computers distributed in open learning centers, special ICT classrooms and normal classrooms. It was also available one movable laptop trolley. Almost all of the computers were connected to broadband. Other facilities such as digicams were also in place in order to support students in their learning.

The inspectors' evaluation: one year later

The inspectors gave quite a positive evaluation on average, but the scores highlighted a significant lack of coherence of vision and weaknesses in the teaching process. According to the Helen Parkhurst principal, Pieter Hogenbirk, also the communication with parents and equipment should have been improved, although inspectors' remarks were not negative. The school teachers and staff reacted to the evaluation given by P2V inspectors by underlining both the nature of a Dalton school, where pupils are given responsibility of the learning process, and the lack of adequate ICT teacher training.

How to respond to the challenge and improve the use of ICT at Helen Parkhurst school? One year later, Pieter Hogenbirk is eager to show the inspectors the initiatives his school has undertaken in the last year, following the path they traced with their evaluation on September 2007. His actions have mainly targeted four fields, in order to make the overall ICT policy more coherent: equipment, teachers training and valorisation, communication with parents and assessment of the use of ICT.

Now, at the beginning of the 2008 school year, the situation is as follows:
- The computer/pupil ratio is now 1/3.8. In particular, eight digital trolleys (140 laptops in total) are now at the students' disposal, a wireless connection has been set up, while also the number of computer for teachers and management staff has been implemented (220 in total), as well as the number of digital whiteboards.
- More teachers have followed and are following different training sessions via the system of voluntary vouchers. Furthermore, the participation in the Grassroots programme of the Kennisnet gives the opportunity to individual teachers to valorise his/her teaching project, the Dutch government awarding the best ones with 500 euros per teacher.
- The digital learning environment used by the school is now accessible also from home. The parents have given a very good response, as they can now access all the information about their children (grades, attendance, timetables, teachers' communications, etc.).
- The setting of research projects on ICT effects as well as priority themes for 2008 such as a more advanced use of ICT (development of a new web portal, pilots in digital examinations, etc.) and the participation in international ICT projects.

Snapshots from the European School

Another major moment of the SICI workshop was the focus on the use of ICT at the European School N.2, located in Woluwe in Brussels. The principal, Richard Galvin, and the responsible for ICT, Frank Charlier, after a warm welcome gave a general overview of the functioning of the European Schools' network and an in-depth explanation about the use of ICT in the school. A visit to the building's facilities and classrooms completed the overview.

The school, well equipped (including also 30 smart boards and almost a beamer per class, geography, music, science and ICT labs, equipped library, etc.), is using at the moment the Microsoft Learning Gateway solution as learning management system. But next year it will switch to Studywiz, by the Australian Etechgroup, whose cost is respectively 3 euros per primary pupil and 5 per secondary student. The distance learning programmes are run by Adobe Connect, while the operative system is Windows XP or Vista. The trend is towards wireless and mobile in classrooms. The annual budget for ICT is 300.000 euros, which in majority is used for funding equipment.

The European School ICT focus is on teacher presentation tools and on the sharing of contents, as European Schools are a network of 14 institutions based in seven different countries and working with several languages in each school. Therefore distance learning and cooperation between teachers is a priority issue. The use of ipods (especially for teaching languages in primary school), gaming and edutainment as well as and learning to learn are other important issues.

On the contrary, the use of social networks such as Facebook is not enhanced because, as Richard Galvin recalled, "we should not forget that we are a school". Neither teachers' TV is considered a major tool, as teachers prefer learning objects to assemble by themselves according to their needs.

Other important issues for the European School are also esafety, accessibility (for students, parents and teachers), administration's efficiency and a sustainable technical support.

Similarities more than differences: the path to follow in the future

All the participants were asked to answer two questions: What can I do? What can my organization do? A summary of the comments and wishes:
• Hopefully the framework will be used as self-evaluationform by schools.
• Good practices or grade illustrations will provide an effective use of ICT.
• Use the framework to access ICT in education for children with special needs and children with other learning problems.
• Design a study of the impact and attainment of ICT in education.

At the end of the meeting, Hester Visser from the Dutch Inspectorate gave the direction to follow for the future, which must be guided by the similarities encountered on the ground in every country and school during the P2V Inspection visits.

"At the beginning of our experience, I expected to find a lot of differences, but now what I remember the most are similarities. Education in Europe, its issues and its challenges, are almost the same across all the countries. Of course, sometimes we are surprised to see how people work in schools, but for the future we should focus and work on similarities between our educational systems".

At the end of the meeting Roger Blamire gave a presentation about EUN and about the other workpackages 4 and 5 of P2V: Policy Peer Learning and Schoolvisits. The participants were also informed about the EUN Eminent Conference in Rome on the 4th and 5th of December 2008.