Point of departure
What?
By online learning materials, the Working Group refers to entities available online comprising the content, tools and instructions required for teaching and learning (all elements need not necessarily be part of all online learning materials). The Working Group excluded from its study online material that was not primarily produced as learning material, such as the online services of periodicals and the public content and information, databases, and studies produced by museums, archives and libraries.

For whom?
The main target groups for the work as defined by the Working Group included the producers of learning materials (entire production teams, including teachers) and their users (teachers and pupils). In addition to these target groups, it is also necessary that learning material publishers, decision-makers in educational administration, education providers, researchers and those working in teacher training are aware of and employ the quality criteria.

Why?
The work of the Working Group is essential when creating a common view amongst the interest groups about what constitutes high-quality, appropriate and pedagogically sound online learning materials. The content production has been imbued with increased emphasis on the development of e-learning and the quality of the content is closely linked to the guidance and support activities related to e-learning.

The quality of online learning materials is ultimately affected by the same factors that affect any learning material: the selection of relevant content, familiarity with the target audience, expertise of the content producers, a didactic approach, the applied concept of learning, and the quality of communication and expressions. The quality of online learning materials as well as its usability as a support for teaching and learning is, however, also governed by a number of new phenomena and factors. The characteristics specific to online materials, such as updatability, interactiveness and communality, are to be taken into account
when defining quality criteria for them.

Providing online materials in the form of new media, particularly at its early stages, has not been free of certain “teething problems”, such as inappropriate elements originating from other media, unfeasibility, excessive features, compatibility problems, when the dimensions and possibilities of the new media were being tried and tested.

The use of online media has introduced new producers and new members to production teams who are not necessarily skilled in learning material production although they might be experts in the employed media, and vice versa. Online access allows for a wider use of the materials. Support materials that were previously used by individual teachers can now be shared over the Internet by several users. This conversion process means not only transferring information or materials via a new distribution channel, as it also involves several other important factors: appropriateness to the media, its interactive nature and thematic context, its general applicability outside the teaching context, and copyright and user rights.
The online environment enables distance learning, where not only the learning material used in face-to-face teaching but also the instruction and steering of the learning process are provided online. Many online learning materials make up complete modules or courses that include the actual learning materials (available online or otherwise) and online interaction. Sometimes these two cannot be categorically separated from each other, and from the point of view of learning this may not even be sensible – the content to be learnt exists, in a manner, in both the content and the interaction, and the true quality of the learning material will only become evident once in use and in the right context.

The range of online materials
In order to illustrate the types of online learning materials available, the Working Group categorised the various materials, although it soon had to acknowledge the relativity of such categorisation and decided not to present any specific model of it. The format of the learning materials evolves as online applications and functions improve and producers and users create innovative approaches.

Because materials vary widely in their form, functions and scope, it is obvious that a set of quality criteria may not be universally and absolutely applicable. The Working Group has worked on the premise that the quality criteria are constructed to be modular and flexible, so that they can serve as a basis for a compact and functional set of criteria for the evaluation of different kinds of online materials from different points of view.

The significance of meta-information
A user initially selects and evaluates online learning materials on the basis of information available about a given material. This selection work is made easier by systematic content descriptions so that different materials can be easily compared. Metadata plays an important role, indicating the quality and usability of the material in the initial evaluation. The Working Group finds that the use of metadata and the uniformity of metadata descriptions in Finnish online learning material production should be increased.

The quality of online learning materials is defined in production and use The guiding principle for the Working Group has been the view that evaluating the quality of learning materials separately from the production process and the use context gives only a limited picture of their quality. The development of online learning materials and innovative solutions is possible only if attention is paid to the production and quality management processes of those developing the materials. In large-scale, ongoing professional production, several quality assurance methods, either standardised or developed by the organisations themselves, are already in use. Those engaging in more sporadic production of online learning materials will need quality management tools and criteria that are jointly developed. Although the Working Group was assigned to draw up quality criteria specifically for online learning materials, it often adopted a wider approach related to e-learning in general. Quality criteria for online learning materials should, in fact, form an integral part of the general quality management and development of e-learning.

Quality criteria for online learning materials
The Working Group has defined quality criteria through a process in which Working Group members, as well as the interest groups involved and outside experts, have all participated. The work has been based on existing Finnish and international quality criteria and indicators. A research group from the Department of Psychology, University of Helsinki, compiled a literature overview to serve as a basis for studying the quality of online learning materials.

In terms of usability and production quality, the Working Group adopted the criteria issued by the government for public online services, which were then adapted and edited to suit the context of online learning materials. The pedagogical criteria were drafted in the workshops of the Working Group. The accessibility criteria were commissioned by an outside expert. At the time of writing, this work is still under way, so in the following we shall only present an outline of the criteria. In the final version, each section will be illustrated with practical descriptions and examples.

The criteria comprise four sections:
1 Pedagogical quality refers to features in the learning material that support learning and the applicability of the material in teaching and learning processes.
2 Usability refers to the technical structure and interface design of the learning material as well as the ease of use arising from these features. Usability is dependent on the user’s experience.
3 Accessibility in this context means that the online learning materials are accessible to everyone regardless of their age, physical or mental capacity, disabilities or health. The Working Group is aware that usability and accessibility
are to a certain extent two different interpretations of what is essentially the same thing. The Working Group wanted, however, to dedicate an entire section to accessibility so as to emphasise the inclusive nature of education.
4 Production quality. The production of online learning materials fulfils quality criteria if it is carried out in a controlled and documented manner, steered by knowledge-based, skillsbased and learning-based goals, and if the end product meets professional standards.

Promoting quality awareness
The Working Group will produce an evaluation tool based on the quality criteria issued by it to benefit the users and producers of the learning materials and other interest groups. Quality assurance has been selected as the theme in the forthcoming

national conferences on e-learning and information and communication technology in teaching. The criteria and quality assurance work will also be promoted by disseminating information to the various target-group communities and through publications. It has become evident in the course of the Working Group’s quality criteria project how relative and diverse quality is as a concept. The idea of quality changes with time and context. The Working Group has deemed it important to establish a forum for ongoing debate on the quality of (online) learning materials and suggests that an annual event for developers and researchers of online learning materials be organised, where the interested parties could update their knowledge and chart the situation concerning the online learning materials industry and bring to general attention the various aspects of online learning materials and their quality.