The general purpose of an evaluation or a midterm review can be divided into five sub-goals:

• relevance
• effectiveness
• efficiency
• impact
• sustainability

Relevance

Relevance is the extent to which an aid activity deals with the problems and the needs of the target group. Relevance may change over the years. This may be the result of changes in the nature of the problem, changes in policy or the institutional context. The relevance of a project must be discussed at length before starting the project. A context analysis and a problem analysis must be conducted. In practice, sport development projects look at relevance to a limited extent in advance, but also in retrospect. It is seldom the purpose of an evaluation. Example of relevance questions would be: To what extent was it useful to set up a new sports project in a certain region? To what extent was the selected partner organisation a good choice for tackling the problem? To what extent was this need already being met by other organisations? Would it have been better to work together with other initiatives rather than build up an entirely new sport structure? But also: is there any point to using sport to work on gender issues in a cultural context in which women's sport is taboo? Might there have been better methods?

Effectiveness

Effectiveness is the extent to which an aid activity achieves the goals it has set, or the results of the project tally with the project's original purpose. Unfortunately, so far there are few good examples of evaluation studies in the area of sport and development cooperation.

Under a Memorandum of Understanding (MoU) between the Dutch Ministry of Health, Welfare and Sport and its South African counterpart, the Royal Dutch Hockey Association (KNHB) trained a total of some one hundred South African hockey coaches between 1999 and 2005. It was a new project without comparable previous experiences. A three-year agreement was concluded with the South African Hockey Association, one object of which was to put hockey in South Africa on a higher plane through the training of coaches. An SAHA staff member was enlisted to get the project off the ground - a process of trial and error which ultimately turned out to be quite successful. In its evaluation KNHB investigated to what extent the objective - organising training courses - had been achieved. This primarily reviewed activities and results. It did not look at efficiency, impact, relevance or sustainability.

Efficiency is measured on the basis of the human and material costs incurred in attempting to achieve a certain result.

Is the aid activity the most efficient activity, or would the same result or even a better result have been achieved in some other way? For example, is it possible to make use of cheaper local knowledge? Can a southern expert be enlisted in the place of a northern one? Are there less expensive sports activities that have the same or even more effect? Efficiency questions are often neglected. Organisations receive funding for a certain project. Once the funds have been allocated, no-one bothers to investigate whether things might be done more cheaply in some other way.

Impact

Impact is the general effect (both positive and negative) of the policy, project or programme on the target group. Often different forms of impact can be distinguished, direct and indirect, intended and unintended.

Various sport and development organisations offer courses for trainers. What happens with the knowledge gained in the course? Where do the trainers wind up? Do they wind up in a position where they can usefully apply their knowledge and use it to further develop the sport and strengthen the organisation? In 'A Perfect Match', Aad van der Meer reviewed the impact of the City Link Haarlem-Mutare sport project on the local sport structure and sport expertise in the municipality of Mutare.

Sustainability

Sustainability means that the results of the activity endure after the financing has stopped. It must be investigated in advance whether the objectives achieved can and will remain in existence without further external support.

Through its sport development pyramid, KNVB has the ultimate aim of having local instructors train trainers. In how far is this objective being achieved? In the collaborative project of KNVB and MYSA, this would seem to be largely the case. The instructors trained by KNVB train new trainers, and referees train new referees.

Most organisations active in the field sport and development, evaluate these sub-goals only to a limited extent. They do have some insight into the effectiveness of their activities in relation to the objectives formulated in advance, but knowledge about the efficiency, impact, sustainability and relevance of the activities is generally very limited.

ARCA/Todos: well begun is half done

A good example of investigating the relevance of a project is ARCA. ARCA is a Brazilian NGO that seeks to provide non-formal education to children and young people aged 6 to 18 who live in high-risk situations in slums (favelas). Their situation is high-risk because Brazilian favelas are characterised by high unemployment, poor schools, crime, drugs, weapons and violence. The object is to reinforce, through games and sports education, the target group's sense of their own value as well as the social values that help them to build lives for themselves and that help their community to function better.

ARCA is supported in the Netherlands by Todos, an organisation that works on consciousness-raising and educating the public about the lives of young people in slums. Todos shows that sport and games can play an important role in improving their living conditions. Generally speaking, an ARCA project starts with three investigations:

• one to show the need and possibilities for a project in a slum;
• one to study the possibilities for generating support for the project;
• one to assess the starting situation in the areas in which ARCA thinks the project can contribute.