LEONARDO
A Model for the Evaluation
of Continuing Training
A process to determine how and whether the training project (in terms of input, output and organisational procedures) fits predetermined, structural and procedural standards (e.g. I.S.O.).
Training aimed at preserving and developing the skills of different occupational groups.
The entire set of procedures devised to ensure that the project has been properly executed. Financial control: accounting verification of the projects financial documentation and its compliance with administrative, civil and penal laws. Technical control is the administrative function aimed at verifying that the input (human resources, equipment, and material) to be provided through the project conforms to the technical and international specifications and to what foreseen in the Terms of Reference.
The coherence of the output of a training project with the needs of the user stakeholders.
What must be paid (quantifiable or not) in exchange for something else, often used relative to purchase of input. Costs are either direct and indirect, the difference being that the latter are not considered in calculating the projects rate of return. Indirect costs include those for environmental protection and social security, which are calculated separately, not forming part of the projects account. Cost may be estimated at market prices or shadow prices; in the latter case they reflect the alternative use of the resources that have been used to produce the goods in question (opportunity cost) as well as national policy objectives.
Method of appraisal or evaluation consisting of comparison of the costs and benefits of the project. The results may be expressed in a variety of forms, such as Rate of Return (i.e. the index indicating and measuring the relative profitability of a project) and Net Present Value (i.e. the difference between the value of the flow of benefits and costs, actualised at a date of project inception). While the calculation of financial advantage is a form of cost-benefit analysis, it does not provide the most satisfactory gauge of the net return of a project to the economy. Often market prices do not reflect the real economic value of input and output in terms of relative scarcity (or opportunity cost) or economic policy objectives. In such cases people use accounting prices of the economic policy objectives or shadow prices and national parameters to conduct an economic cost-benefit analysis.
COST-EFFECTIVENESS ANALYSIS OR COST MINIMISATION
A method of evaluation consisting of comparing alternative projects or alternative techniques within a single project when the material objectives are pre-set and the value of benefits cannot be adequately measured: for instance, projects in education, health and vocational training. If it may be assumed that the benefits (unquantifiable) nonetheless exceed the costs and that the necessary adjustments are made for differences in benefits between the alternatives, the purpose of the analysis becomes to minimise costs.
EARNINGS PERFORMANCE STUDY
A technique of training evaluation based on the idea that an individuals salary over the course of their life will exactly reflect contribution to output. This theory is based on the fact that investment in education and training, by improving individual skills and knowledge, will influence individuals earning capacity over time.
With reference to training, the benefits of using courses or modules for a larger number of trainees studying in related areas.
The extent to which a project or programme achieves the objectives set during the planning stage.
Ratio between input (human and material) and output obtained. High efficiency implies attaining the objectives at the lowest possible cost.
Assessment of the training project in terms of consistency between means and objectives. Evaluation is conducted via the application of a set of procedures and indicators, aimed at checking the degree of attainment of specific objective functions determined by the evaluating stakeholder. To be divided into: (a) Appraisal, i.e. the critical examination of a project as described, generally, in an initial identification report. During this stage the stakeholder selects and grades alternative solutions from the standpoint of i) feasibility (technical, economical and institutional) and ii) costs-benefits ratio. Appraisal immediately precedes approval by funding authorities; (b) On-Going Evaluation, i.e. the evaluation of a project carried out when it is under way, with the purpose of drawing indications for the successful conclusion of the project; (c) Final evaluation, i.e. the evaluation carried out when the project has been completed, with the purpose of verifying whether, and how, the project attained its pre-set objectives (in terms of effectiveness and efficiency) as well as to imagine possible adjustments for similar future operations; (d) Impact evaluation is the assessment of the effects of the project on the concerned stakeholders at a designated time after the completion of the activities.
EVALUATION OF COSTS BY STAGE, OUTPUTS OR PROCESS ELEMENTS
Applied to a training process or structure, a technique involving the quantification, in monetary terms, of the cost of the components of the training activity or facility. These costs are then reduced to standard units in order to quantify the resources needed to train one person, or the cost of the single unit of training time or a single stage of training.
Consequences of a project (beneficial or harmful) that are not reflected in market exchanges. For instance, a project may produce environmental damage, or it may make it easier for certain firms to initiate a given activity and so on; these effects are not registered in project cash-flows.
Study activity aimed at analysing all the different technical options envisaged to carry out a pre-identified activity and their realisability given the set of actual constraints. A feasibility study usually covers technical, financial, social and institutional aspects with respect to a training project.
Refers to two aspects of project assessment: i) using the data collected during monitoring and on-going and/or final evaluation as a guide to improve the current and future projects; ii) the adpting of counter measures to overcome the weaknesses and shortcomings observed during the project implementation.
The increase in benefits generated by the project. This is generally measured as the increase in goods and services or on their value that takes place as a consequence of the project.
A term used by economists to refer to investment in skills. This may refer to training or experience. It is assumed that wages reflect this investment.
Qualitative and/or quantitative measures which serve as a proxy for different aspects of the process being evaluated. An indicator represents complex phenomena and measures their evolution over time. In choosing indicators the relevant considerations are their importance for stakeholders, their significance, the ease and reliability of their measurement, their synthetic capacity and the repeatability of their measurement over time. There must be a causal link between the indicators selected and the set of objectives.
The resources, both tangible and intangible, that go into a project in order to produce the expected output and achieve the project objectives.
I.S.O. - International Standardising Organisation
International association, established in 1946 by the unifying bodies (UNI), in charge of drawing and disseminating unified rules.
Assessment and analysis of tasks, skills and activities. These are investigated with respect to their functions which are seen in terms of duration, interactions with regulations, and costs.
Process which results in a relatively steady change in the stakeholders knowledge, and/or attitudes and/or skills.
Tool that ranks objectives, activities, hypothesis and conditions of a project/programme within a matrix, according to their hierarchical level. The logical framework enables the design of a specific activity or an integrated set of activities to be checked and, at the same time, it shows the relations existing between project activities and objectives. The utilisation of the logical framework makes it possible to design integrated programmes which are reliable in terms of efficiency and effectiveness, and serves to facilitate their planning implementation, monitoring and evaluation (see Commission of the European Communities, Manual - Project Cycle Management - Integrated Approach and Logical Framework, Brussels, February 1993).
Basic element of the managers job comprising of a broad spectrum of monitoring and trouble-shooting activities, aimed at ensuring the satisfactory carrying out of the project. In many cases, it is the key element of the managers control procedures, reflecting a personal style and approach to project activities.
A methodological scheme through which different interacting elements are represented as a whole. At the same time, the basic links and relations which are supposed to exist between these elements are clearly defined. The objective is to establish a replicable basis of reference for the design and evaluation of activities which are inherent to these elements.
A management function which, through systematic data collection, checks whether the inputs used in a project are sufficient; whether the relevant stakeholders are adequately trained and qualified; whether the activities conducted correspond to those in the Terms of Reference and whether the objectives of the working plan have been attained. A basic feature of training evaluation is the link between monitoring and correspondence.
The capacity of a training project to generate a virtuous circle, disseminating the key elements in its approach (also known as transferability) or making possible the repetition of the scheme with greater effectiveness and efficiency. The degree of transferability of a vocational training project depends on the availability of complete documentation collected in the execution of the feasibility study and the willingness of the concerned stakeholders to actually transfer their skills and competences.
Preliminary phase of the training project design. At this stage relevant stakeholders define project objectives, procedures and implementing measures, as well as anticipated results.
The purpose or goal of a project or programme. The extent to which an objective has been attained must be measurable by indicators within a Logical Framework. In relation to vocational training projects, the objective of the activity is the modification (in terms of upgrading and improvements) of the skills of concerned stakeholders.
The value of what is given up. The opportunity cost of a good is what its purchaser could have bought or obtained by allocating the scarce resources in a different way. For instance: the opportunity cost of an individuals working day is the value of what that person would have produced had he/she not interrupted regular activity to work on the project in hand.
The way people interact (acting together towards the same goal but carrying out different tasks) to determine and realise agreed pre-identified objectives.
Specific goods or services which are expected to be produced through the use of input.
A concept to be divided into: (i) Market Price or the price of goods in the domestic market; (ii) Economic Price which reflects the relative value that should be assigned to the input and output if the purpose of the economy is to maximise production. There is no consideration of income distribution or other objectives except efficiency; (iii) Shadow price or the price that reflects the social value of goods and services; (iv) Social Price is a price reflecting the value which society as a whole assigns to an output, taking into consideration not only productive efficiency but the achievment of significant social objectives such as the equalisation of income distribution, the reduction in the consumption of substances generally considered harmful (alcohol, tobacco) and increased output of goods satisfying basic needs (education, training).
Set of inter-linked activities structured, considering time and space, in a way to transform a given initial input in a specific output.
The ratio between the outputs and the inputs used in production. Changes in labour productivity are measured by an index number consisting of the number of hours of work divided by the obtained output.
A number of schemes and activities, differing according to the economic branch or sector, but converging on the same objectives.
A single scheme or a set of integrated activities intended to achieve set objectives within a specified time and following a precise plan. A project may form part of a programme.
The six stages of a PROJECT: (1) design and indicative planning; (2) identification (pre-feasibility); (3) Formulation (feasibility); (4) funding; (5) implementation and monitoring; (6) evaluation. (see, Commission of the European Communities, Manual - Project Cycle Management - Integrated Approach and Logical Framework, Brussels, February 1993).
The coordination and control of different elements of a project.
According to I.S.O. requirements, quality is the combined set of rules and characteristics of a good or service that determine its capacity to satisfy explicit or implied requirements or needs of concerned stakeholders. In the evaluation of vocational training projects quality is the overall indicator. High quality training inputs and output involve at the same time, a high level of efficiency, effectiveness, correspondence, as well as a high multiplier effect.
In the framework of a vocational training project, the certainty of employing training methods which are established, well-known, tested, and, a priori, seem to guarantee high quality standards.
The main tool for presenting the data collection. List of relevant information describing the major features of the project/action to be evaluated.
Structured set of knowledge, crafts and attitudes enabling the realisation of specific tasks. Skill Audit is the survey of skills held by the workforce.
A person or institution involved in some way in the training project. Stakeholders may be grouped in the following categories (i) Decision -maker or Policy maker, the stakeholder responsible for initiating a specific training project. Decision-makers may be Financing Institutions, i.e. public or private entities (companies, international organisations, governments) financing training projects.(this is the case of training implemented with national and/or EU funds) or training entrepreneur, i.e. an individual or company offering a training service on the market to trainees who pay for it; (ii) Training-Manager, or the person responsible for carrying out training activities and taking care of the organisational (manager) and didactic (trainer) matters; (iii) Individual User, i.e. a person who applies to participate in a vocational training project in order to increase his/her human capital and obtain relevant training and qualifications; (iv) Institutional User, i.e. the agency, organisation or company interested in selecting a training activity aimed at achieving objectives.
The total combined quantity of input available to an organisation in a project. We use the term synergy to describe the situation when the quantity is larger than the sum of each single input made available by each single component of the organisation.
Activity based on data collected throughout the job analysis and aimed at assessing the job content which may contribute to the identification of training needs.
The set of main issues describing the broad contents of the requested vocational training project proposal. It must comprise of at least: (i) a description of project objectives and training results and achievements; (ii) the broad definition of foreseen activities, with description of relevant inputs, didactic methodologies and outputs; (iii) an estimated cost for carrying out the project; (iv) an indication of the criteria for the organisation and planning of the requested activities.
Training activity characterised by training situations directly implemented in work places. Trainees are assisted by tutors and/or supported by experts with specific skills.
Within a vocational training project, the stakeholder who (i) supports, facilitates, activates and fosters the learning process and (ii) assists the trainer and co-ordinates classroom activities.