Glossary
- a self‐evaluation process conducted by the faculty, the administrators, and the staff of the institution or academic programme, resulting in a report that takes as its reference the set of standards and criteria of the accrediting body;
- a study visit, conducted by a team of peers, selected by the accrediting organization, which reviews the evidence, visits the premises, and interviews the academic and administrative staff, resulting in an assessment report, including a recommendation to the commission of the accrediting body;
- an examination by the commission of the evidence and recommendation on the basis of the given set of criteria concerning quality and resulting in a final judgment and the communication of the formal decision to the institution and other constituencies.
The instrument by which one institution, without its own degree awarding powers or choosing not to use its awarding powers, gains wide authority to award, and/or gains recognition of its qualifications from another competent authority. This authority might be the State, a government agency, or another domestic or foreign higher education institution.
Institutional Accreditation: The terms refer to the accreditation of an entire institution, including all its programmes, sites, and methods of delivery.
Regional Accreditation (USA): Accreditation granted to a higher education institution by a recognized accrediting association or commission that conducts accreditation procedures in a particular geographic area. The United States has six regional accrediting commissions, which conducts accreditation of education institutions in a usually that of three or more states.
Specialized Accreditation: The accreditation of individual units or programmes (e.g. professional education), by “specialized” or “programme” accrediting bodies applying specific standards for curriculum and course content.
Duration of Accreditation: Accreditation decisions are limited in time. The duration of validity of the accreditation license is established by the accrediting body, which generally holds the right to suspend or to renew the license, upon the satisfactory resolution of any identified issues.
Accreditation Status: The formal recognition benefiting an institution or specialized programme for meeting the appropriate standards of educational quality established by the accrediting body at a regional, national, or specialized level.
Portfolio for Accreditation: An accumulation of evidence about specific proficiencies and the characteristics of an institution in relation to a specific type of activity, especially to learning standards. This operation can be performed either by the concerned institution or by an external assessor.
Accreditation Body: A (non‐)governmental or private educational association of national or regional scope that develops evaluation standards and criteria and conducts peer evaluations and expert visits to assess whether or not those criteria are met. It is entitled to accord formal status and sometimes a license to operate to individual higher education institutions or programmes, following the successful examination of the application and evaluation of the respective educational unit.
There are different types of accreditation bodies (e.g. agencies, councils, commissions, etc.), focused on general accreditation, specialized accreditation, professional accreditation, regional accreditation, national accreditation, distance education accreditation, etc.
- A technically designed process for evaluating student learning outcomes and for improving
student learning and development as well as teaching effectiveness.
Institutional Audit/Institutional Review: An evidence‐based process carried out through peer review that investigates the procedures and the mechanisms by which an institution ensures its quality assurance and quality enhancement.
Audit Report/Evaluation Report/Assessment Report:
- The document prepared following a quality assessment peer review team site visit that is generally focused on institutional quality, academic standards, learning infrastructure, and staffing. The report about an institution describes the quality assurance (QA) arrangements of the institution and the effects of these arrangements on the quality of its programmes. The audit report is made available to the institution, first in draft form for initial comments, and then in its final, official form. It contains, among other things, the description of the methodology of the audit, the findings, the conclusions of the auditors, and various appendices listing the questions asked. In Europe, the document is often called an “evaluation report” or an “assessment report”.
- Such a report may also be prepared about an accreditation agency, describing its quality assurance arrangements and the effect of these arrangements on the quality of the programmes in the institutions for which it is responsible.
Internal Audit: There are currently three main modes for the provision of internal audit within higher education:
- in‐house teams employed as staff members by the respective institutions;
- audit consortia (which may provide services to a number of clients both within and outside the sector);
- accountancy firms that undertake internal audits
Benchmarking gives the organization (or the programme) the external references and the best practices on which to base its evaluation and to design its working processes. Benchmarking is also defined as:
─ a diagnostic instrument (an aid to judgments on quality);
─ a self‐improvement tool (a quality management/assurance tool) allowing organizations (or programmes) to compare themselves with others regarding some aspects of performance, with a
view to finding ways to improve current performance;
─ an open and collaborative evaluation of services and processes with the aim of learning from good practices;
─ a method of teaching an institution how to improve;
─ an on‐going, systematically oriented process of continuously measuring the work processes of one organization and comparing them with those of others by bringing an external focus to internal activities.
Benchmarking implies specific steps and structured procedures. Depending on what is being compared or the type of information an institution is gathering, there are different types of benchmarking: strategic benchmarking (focusing on what is done, on the strategies organizations
use to compete); operational benchmarking (focusing on how things are done, on how well other organizations perform, and on how they achieve performance), or databased benchmarking (statistical bench‐marking that examines the comparison of data‐based scores and conventional performance indicators). Within different types, benchmarking may be either vertical (aiming at quantifying the costs, workloads, and learning productivity of a predefined programme area) or horizontal (looking at the costs of outcomes of a single process that cuts across more than one programme area). Some examples of benchmarking programmes are:
- The USA was the first country to introduce benchmarking activities into higher education in
the early 1990s. The NACUBO (National Association of Colleges and University Business
Officers) Benchmarking Project has been established longer than any other project in the
field. It started in 1991‐1992 and has had a statistical and financial approach to benchmarking.
- In the United Kingdom, benchmarking, as a quality assurance tool in higher education, came to the forefront only after the 1997 Dearing Committee Report.
- In Europe, benchmarking in higher education is not common, but a series of initiatives has been
developed: The Copenhagen Business School (CBS) benchmarking analysis of twelve higher
education institutions, 1995; The German Benchmarking Club of Technical Universities (BMC), 1996.
Internal Benchmarking: Benchmarking (comparisons of) performances of similar programmes in different components of one higher education institution. Internal benchmarking is usually conducted at large decentralized institutions with several departments (or units) conducting
similar programmes.
(External) Competitive Benchmarking: Benchmarking (comparisons of) performance in key areas, on specific measurable terms, based upon information from institution(s) that are viewed as competitors.
Functional (External Collaborative) Benchmarking: Benchmarking that involves comparisons of processes, practices, and performances with similar institutions of a larger group of institutions in the same field that are not immediate competitors.
Trans‐Institutional Benchmarking: Benchmarking that looks across multiple institutions in search of new and innovative practices.
Implicit Benchmarking: A quasi‐benchmarking that looks at the production and publication of data and of performance indicators that could be useful for meaningful cross‐institutional comparative analysis. It is not based on the voluntary and proactive participation of institutions (as in the cases of other types), but as the result of the pressure of markets, central funding, and/or coordinating agencies. Many of the current benchmarking activities taking place in Europe are of this nature.
Generic Benchmarking: A comparison of institutions in terms of a basic practice process or service (e.g. communication lines, participation rate, and drop‐out rate). It compares the basic level of an activity with a process in other institutions that has similar activity.
Process–Based Benchmarking: Goes beyond the comparison of data‐based scores and conventional performance indicators (statistical benchmarking) and looks at the processes by which results are achieved. It examines activities made up of tasks, steps which cross the boundaries between the conventional functions found in all institutions. It goes beyond the comparison of data and looks at the processes by which the results are achieved.
Culture of Evidence: This term is a mindset acquired in a higher education institution and based on clear ethical values, principles, and rules, which consists of the self‐evaluation of its learning outcomes engaging the teaching staff and the academic administration. As formulated within the new WASC (Western Association of Schools and Colleges) standards, the culture of evidence requested from a higher education institution implies that the institution is encouraged to be able to provide empirical data proving the consistency of its own mission.
Descriptors (Qualification): Qualification descriptors are statements that set out the outcomes of principal higher education qualifications at given levels (usually of an awarded degree) and demonstrate the nature of change between levels.
The first part of a qualification descriptor (of particular interest to those designing, approving, and reviewing academic programmes) is a statement regarding outcomes, i.e. the achievement of a student that he or she should be able demonstrate for the award of the qualification.
The second part (of particular interest to employers) is a statement of the wider abilities that the typical student could be expected to have developed. Upon periodical review of the existing qualification descriptors and in light of the development of other points of reference, such as benchmark statements, additional qualification descriptors at each level are elaborated. In view of the creation of the European Higher Education Area, a set of descriptors for Bachelor and Master’s degrees (known as the ‘Dublin Descriptors’) was developed by an international group of higher education experts (Joint Quality Initiative) and serves as reference for a number of national, regional and institutional needs.
Increased educational efficiency is achieved when the same amount and standard of educational services are produced at a low cost.
External Evaluation: The process whereby a specialized agency collects data, information, and evidence about an institution, a particular unit of a given institution, or a core activity of an institution, in order to make a statement about its quality. External evaluation is carried out by a team of external experts, peers, or inspectors, and usually requires three distinct operations:
- An analysis of a self‐study report;
- A site visit;
- The drafting of an evaluation report.
Indicators identify performance trends and signal areas in need of action and enable comparison of actual performance with established objectives. They are also used to translate theoretical aspects of quality into procedures, a process known as operationalization.
An indicator must be distinguished from a measure, which is data used to determine the level of performance of an attribute of interest, and from a standard, which is the level of acceptable performance in terms of a specific numeric criterion.
Another distinction is made between the different types of indicators:
- Indicators of economy (following and respecting budgets);
- Indicators of efficiency (actual productivity or output per input unit);
- Indicators of effectiveness (degree of attainment of objectives).
A third and relatively consequent distinction is made between:
- Context indicators, that relate to the specific environment of a higher education institution or programme (social, economic, political, geographical, etc.);
- Input indicators, that relate to the logistical, human, and financial resources used by a higher education institution;
- Process indicators, that refer to the use of resources by a higher education institution, to the management of the inputs, and to the functioning of the organization;
- Output indicators, that concern the actual achievements or products of the higher education institution. This latter framework is also known as the CIPO‐model (i.e. Context, Inputs, Process, Outputs).
Performance Indicators: a range of statistical parameters representing a measure of the extent to which a higher education institution or a programme is performing in a certain quality dimension. They are short-term or long‐term qualitative and quantitative measures of the output of a system or programme. They allow institutions to benchmark their own performances or allow comparison among higher education institutions. Performance indicators work efficiently only when used as part of a coherent set of input, process, and output indicators. As higher education institutions are engaged in a variety of activities and target a number of different objectives, it is essential to be able to identify and to implement a large range of performance indicators in order to cover the entire field of activity. Examples of frequently used performance indicators, covering various institutional activities, include the number of applications per place, the entry scores of candidates, the staff workload, the employability of graduates, research grants and contracts, the number of articles or studies published, the staff/student ratio, institutional income and expenditure, and institutional and departmental equipment and furniture. Performance indicators are related to benchmarking exercises.
Simple Indicator: A more general type of indicator, expressed in the form of absolute figures, intends to provide a relatively unbiased description of a process. Simple indicators are less relative than performance indicators as they exclude any judgments or points of reference.
- to persons meeting pre‐determined qualifications to engage in a given occupation and/or use of a particular title;
- to programmes, based on the evaluation of appropriate plans, to operate before obtaining accredited status;
- to institutions to perform specified functions.
Licensing (in the case of persons) is usually obtained through examination or graduation from an accredited institution. In some countries, a period of practical experience may be required. In such a case, state licensing should not be confused with institutional or specialized accreditation.
Generally, each outcome statement should describe one effect of the instructional programme, and not accumulate several into one statement. Also, the statements should be clearly detailed and easily understandable by all teaching staff and students in the given area or department.
Outcomes Assessment: The process of evaluation and improvement of specific results of a higher education institution in order to demonstrate its institutional effectiveness. Assessment may concern the performance of teaching staff, the effectiveness of institutional practices,
and the functioning of departments (e.g. programme reviews, budget reviews, etc.).
It is a formative procedure used for institutional self‐study, financial retrenchment and improved understanding of the current needs of students.
Student Learning Outcomes: Statements of what a learner is expected to know, understand, and be able to demonstrate after completion of a process of learning as well as the specific intellectual and practical skills gained and demonstrated by the successful completion of a unit, course, or programme. Learning outcomes, together with assessment criteria, specify the minimum requirements for the award of credit, while grading is based on attainment above or below the minimum requirements for the award of credit. Learning outcomes are distinct from the aims of learning in that they are concerned with the achievements of the learner rather than with the overall intentions of the teacher.
Student Outcome Assessment: The act of assembling, analyzing, and using both quantitative and qualitative evidence of teaching and learning outcomes, in order to examine their congruence with stated purposes and educational objectives and to provide meaningful feedback
that will stimulate improvement.
Countable Outcomes: Results that can be quantified, examples of measurable outcomes include: numbers of persons who gain employment after graduation.
A review is usually based on a self‐evaluation report provided by the institution and can be used as a basis for indicators or as a method of judgment for (external) evaluation in higher education.
Professional Recognition: Refers to the right to practice and the professional status accorded to a holder of a qualification.
Sometimes it is equivalent to a license to practice.
Qualifications Framework: A comprehensive policy framework, defining all nationally recognized qualifications in higher education in terms of workload, level, quality, learning outcomes, and profiles. It should be designed to be comprehensible through the use of specific
descriptors for each qualification covering both its breadth (competencies associated with learning outcomes) and its depth (level). It is structured horizontally in order to cover all qualifications awarded in a system, and vertically, by level. Its purpose is to facilitate:
- curriculum development and design of study programmes;
- student and graduate mobility; and
- recognition of periods of study and credentials.
While certain higher education systems have their own qualification frameworks, others allow for the development of a wide variety of qualifications without providing an explicit framework. The emerging European Higher Education Area, is regarded by many as needing a pan‐European Qualification Framework.
Quality may thus take different, sometimes conflicting, meanings depending on
- the understanding of various interests of different constituencies or stakeholders in higher education (e.g. students; universities; disciplines; the labour market; society; a government);
- its references: inputs, processes, outputs, missions, objectives, etc.;
- the attributes or characteristics of the academic world worth evaluating;
- the historical period in the development of higher education.
A wide spectrum of definitions of academic quality has been used:
─ Quality as excellence: a traditional, elitist academic view, according to which only the best standards of excellence (usually meaning a high level of difficulty and of complexity of a programme, the seriousness of the student testing procedures, etc.) are understood as revealing true academic quality.
─ Quality as fitness for purpose: a concept that stresses the need to meet generally accepted standards such as those defined by an accreditation or quality assurance body, the focus being on the effectiveness of the processes at work in the institution or programme in fulfilling its objectives and mission. Sometimes quality in this sense is also labeled as a value for money approach owing to the (implicit) focus on how the inputs are effectively and efficiently used by the processes and mechanisms involved. Sometimes it is the value‐added approach when results are evaluated in terms of changes obtained through various educational processes (e.g. teaching and learning processes. A variation of the latter is the quality as transformation approach, which is strongly student‐centered. This approach considers quality as a transformation process, where the better is a higher education institution the more fully it reaches its aim - to provide students with specific skills, knowledge and viewpoints, which help him “to live and work in a modern society –a knowledge society”.
─ Quality as fitness of purpose: a concept that focuses on the defined objectives and mission of the institution or programme with no check of the fitness of the processes themselves in regard to any external objectives or expectations. Within this approach, one may distinguish alternative approaches developed in the 1990s: quality as threshold whereby certain norms and criteria are set, which any programme or institution has to reach to be considered to be of quality. In many European higher education systems, own variant defining quality is used: quality as a basic standard, closely linked to accreditation, and quality as consumer satisfaction (quality perceived as closely linked to the growing importance of market forces in higher education, that focuses on the importance of the external expectations of consumers -students, families, society at large- and other stakeholders);
─ Quality as enhancement or improvement: focusing on the continuous search for permanent improvement, stressing the responsibility of the higher education institution to make the best use of its institutional autonomy and freedom.
Achieving quality is central to the academic ethos and to the idea that academics themselves know best what quality is.
Each approach has advantages and disadvantages, being more or less suitable for a specific period of time or national context. In terms of evolution, there are permanent movement and oscillations between relative versus absolute, internal versus externally oriented, and basic versus more advanced and sophisticated notions of quality.
However, common to all of these quality approaches is the integration of the following elements:
- the guaranteed realization of minimal standards and benchmarks;
- the capacity to set the objectives in a diversifying context and to achieve them with the given
input and context variables;
- the ability to satisfy the demands and expectations of direct and indirect consumers and stakeholders;
- the drive towards excellence (Van Damme, 2003).
Quality Assessment/Quality Review: The actual process of external evaluation (reviewing, measuring, judging) of the quality of higher education institutions and programmes. It consists of those techniques, mechanisms, and activities that are carried out by an external body in order to evaluate the quality of the higher education processes, practices, programmes, and services. Some aspects are important when defining and working with the concept of quality assessment:
- the context (national, institutional);
- the methodology (self‐assessment, assessment by peer review, site visits);
- the levels (system, institution, department, individual);
- the mechanisms (rewards, policies, structures, cultures);
- certain quality values attached to quality assessment (such as academic values, traditional values (focusing upon the subject field), managerial values; pedagogical values; employment values.
Quality Assurance: An all‐embracing term referring to an ongoing, continuous process of evaluating (assessing, monitoring, guaranteeing, maintaining, and improving) the quality of a higher education system, institutions, or programmes. As a regulatory mechanism, quality assurance focuses on both accountability and improvement, providing information and judgments (not ranking) through an agreed upon and consistent process and well‐established criteria. Many systems make a distinction between internal quality assurance (i.e. intrainstitutional practices in view of monitoring and improving the quality of higher education) and external quality assurance (i.e. inter‐ or supra‐institutional schemes assuring the quality of higher education institutions and programmes).
Quality assurance activities depend on the existence of the necessary institutional mechanisms preferably sustained by a solid quality culture. Quality management, quality enhancement, quality control, and quality assessment are means through which quality assurance is ensured.
Quality assurance varies from accreditation, in the sense that the former is only a prerequisite for the latter. In practice, the relationship between the two varies a great deal from one country to another. Both imply various consequences such as the capacity to operate and to provide educational services, the capacity to award officially recognized degrees, and the right to be funded by the state. Quality assurance is often considered as a part of the quality management of higher education, while sometimes the two terms are used synonymously.
Quality Control: The process of quality evaluation that focuses on the internal measurement of the quality of an institution or a programme. It refers to a set of operational activities and techniques elaborated and used to fulfill requirements of quality. Often used interchangeably with quality management and quality assurance, it refers to an aggregate of actions and measures taken regularly to assure the quality of higher education products, services, or processes, with an emphasis on the assurance that a prescribed threshold of quality is met. It aims both at monitoring the process and at eliminating certain causes generating an unsatisfactory functioning. Sometimes a minimal quality control (mostly in the shape of some kind of certification) exists serving as a filtering mechanism in confirming that a higher education institution is fulfilling minimal agreed upon quality requirements and has appropriate quality monitoring procedures in place.
Quality Culture: It refers to a set of shared, accepted, and integrated patterns of quality (often called principles of quality) to be found in the organizational cultures and the management systems of institutions. Awareness of and commitment to the quality of higher education, in conjunction with a solid culture of evidence and with the efficient management of this quality (through quality assurance procedures) are the ingredients of a quality culture. As quality elements change and evolve over time, so must the integrated system of quality supportive attitudes and arrangements (quality culture) change to support new quality paradigms in higher education.
Quality Management: An aggregate of measures taken regularly at system or institutional level in order to assure the quality of higher education with an emphasis on improving quality as a whole. As a generic term, it covers all activities that ensure fulfillment of the quality policy and the quality objectives and responsibilities and implements them through quality planning, quality
control, quality assurance, and quality improvement mechanisms.
Total Quality Management (TQM): A particularly influential comprehensive approach to quality management that places emphasis on factors such as continuous improvement, customer focus, strategic management, need for explicit systems to assure quality of higher education, and a view of leadership and supervision that stresses employee empowerment and delegation. Such an approach to quality management emphasizes assessment that is undertaken of:
- defined objectives or standards (set internally or by external funding bodies);
- measures of customer satisfaction;
- expert and professional judgment;
- comparator organizations.
TQM is considered to have a close conceptual and philosophical link with benchmarking methodologies. Such an approach has been mostly applied in the economic sector, being less used in the academic world.
Quality Audit: The process of quality assessment by which an external body ensures that (i) the institution of programme quality assurance procedures or (ii) that the overall (internal and external) quality assurance procedures of the system are adequate and are actually being carried out. Quality audit looks to the system for achieving good quality and not at the quality itself. A quality audit can be performed only by persons (i.e. quality auditors) who are not directly involved in the areas being audited. The results of the audit must be documented (audit report).
Quality Planning: It consists of the set of actions that establishes the objectives and the conditions referring to the quality of higher education and to the application of the mechanism of the quality system. Quality planning includes product planning (identification, classification,
and determination of the importance of the features referring to quality as well as to the establishment of the objectives, the conditions referring to quality, and its restraints), managerial and operational planning (including its organization and programming), an elaboration of quality plans, and the provision of quality improvement measures.
The ranking process starts with the collection of data from existing data sources, site visits, studies, and institutional research. Following collection, the type and quantity of variables are selected from the information gathered. Then, the indicators are standardized and weighted from the selected variables. Finally, calculations are conducted and comparisons are made so that institutions are sorted into “ranking order”.
The results of ranking/league tables (the “scores” of each assessed institution) may thus vary from one case to another, depending on the number of indicators used or on the indicators themselves. Ranking indicators or criteria usually take into consideration scientific, pedagogic, administrative, and socio‐economic aspects: student/staff ratio, A‐level points (held by first-year students), teaching and research (as marks received in teaching and research assessments by individual departments), library and computer spending, drop out rates, satisfaction, study conditions, employment perspectives, etc.
With regard to institutions, recognition refers to the acknowledgement of quality assurance agencies or accrediting organizations, deemed to be trustful, efficient, and accountable institutions of quality assurance, following particular recognition standards set by the competent (usually foreign) recognition authorities.
Academic Recognition: Approval of courses, qualifications, or diplomas from one (domestic or foreign) higher education institution by another for the purpose of student admission to further studies. As regards the European Higher Education Area, three main levels of recognition can be considered, as well as the relevant instruments (as suggested by the Lisbon Convention and the Bologna Declaration):
- recognition of qualifications, including prior learning and professional experience, allowing entry or re‐entry into higher education;
- recognition of short study periods in relation to student mobility, having as the main instrument the ECTS (European Credit Transfer System);
- recognition of full degrees, having as the main instrument the Diploma Supplement.
Mutual Recognition: Agreement by two or more institutional bodies to validate each other’s degrees, programmes, or institutions and affirmation by two or more quality assurance or accrediting agencies that the methodology of the agencies are sound and that the procedures are functioning accordingly.
Standards may take a quantitative form, being mostly the results of benchmarking, or they may be qualitative, indicating only specific targets (e.g. educational effectiveness, sustainability, etc.). When quantitative, the standards include threshold levels that have to be met in order for higher education institutions or programmes to be accredited. More often than not, the thresholds or the “basic standards” are defined at the level of minimally acceptable quality. On other occasions, the standards refer to the highest level of quality, thus being considered as “standards of excellence”.
These may result from a benchmarking exercise or be asserted implicitly, being so recognized by the peers in a collegiate way. Standards may have different reference points: inputs; outputs, and processes. Standards can be general (for a degree level, e.g. a Bachelor’s or a Master’s Degree) or subject‐specific (e.g. discipline benchmarking statements in the United Kingdom).
Standards may also vary by different types of standard setting methods (such as criterion‐referenced, minimal competency, or objective setting methods). In order to judge properly whether or not a particular standard of quality is met, it has to be formulated clearly and explicitly and related to specific criteria which can be further divided into more operational indicators.
Standards are thus related to a specific culture of evidence. In the context of the growing diversity of higher education, the translation of academic quality into standards and indicators has become complex.
Often, a more dynamic approach to defining and assessing standards is visible (a mixture of reality‐based components and potentiality‐focused ones). The challenge is threefold:
- to diminish the number of reference standards (inputs; outputs, and processes);
- to relate them to appropriate performance indicators while also making use of specific criteria within a consistent culture of evidence;
- to provide for sufficient flexibility in the formulation of standards in order to allow for innovative academic developments.
“Standards” are often used synonymously with criteria, as in the United States, while in Europe, standards are becoming increasingly distinct from criteria.
Content Standards: Level of core competencies, relevant knowledge, and skills within a subject area, i.e. everything a student should know and be able to do. Content standards shape what goes into the curriculum and refer to required inputs.
Educational Standards: Level of requirements and conditions regarding different stages of the educational process and the relationship between those stages. Various types of educational standards exist with regard to learning resources, programmes, and results, in general, and student performance (content standards, performance standards, proficiency standards, and opportunity‐to‐learn standards).
Performance Standards: Levels of achievement that are deemed exemplary or appropriate, i.e. specifications of the required level of quality of a student’s work to meet the content standards. Performance standards shape expectations for educational outcomes.
or by a relevant examining board (validation by an outside examining body).
