This paper was presented at the
5th European Conference: "Competence Modelling for European HR and Policies:
Bridging Business, Education, and Training" (COME-HR)
– Brussels, Belgium 2011-11-09.
Slides for the presentation are available on
Slideshare
Reformatted for Simon Grant's publications
Simon Grant and Cleo Sgouropoulou
JISC CETIS, Institute for Educational Cybernetics
University of Bolton
Deane Road, Bolton, BL3 5AB, UK
asimong@gmail.com
Department of Informatics
Technological Educational Institute of Athens
Ag. Spyridonos, 12210, Egaleo, Greece
csgouro@teiath.gr
In areas where competence can be greater or lesser, a level of competence defines a reference point that someone may have, or may not yet have, attained. Levels may be specific to an area or, often, generic, in which case they are assessed for specific areas of ability. Levels must first be defined in frameworks, and then competence concepts can be assigned levels following those frameworks. The eCOTOOL competence model offers information structures both for defining levels and for assigning them. This is intended to contribute to effective interoperability specifications. Examples of defined levels stretch back in history to craft guilds, and today they come in many forms. Examples are here presented, which the eCOTOOL model covers well. The eCOTOOL competence model offers a good way of understanding what a level of competence is.
Keywords: level, ability, skill, competence, framework, level definition, level assignment, information model.
Information about people’s abilities and competence has always been valuable when decisions are made about who should undertake some work, or who should be engaged to perform a particular role as part of a job. As a consequence, it is also valuable to have information about what courses of learning, education or training (LET) are designed to result in the acquisition of what competence. With the widespread development of information and communications technology (ICT), it becomes important to be able to represent information about abilities and competence in ways that can easily be communicated electronically, so that ICT tools can process such information efficiently and effectively. Such tools could potentially help individuals, institutions and employers in (among other things) designing, planning and recording LET; in expressing claims and requirements for abilities and competence; and in recruitment and selection.
Some attempts have been made to represent information about competence in standardised ways. Well-established examples of this include RDCEO from IMS (2002) and its generalisation, RCD from IEEE (2007). Newer examples include the draft specification called “InteropAbility” (InterCom, 2011); work within the European ICOPER (2011) and eCOTOOL projects, and work underway in the MedBiquitous (2011) consortium’s "Competencies" working group.
In the ICOPER project, for example, the PALO model (Najjar et al., 2010, 2011) represents level elements thus: "This element captures ranking information about the learning outcomes and/or assessment records of learners. This includes proficiency level, interest level, weight, ageing." However, this is not intended to, and does not, cover the definition of levels.
In all cases known to the authors (except eCOTOOL) the representation of levels of competence is solely in terms of assignment, rather than definition, of levels. (The distinction will be clarified below.) For assigning educational levels, a long-standing approach shared by many of the existing initiatives is with Dublin Core (DC) metadata. DC has a term entitled "Audience Education Level" whose definition is "a class of entity, defined in terms of progression through an educational or training context, for which the described resource is intended." However, DC does not give any particular level scheme, system or framework; nor does it set out to represent information relevant to defining level schemes.
The eCOTOOL competence model (Grant, Sgouropoulou & Thanopoulos, 2011; eCOTOOL, 2011) is intended to provide a better, practical approach to representing the structure of concepts related to occupational competence. It offers an important step on the path towards interoperability of level definitions by including a model for level definition as well as a related definition of level assignment. This eCOTOOL model is built on some concepts that are useful to rehearse first.
People generally want to compare people's ability, and to engage the most suitable person they can find for any particular job, from an organisational role of responsibility, to a one-off job anyone might ask a tradesperson to do. Informally, they may do this through word of mouth reputation, but this is liable to be unreliable, mainly because different people's view on what is "good" or "better" may differ, unless the scale is defined more precisely. To achieve greater clarity, here a few useful concepts are defined.
An ability concept against which we can assess people as better or worse may be called a "rankable" concept, as in principle people can be ranked against it. Sports and entertainment give many familiar examples of this. Whether someone is a better or worse football player, or a better or worse singer, is at least loosely correlated with how successful they are in their career and how much they are paid. Rankable competence concepts from other areas could be seen to include programming ability, musical ability, typing speed and accuracy, linguistic proficiency in different languages. There are many more.
In contrast, if an ability concept is expressed in a way that it is either possessed or not – "attained" or "not yet attained" – we call it a "binary" concept. Examples of binary concepts might be:
In each case, it is reasonable to expect an assessment to give a yes or no result.
On the basis of understanding rankable and binary ability or competence concepts, it makes sense to think of levelling as an approach to defining binary concepts from out of what may be a rankable concept continuum. Levels are one good way of giving better definition to a competence concept that allows judgements of better or worse. They are a step forward from simply letting individuals claim they are good at something.
A clear distinction can be made between the ideas of defining and assigning levels. Most people’s experience with levels is confined to judging to which level something should be assigned. One can ask, what level of learning are particular materials and objects suitable for? Are a set of instructions suitable for beginners or experts? Language courses may start assuming no knowledge of the language at all, but equally may assume a certain proficiency, on top of which more will be learned. A course in calculus would most likely be incomprehensible to someone who had not mastered basic algebra. Thus we assume a set of levels, and assign things to those levels, perhaps without reflection on how the levels are defined.
In the field of European education and mobility, the EQF (European Commission, 2008) is a good example of a level framework to which one can assign things. In each of the three defined areas (knowledge, skills and competence) the EQF defines 8 levels. For example, the EQF says that competence level 5 applies to areas where people "exercise management and supervision in contexts of work or study activities where there is unpredictable change; review and develop performance of self and others". It is important to understand that this it is not assigning EQF competence level 5 to a descriptor. Rather, the given words, agreed by the Commission, define EQF level 5 competence. The EQF levels are not concepts that can be recognised independently of words used to describe them. When one assigns another concept to EQF level 5 competence, one compares the definition of the competence-related concept with the descriptor above, and one makes a judgement that EQF level 5 competence is the best fit of the level definitions offered in the EQF.
Wide ranging level frameworks like the EQF tend to define levels that we can call "generic", as they apply to many different specific areas of ability. For instance, EQF knowledge level 4 is described as "factual and theoretical knowledge in broad contexts within a field of work or study", and it is obvious that this is only able to be assessed in relation to a specific field or area. But as well as this kind of framework with generic definitions, there are also frameworks built for particular occupational areas that give specific criteria.
For example, in the Knowledge and Skills Framework (KSF) of the UK’s NHS (2004) there are 4 levels, applying to all their dimensions of knowledge and skill, but these 4 levels have no generic descriptors of their own. The KSF just gives specific descriptors for each level of each dimension of knowledge and skill. A example that is similar in some ways is the European e-Competence Framework 2.0 (e-CF), developed by the CEN Workshop on ICT Skills (2010), which defines both generic and specific descriptors for their 5 levels. The e-CF, as well as defining their own levels, also assigns the e-CF levels to EQF levels.
Competence-related concepts come in many forms. There are very broad and vague concepts such as "communication skills" that may act as headings for gathering evidence, but are not definite enough to allow any reliable assessment of people. A concept is rankable, as introduced above, when it gives enough definition to allow a comparative assessment of people against the concept, and a concept is binary, as introduced above, when it is defined sufficiently clearly to allow a "yes/no" assessment of whether a person has that ability.
Specific levels of competence are normally defined in such a way that they can be assessed as binary concepts. Generic levels, on the other hand, are not in themselves binarily assessable. To get a binarily assessable concept, one must put together a generic level and a specific field or area. The definition of the specified field or area would ideally in itself be a rankable concept.
The eCOTOOL competence model is based around an earlier idea that appears in other models of competence, including the MedBiquitous (2011) work mentioned above. This basic idea is that competence-related concepts can be defined separately, standing alone, and that information about the relationships between these concepts can also be given separately. Broader (greater) abilities are seen as containing (or including, or relying on) narrower (lesser) ones. This allows competence frameworks to be built up from separate definitions, and it maximises the potential for reuse of concept definitions, rather than requiring every framework to redefine, often in only slightly different words, what are essentially the same competence concepts.
But structuring competence definitions hierarchically does not of itself imply a treatment of levels. This is where the eCOTOOL model comes into its own, making use of the concepts discussed in the previous section, offering a new approach to modelling levels of competence. Most importantly, assigning and defining levels are seen in the eCOTOOL model as separate, though closely related, ideas.
The eCOTOOL model allows any ability definition to be assigned a level within any number of level schemes or frameworks. The example given with the eCOTOOL high-level competence model explanation (Grant & al., 2011) (written to give background to pilot testing of the eCOTOOL tools) gives an example from UK National Occupational Standards (NOS). "Service and maintain domestic natural gas systems and components" can plausibly be assigned to level 2 of the UK National Qualifications Framework,1 level 3 of the EQF (both of which are widely known schemes), and level 2 of eCOTOOL’s own level scheme. Any number of level schemes can be devised, for different purposes, and there is nothing to prevent anyone assigning any competence-related concept to any level scheme, published or not published. This is supported by eCOTOOL, with the constraint only that each level scheme should be identified by its own unique URI.
The eCOTOOL high-level competence model envisages the definition of levels as subtly different from the kind of "functional analysis" (Carroll & Boutall, 2011) in which broader concepts are analysed in terms of narrower concepts. The example of level definitions given in eCOTOOL’s high-level model relates to a higher education benchmark. Some learners study the topic of moral, ethical and social implications of agriculture. The subject benchmark accepts that the definition "recognise the moral, ethical and social issues related to the subject” can be displayed by learners to different levels. The benchmark specifies three generic levels, threshold, typical and excellent, that are defined for competence in this subject area. (The subject is there understood to be agriculture, horticulture, forestry, food and consumer sciences.) "Threshold" is defined as "recognise the existence of moral and ethical issues associated with the subject"; "typical" is defined as "recognise and be able to comment on the moral and ethical issues associated with the subject" and "excellent" is defined as "recognise, explain and evaluate the moral and ethical issues associated with the subject". The eCOTOOL example then goes on to suggest that, perhaps, threshold could be associated with the number 30, typical with 50 and excellent with 70, though these numbers are given as just one possibility. In practice, it would be up to the body defining the level scheme to allocate the numbers that it thought both appropriate, and most likely to be familiar and useful to those using the levels.
More precision is added by the eCOTOOL (2011) technical model. There, a single level definition is specified as having to contain the identifier of one levelled concept, effectively a binary concept that can be directly assessed; the identifier of the level scheme or framework; and a level number. Without the level number, there would be no reliable way of knowing which level is higher than which other level. In addition, the level definition may contain an unlevelled concept identifier, and a level label for human reading.
Once a set of level definitions have been created, it is possible to assign things either directly to the defined levels, or to a point between them, using the numbering system.
Defined levels have been features of society for a very long time. The traditional craft guilds could be said to have defined ability in a craft at three levels: apprentice; journeyman, and master craftsman. (There were other things one could also say they were doing, but these will not be discussed here.) This at least would allow someone to request a specific level of craftsman rather than simply relying on hearsay about who was "good".
To take an initial arbitrary example (found through web search), in the European fluid power industry, an organisation called CETOP2 has a web page called "Managing your level of competence", on which they define three generic levels. Level 1 is "Doing activities according to an established procedure. These activities are repetitive or of short duration. If faced with a problem, then calling for assistance or following a set procedure." Level 2 is "Carrying out various actions which need an understanding of technical factors. These could lead to an interpretation (tolerances, operating methods) or the application of various and non-repetitive instructions. This can imply checks, simple diagnoses and an ability to react to changes. Team work is often essential." Level 3 is written as "Imply a wide and complex range of activities, using one's initiative and making decisions on certain technical aspects (specifications, resources, processes). The ability to carry out a plan of operation, to identify and remedy errors is an important attribute. For these positions, taking responsibility (work quality, results) is expected." CETOP then assigns these levels to competence-based qualifications that they define themselves.
This is much the same in structure as many other generic level schemes, including the EQF (European Commission, 2008), and the language proficiency level scheme used by the Europass Language Passport. As mentioned above, eCOTOOL also offers a generic level scheme with 5 levels, for cases where there is no better option available. The only things that need to be supplied to fit the CETOP scheme into the eCOTOOL model are universal identifiers – for the framework as a whole, and for the three levels. The identifiers preferred by eCOTOOL are URIs, as the Semantic Web is set up to work with them, and they are a great aid to interoperability.
The European e-Competence Framework (CEN Workshop on ICT Skills, 2010), mentioned above, gives examples of more specific levels. For instance, taking an arbitrary example from the middle of the e-CF, e-Competence area D.9 has the title "Personnel Development", and its generic description is "Diagnoses individual and group competence, identifying skill needs and skill gaps. Reviews training and development options and selects appropriate methodology taking into account the needs of the individual and the business. Coaches and/ or mentors individuals and teams to address learning needs." According to the e-CF, this ability can be greater or lesser, and the e-CF defines it at their levels 2, 3, and 4. Level 3 has the definition "Monitors and addresses the development needs of individuals and teams", while level 4 is defined as "Takes proactive action and develops organisational processes to address the development needs of individuals, teams and the entire workforce." Particularly given the earlier examples, it is easy to see how this can be treated effectively within the eCOTOOL model.
The UK’s NHS (2004) KSF, also introduced above, is structured in a way that is quite comparable to the e-CF, and is equally easily represented within the eCOTOOL model.
A rather different explanation of levels is given by the relatively new UK QCF system where there are awards, certificates and diplomas (Skills Funding Agency, 2011). The QCF define level as "An indication of the relative demand, complexity and/or depth of achievement, and/or the autonomy of the learner in demonstrating that achievement." However, there is nothing essentially new about the level scheme itself, which is unchanged from the UK’s National Qualification Framework level descriptors (QCA, 2004). This is a generic level scheme similar in structure to many others.
The eCOTOOL competence model is able to represent separately both level definitions and level assignments as in the examples above. The way in which it clarifies that distinction, and related ones, is a significant step forwards towards a better understanding what a level of competence is.
The information models to be presented as eCOTOOL outcomes will serve as input for building interoperability specifications able to represent information about abilities, skills, and competence, complete with levels as described and used by existing competence frameworks. This will allow ICT tools to hold this information coherently, and in particular web-based tools can soon afterwards be expected to help with a wide range of practical tasks that involve information about competence and its levels.
Carroll, G., and Boutall, T. (2011). Guide to Developing National Occupational Standards. UKCES. Available from https://www.gov.uk/government/publications/national-occupational-standards (accessed 2016-10-24) (accessed from http://www.ukces.org.uk/ourwork/nos-strategy 2011-09-28)
CEN Workshop on ICT Skills (2010). European e-Competence Framework 2.0. (Was available from http://www.ecompetences.eu/ 2011-09-27, as version 3.0 as at 2016-10-24)
eCOTOOL (2011) Deliverable 1.4: eCOTOOL Competence Model. Available through http://www.competencetools.eu/ (accessed 2016-10-24)
European Commission (2008). European Qualifications Framework for lifelong learning. (Was available from http://ec.europa.eu/education/node_en 2011-09-27, now PDF copied here)
Grant et al. (2011). The eCOTOOL high-level competence model explanation. Available here in PDF format in English, German, Italian, Czech, Greek
Grant, S., Sgouropoulou, C. and Thanopoulos, C. (2011) A model for skills and competences and its application in the agricultural sector. In: Metadata and Semantics Research: 5th International Conference MTSR 2011, Izmir, Turkey, October 12-14, 2011: Proceedings. Eds: Elena Garcia-Barriocanal, Aydin Öztürk and Mehmet C. Okur. Springer.
ICOPER (2011). Interoperable Content for Performance in a Competency-driven Society. (Accessed at http://www.icoper.org/ 2011-09-27)
IEEE (2007). Data Model for Reusable Competency Definitions. IEEE Computer Society, Los Alamitos, California.
IMS (2002). IMS Reusable Definition of Competency or Educational Objective Specification. Available from: http://www.imsglobal.org/competencies/ (accessed 2011-09-27 and 2016-10-24)
InterCom (2011). InteropAbility: The Interoperable Competences Framework Specification. (Accessed at http://www.interopability.org/wiki/Main_Page 2011-09-27)
MedBiquitous (2011). Competencies. Web pages, available
from
http://www.medbiq.org/working_groups/competencies/index.html
(accessed 2011-10-08 and 2016-10-24)
Najjar, J. et al. (2010). Model for describing learning needs and learning opportunities taking context ontology modelling into account. (Accessed at http://www.icoper.org/deliverables/ICOPER_D2.2.pdf 2011-09-27)
Najjar, J. et al. (ed.) (2011). Report on implementation and validation of a standardized model for learning needs analysis and the provision of learning opportunities. (Accessed through http://www.icoper.org/results/deliverables/D2-3 2011-09-27)
NHS (2004). The NHS Knowledge and Skills Framework (NHS KSF) and the Development Review Process. Available at http://www.ksf.scot.nhs.uk/uploads/documents/KSF_Handbook.pdf (accessed 2016-10-24)
QCA (2004). National Qualifications Framework - Level descriptors.
Available from archive at
http://www.ofqual.gov.uk/downloads/category/157-nqf-level-descriptors
(accessed 2016-10-24)
Skills Funding Agency (2011). Qualifications and Credit Framework: iGuide.
(Accessed from
http://qcfiguide.skillsfundingagency.bis.gov.uk/ 2011-09-28)