Instructions: How to assess the weights of a course's learning outcomes?

This material presents the arguments for why it is necessary to assess the weight (relative importance) of learning outcomes and which methods to use for that purpose, depending on the educational context.

Participation in this activity contributes to the learning outcome: to analyse learning design in a digital environment and educational context.

Why assess the weight (importance) of learning outcomes?

Assessing the weight (importance or priority) of learning outcomes enables:

• alignment between learning outcomes, learning and teaching activities, and assessment activities (constructive alignment)

• well-founded distribution of time, resources and assessment across learning outcomes

• clearer allocation of ECTS credits according to actual student workload

• higher quality course design and transparent communication with students and colleagues.

Steps in assessing the weights of learning outcomes

Step 1. List and understand the learning outcomes of the course

• Write down all learning outcomes of the course (as stated in the official course description).

• Consider the taxonomy level (e.g., Bloom’s taxonomy) of each outcome – they differ in complexity (e.g., “describe” vs. “analyse” vs. “create”).

• Clearly define what exactly a student needs to know, understand, or be able to do to achieve each outcome, and revise the planned learning outcomes if needed.

Step 2. Select criteria for assessing weight

According to Divjak et al. (2021, 2024), it is recommended to use multi-criteria decision-making for evaluating the importance of outcomes. Three basic criteria teachers may use are:

1. Complexity of the learning outcome
   – How cognitively demanding is the activity the student must perform?
   – It can be assessed using Bloom’s taxonomy (e.g., “remember” < “apply” < “synthesise”), but other taxonomies may also be used.

2. Contribution to achieving programme-level outcomes
   – How important is the course outcome for achieving programme-level learning outcomes (competences)?
   – Is the outcome essential for advanced courses or professional competences?

3. Time required for students to achieve the outcome
   – How much time (in hours) does a student realistically need to achieve that outcome (including studying, assignments, exercises, exam preparation)?

Note: You may identify additional criteria relevant to your programme context and use them to assess the weight of learning outcomes.

Step 3. Choose a method for assessing relative weights. Allocate scores for each criterion.

We recommend using moderately demanding methods for assessing criteria and weights, but in Divjak et al. (2021, 2024) you can also find more complex methods (e.g., AHP for group decision-making) as well as simpler ones (e.g., expert judgement).

Assign a score from 1 to 5 for each learning outcome according to each criterion.

Table 3. Example of determining the relative weights of learning outcomes according to three equally important criteria

Note: In the example, all criteria have equal importance, but you may apply weighting (e.g., complexity × 4, contribution × 3, time × 3) if you want to further differentiate the significance of each criterion.

Step 4. Relative weights of learning outcomes and analysis of results

• After determining the nominal weights of learning outcomes, calculate the relative weights so the total equals 100.

• When calculating relative weights, round the values and check whether the proportions correspond to the initial assessment.

• Ask students how they would assess the weights, but keep in mind that they will likely assess based on workload.

• Ask colleagues for their assessment, but remember that they are likely to consider cognitive complexity more strongly.

• Ask programme leaders or vice-deans, but expect that for them, importance will be based on contribution to programme-level outcomes.

Practical application

This method is successfully used in the design and revision of study programmes (Divjak et al., 2024) and enables teachers to make transparent, well-reasoned and collegial decisions about course structure.

Recommended literature

Divjak, B., Kadoić, N., Žugec, B. (2021). The Use of Decision-Making Methods to Ensure Assessment Validity. IEEE TEMSCON-EUR, 386–391.

Divjak, B., Svetec, B., Rienties, B., et al. (2024). Prioritizing Learning Outcomes in Different Learning Design Contexts. CECIIS 2024, 333–340.

Appendix: Bloom’s taxonomy

Bloom’s taxonomy is one of the best-known and most frequently used models for classifying educational goals and learning outcomes. It was developed to assist teachers in planning, structuring, and evaluating learning.

The original version dates from 1956, and the revised version from 2001.

Original taxonomy (Bloom et al., 1956)

The original taxonomy presents a hierarchical system of cognitive processes aimed at classifying increasing levels of complexity in thinking. It includes three domains:

• cognitive (knowledge and intellectual skills)

• affective (attitudes and values)

• psychomotor (physical and motor skills)

The cognitive domain – the best known – consists of six levels:

1. knowledge – recalling facts and information

2. comprehension – explaining meaning of concepts

3. application – using knowledge in new situations

4. analysis – breaking down information and identifying relationships

5. synthesis – creating something new from elements

6. evaluation – judging value or quality

The original taxonomy supported curriculum development, exam question design, and creation of standardised tests because it offered a clear structure for planning learning outcomes of different complexity.

Revised taxonomy (Anderson & Krathwohl, 2001)

The revised taxonomy introduces a two-dimensional structure and modernises terminology. It uses verb forms that better reflect cognitive processes and adds the knowledge dimension.

Cognitive processes:

1. remembering – recalling and recognising knowledge

2. understanding – interpreting, summarising, explaining

3. applying – using information in new contexts

4. analysing – breaking down, differentiating and relating parts

5. evaluating – judging based on criteria and standards

6. creating – generating new ideas, solutions or products

Types of knowledge:

• factual knowledge – essential facts and terminology

• conceptual knowledge – relationships among ideas

• procedural knowledge – methods, techniques and processes

• metacognitive knowledge – awareness of one’s own thinking and learning processes

The revised taxonomy enables deeper understanding of cognitive levels by linking what the student knows (type of knowledge) with how they use it (cognitive process).

Figure 10: Bloom’s taxonomy

Why use Bloom’s taxonomy?

• • Helps in planning curriculum and formulating learning outcomes that are clear, measurable and hierarchical.

  • Enables alignment of learning outcomes, teaching activities and assessment (constructive alignment).

  • Serves as a tool in instructional design for selecting appropriate teaching methods and tasks based on cognitive complexity.

  • Provides a framework for evaluating teaching materials and assessing student achievement.

  Using Bloom’s taxonomy, teachers can systematically promote higher-order thinking skills, from remembering to creating, and foster deeper, more meaningful learning.

References

Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives. New York, NY: Longman.

Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York, NY: David McKay.

Divjak, B. (Ed.). (2009). Learning outcomes in higher education. Varaždin: TIVA – Faculty of Organization and Informatics.

Forehand, M. (2010). Bloom’s taxonomy: Original and revised. In M. Orey (Ed.), Emerging perspectives on learning, teaching, and technology (pp. 41–47). The Global Text Project.