Assessing the Assessments

How we assess students in our course is diligently linked to how we teach the course. Assessment is a testimony to our teaching. Any assessment devoid of its link with ‘what to assess’, is likely to go haywire. Then comes the question of ‘how to assess.’ Our ability to link it with how we teach determines ‘assessment-of-learning’ vis-à-vis ‘assessment-for-learning.’ 

The end-term hall examination is about to start in the next 10-15 minutes. Many students have yet not taken entry into the hall. They are busy browsing through the notes. Some of them looking at books. Anxiety is visible on their faces. I really wonder: how does it help? One of my colleagues said it is their last hour’s effort to recall facts. Students opined that yes, it does help. I have also heard students saying that a good exam schedule is where there is a gap of a few days in-between. This allows them preparations time. Students have come forward on many occasions demanding a gap of more days between the two examinations. On the contrary, students have also been found blaming the non-availability of gaps for their poor performance in exams. Does it sound familiar?

This means that students do read specifically for examinations which is seemingly different from studying during the course. Blame it on the question paper design, which mostly tests students’ recall of facts, figures, and theories. This makes rote-learning important for passing the exam.

Teaching for exam versus teaching for learning

Rote-learning is an outcome of teaching style. When teaching is exam-centric, students have to be exam-focused. The scenario can be summarized in the following manner, from the teacher’s and student’s point of view:  

Why are we teaching—because students have to appear in the exam

and earn a degree!

Why are you attending classes—because we have to give exam

and pass it for obtaining a degree!

In the above scenario, the teacher focuses on finishing the syllabus, while students are interested in knowing what questions may be asked in the final examination. Even a cursory look at the question paper would reveal how we must have taught the course. Students are conscious of this fact and also get hurt when question papers are not designed properly to assess their learning. Too many questions starting with ‘what’ may require more fact-based recalling, while a question starting with ‘how’ and ‘why’ would require views, opinion, and critical appreciation. The combination of ‘what’, ‘how’, and ‘why’ used in the question paper would largely be linked with the teaching style adopted.

There are several underlying questions that can be as fundamental as to why do we want to assess, what we want to assess, and how do we want to assess. Arguably, answers to these questions would depend on the course learning outcomes and course objectives and it would differ across courses. Nevertheless, the overall contributions to the program-level outcomes and objectives need to be ascertained. Hence, the auditing part of question papers and answer scripts must provide insight into how the comprehensive written examinations have contributed to the achievement of course learning outcomes and the program learning outcomes. The outcome-based learning system makes a lot of difference and allows teachers to focus on students’ learning rather than merely passing the examinations. This also allows students to move away from rote learning, as it focuses more on the development of critical thinking and analytical ability. A comparison of the content-based learning system and outcome-based learning system is provided hereunder.

Objectives of Assessment

The overall aim of the exam and assessment is to facilitate the learning of important aspects in the course and to examine that the student has reached the minimum standard acceptable. It serves several useful purposes, some of them are listed below:

• To identify and list students who are able to achieve minimum level of standard prescribed for the course and also to identify and list students who are not able to meet the standard. This is necessary for making individual intervention plan so that brilliant students are pushed further forward and slow learners are helped to stand above the minimum standards.
• To extend feedback to students which help them improve learning. It is necessary that assessment tools become the learning source for students and hence feedback is extremely important. Feedback shall be oral as well as written as these two forms are not substitute rather they are complimentary. Written feedback shall be provided on the answer scripts itself so that students can see and use it for learning improvements. Answer scripts shall be shared with students and discussions to be made on questions. Few best answer scripts shall be shared with students.
• To use hall examinations assessment as a means of encouragement. This is to be reiterated that nothing works better for students than seeing one’s progress and success. Examinations must give idea of progression to students. Faculty must encourage students and congratulate them on progress made.
• To notice and encourage learning beyond curriculum. Students would tend to learn many things beyond the intended learning in the course. Faculty must identify some of the important beyond-course learning and encourage student to take it forward.
• To felicitate students with extraordinary achievement and outcome. Awarding and recognizing students with extraordinary results essentially set high standard which also becomes a guideline for others to follow. Certificates/Prizes/Medals should be instituted for students achieving highest accomplishment.

 What learning is to be evaluated?

It is desirable that students not only remember facts and figures but more importantly they are able to present them in a perspective, analyze them, and provide interpretation. Students should be able to synthesize thinking and apply knowledge. In the context of employability, students must also be abreast with the real world and the current state of affairs in the economy and industry, especially how things are applied in the real world. In a model of holistic learning, it is important that both cognitive and non-cognitive competencies are assessed. Benjamin Bloom in the mid-1950s provided three domains of learning:

• Cognitive, i.e. knowledge based
• Affective, i.e. attitude based
• Psychomotor, i.e. manual skills based

While preparing hall-examination questions, the above classification comes in very handy. The decision on the level of cognition to be tested can be helpful in selecting the kind of verb that can be used for writing questions. This is the most popular hierarchy used in course design and question writing. Bloom’s Taxonomy of Cognition is as follows:

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

The revised version of Bloom’s Taxonomy of cognition is provided by Anderson and Karthwohl (2001) which makes it pretty clear in terms of implementation. The revised Bloom’s Taxonomy of Cognition as provided by Anderson and Karthwohl is as follows:

• Create: Combining parts to make a new whole
• Evaluate: Judging the value of information or ideas
• Analyze: Breaking down information into component parts
• Apply: Applying the facts, rules, concepts, and ideas
• Understand: Understanding what the facts mean
• Remember: Recognizing and recalling facts

Designing question papers

Following is the list of suggested verbs mapped against each cognition level which can be used for writing questions intended to measure a particular cognition:

• Cognition Level:   Remember
  • Verbs to be used:   Define, repeat, record, list, recall, name, relate, underline.

• Cognition Level:   Understand
  • Verbs to be used: Translate, restate, discuss, recognize, express, identify, locate, report, review, tell.

• Cognition Level:   Apply
  • Verbs to be used:   Interpret, apply, employ, use, demonstrate, illustrate, operate.

• Cognition Level:   Analyze
  • Verbs to be used: Distinguished, analyze, differentiate, appraise, calculate, test, compare, contrast, criticize, debate, relate, solve, examine, categorize.

• Cognition Level:   Evaluate
  • Verbs to be used: Judge, appraise, evaluate, rate, compare, revise, assess, estimate.

• Cognition Level:   Create
  • Verbs to be used: Compose, plan, propose, design, formulate, assemble, collect, construct, create.

The revised Bloom’s Taxonomy and combinations of verbs listing provides an easy way out to relate each intellectual skills which can be tested through different question stems. A further elaboration on writing questions based on competencies to be assessed has been provided by Kate Exley (2010; revised 2012) as reproduced below.

Auditing question papers

A suggested checklist for auditing of question papers is provided here:

• What is the time budgeting of the question paper?
• What are the various levels of difficulty set in the question paper, to allow high standard and minimum standard to be achieved?
• What are the cognitive levels aligned in the question paper, to allow higher order thinking skills?
• What is the alignment of the question paper with the course teaching and content?
• What is the alignment of questions with the learning outcomes of the course?
• Is the language simple, clear, and unambiguous?