Assessing the Assessments

How we assess students in our course is diligently linked to how we teach the course. Assessment is a testimony to our teaching. Any assessment devoid of its link with ‘what to assess’, is likely to go haywire. Then comes the question of ‘how to assess.’ Our ability to link it with how we teach determines ‘assessment-of-learning’ vis-à-vis ‘assessment-for-learning.’ 

The end-term hall examination is about to start in the next 10-15 minutes. Many students have yet not taken entry into the hall. They are busy browsing through the notes. Some of them looking at books. Anxiety is visible on their faces. I really wonder: how does it help? One of my colleagues said it is their last hour’s effort to recall facts. Students opined that yes, it does help. I have also heard students saying that a good exam schedule is where there is a gap of a few days in-between. This allows them preparations time. Students have come forward on many occasions demanding a gap of more days between the two examinations. On the contrary, students have also been found blaming the non-availability of gaps for their poor performance in exams. Does it sound familiar?

This means that students do read specifically for examinations which is seemingly different from studying during the course. Blame it on the question paper design, which mostly tests students’ recall of facts, figures, and theories. This makes rote-learning important for passing the exam.

Teaching for exam versus teaching for learning

Rote-learning is an outcome of teaching style. When teaching is exam-centric, students have to be exam-focused. The scenario can be summarized in the following manner, from the teacher’s and student’s point of view:  

Why are we teaching—because students have to appear in the exam

and earn a degree!

Why are you attending classes—because we have to give exam

and pass it for obtaining a degree!

In the above scenario, the teacher focuses on finishing the syllabus, while students are interested in knowing what questions may be asked in the final examination. Even a cursory look at the question paper would reveal how we must have taught the course. Students are conscious of this fact and also get hurt when question papers are not designed properly to assess their learning. Too many questions starting with ‘what’ may require more fact-based recalling, while a question starting with ‘how’ and ‘why’ would require views, opinion, and critical appreciation. The combination of ‘what’, ‘how’, and ‘why’ used in the question paper would largely be linked with the teaching style adopted.

There are several underlying questions that can be as fundamental as to why do we want to assess, what we want to assess, and how do we want to assess. Arguably, answers to these questions would depend on the course learning outcomes and course objectives and it would differ across courses. Nevertheless, the overall contributions to the program-level outcomes and objectives need to be ascertained. Hence, the auditing part of question papers and answer scripts must provide insight into how the comprehensive written examinations have contributed to the achievement of course learning outcomes and the program learning outcomes. The outcome-based learning system makes a lot of difference and allows teachers to focus on students’ learning rather than merely passing the examinations. This also allows students to move away from rote learning, as it focuses more on the development of critical thinking and analytical ability. A comparison of the content-based learning system and outcome-based learning system is provided hereunder.

Objectives of Assessment

The overall aim of the exam and assessment is to facilitate the learning of important aspects in the course and to examine that the student has reached the minimum standard acceptable. It serves several useful purposes, some of them are listed below:

  • To identify and list students who are able to achieve minimum level of standard prescribed for the course and also to identify and list students who are not able to meet the standard. This is necessary for making individual intervention plan so that brilliant students are pushed further forward and slow learners are helped to stand above the minimum standards.
  • To extend feedback to students which help them improve learning. It is necessary that assessment tools become the learning source for students and hence feedback is extremely important. Feedback shall be oral as well as written as these two forms are not substitute rather they are complimentary. Written feedback shall be provided on the answer scripts itself so that students can see and use it for learning improvements. Answer scripts shall be shared with students and discussions to be made on questions. Few best answer scripts shall be shared with students.
  • To use hall examinations assessment as a means of encouragement. This is to be reiterated that nothing works better for students than seeing one’s progress and success. Examinations must give idea of progression to students. Faculty must encourage students and congratulate them on progress made.
  • To notice and encourage learning beyond curriculum. Students would tend to learn many things beyond the intended learning in the course. Faculty must identify some of the important beyond-course learning and encourage student to take it forward.
  • To felicitate students with extraordinary achievement and outcome. Awarding and recognizing students with extraordinary results essentially set high standard which also becomes a guideline for others to follow. Certificates/Prizes/Medals should be instituted for students achieving highest accomplishment.

 What learning is to be evaluated?

It is desirable that students not only remember facts and figures but more importantly they are able to present them in a perspective, analyze them, and provide interpretation. Students should be able to synthesize thinking and apply knowledge. In the context of employability, students must also be abreast with the real world and the current state of affairs in the economy and industry, especially how things are applied in the real world. In a model of holistic learning, it is important that both cognitive and non-cognitive competencies are assessed. Benjamin Bloom in the mid-1950s provided three domains of learning:

  • Cognitive, i.e. knowledge based
  • Affective, i.e. attitude based
  • Psychomotor, i.e. manual skills based

While preparing hall-examination questions, the above classification comes in very handy. The decision on the level of cognition to be tested can be helpful in selecting the kind of verb that can be used for writing questions. This is the most popular hierarchy used in course design and question writing. Bloom’s Taxonomy of Cognition is as follows:

  • Knowledge
  • Comprehension
  • Application
  • Analysis
  • Synthesis
  • Evaluation

The revised version of Bloom’s Taxonomy of cognition is provided by Anderson and Karthwohl (2001) which makes it pretty clear in terms of implementation. The revised Bloom’s Taxonomy of Cognition as provided by Anderson and Karthwohl is as follows:

  • Create: Combining parts to make a new whole
  • Evaluate: Judging the value of information or ideas
  • Analyze: Breaking down information into component parts
  • Apply: Applying the facts, rules, concepts, and ideas
  • Understand: Understanding what the facts mean
  • Remember: Recognizing and recalling facts

Designing question papers

Following is the list of suggested verbs mapped against each cognition level which can be used for writing questions intended to measure a particular cognition:

  • Cognition Level:   Remember
    • Verbs to be used:   Define, repeat, record, list, recall, name, relate, underline.
  • Cognition Level:   Understand
    • Verbs to be used: Translate, restate, discuss, recognize, express, identify, locate, report, review, tell.
  • Cognition Level:   Apply
    • Verbs to be used:   Interpret, apply, employ, use, demonstrate, illustrate, operate.
  • Cognition Level:   Analyze
    • Verbs to be used: Distinguished, analyze, differentiate, appraise, calculate, test, compare, contrast, criticize, debate, relate, solve, examine, categorize.
  • Cognition Level:   Evaluate
    • Verbs to be used: Judge, appraise, evaluate, rate, compare, revise, assess, estimate.
  • Cognition Level:   Create
    • Verbs to be used: Compose, plan, propose, design, formulate, assemble, collect, construct, create.

The revised Bloom’s Taxonomy and combinations of verbs listing provides an easy way out to relate each intellectual skills which can be tested through different question stems. A further elaboration on writing questions based on competencies to be assessed has been provided by Kate Exley (2010; revised 2012) as reproduced below.

(The above table is reproduced from Kate Exley (2010; revised 2012), Writing Good Exam Questions, LSGTM, London)

Auditing question papers

A suggested checklist for auditing of question papers is provided here:

  • What is the time budgeting of the question paper?
  • What are the various levels of difficulty set in the question paper, to allow high standard and minimum standard to be achieved?
  • What are the cognitive levels aligned in the question paper, to allow higher order thinking skills?
  • What is the alignment of the question paper with the course teaching and content?
  • What is the alignment of questions with the learning outcomes of the course?
  • Is the language simple, clear, and unambiguous?

Author: Dr Prabhat Pankaj

Dr. Prabhat Pankaj is a postgraduate in Economics and a Ph.D. in applied economics. He is a teacher by choice and started his career 30 years ago in 1991 from Arunachal University. He has been teaching Economics at postgraduate and undergraduate levels for about 30 years, in Universities and B-Schools in India and abroad, including 7 years in Bhutan. Dr. Pankaj has also obtained his Executive Education in "Management and Leadership in Higher Education" at Harvard University, Boston, USA. Furthermore, He has written for the Times of India and other popular publications. Currently, he is serving as the Director of Jaipuria Institute of Management, Jaipur.

13 thoughts

  1. Student assesment is a means for evaluating the knowledge and skill sets of students. My experience from teaching for over four decades is that in an hetrogenous group of students in a class, we teachers need to imbibe an attitude of love for learning. Learning has two sides of a coin, one is the assimilation of information and the other is the application of information in our day to day life. As a teacher of economics, I try to inculcate a blend of subject knowledge and its real world application. The evaluation of students should be focused on three fundamental testing skills namely, conceptual knowledge, application of knowledge and most of all,critical thinking. Any feedback must be relavant to these focused areas of learning. Examination is not the sole purpose for evaluating students. It is the employabilty of these students in the real world. What I have realized in the course of my career is that it is not the grades that get a student the job but rather it is the application of his knowledge and skills to the real world issues that matters most.

    Liked by 1 person

  2. Very insightful and thought provoking piece of work. It’s pertinent for the academicians to understand the application of pedagogical tools and assessment methods that will help students to become an active learner instead of passive learners. The introspection and innovation will automatically help students in enhancing their employability.

    Liked by 1 person

  3. Excellent article which questions the very foundation of our teaching, learning and evaluation process. As long as rote learning and memory based evaluation are the norm, we shall continue to have anamolies between academic and real life excellence. When I look around my bat mates in colleges and schools, I find innumerable cases where the average student has excelled in real life and toppers have not done so well! When I was a full time faculty at one point of my life in the 90s, I experimented with open book exam, unheard of during that time. Majority failed and the Principal questioned my unusual evaluation technique! Subsequent exams I was forced to follow the standard process!!

    Liked by 1 person

  4. Sir i found it well researched and insightful article. As an academician such things must be clear to faculty members but as per my experience such insights are rear to find. Sir this article will become a benchmark for me to evaluate my students and guide my faculty members. I hope you will share such knowledge nuggets in future too. Sir please also, guide how to evaluate diversity of students in big institutions where students belongs to every walk of life.
    .

    Liked by 1 person

  5. A very apt article Dr Pankaj
    It is an art to design an evaluation than encourages abd inspires students.
    Live projects based assignments in MBA curriculum have that potential.
    Looking forward to more insights on this.

    Liked by 1 person

    1. Thank you Prof Rajesh .. coming from you is indeed makes me so happy. Yes, happiness projects/assignments/workshops/interventions are so very critical in creating a positive mindset leading to happiness. We are running a full 3 credit course on happiness and offer it to our students.

      Like

Leave a Reply to Dr. Hari Prapan Sharma Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s